iTunes Without the iTunes Part

I have been archiving all of my purchased media for quite some time now, I like the flexibility of a digital copy, but also flexibility to view it how and where I choose (i.e. Plex).  That usually means a lot of disc ripping, and I have set up a decent system and workflow to do it.

Most of my movies come with a digital copy, usually on iTunes.  This is handy and useful, since I can stream to most of my devices that way.  Movies on iTunes sometimes have really good deals, and it’s tempting to grab something that way.  And once in a while, my physical media is damaged – sometimes incredibly slightly – and I am just unable to rip it.

Files contained in Apple’s digital library aren’t perfect though, since if I want to watch a movie at someone else’s place, it either means bringing my precious physical media (if I have it) or bringing a device to their place to watch it!  I can’t make a more highly compressed and smaller versions for those odd times I want lots of movie in little space.  It’s the inconvenience of DRM.

After a bunch of Google searches TuneFab M4V converter caught my attention.  They claim to be able to remove the DRM from my iTunes library and give me a high quality version I can use in more ways, including the above.  I thought I’d give it a try.  They offered either a free license (for high-volume bloggers) or a 30% off coupon for a review.  I’d probably review it anyway, so I’ll try for the coupon!

They offer the product on two plaftorms, Mac and Windows – separately licensed.  The first thing I tried was to download and install a version on my Mac.  When I ran it it came up with a message stating that High Sierra was not supported.  Since Mac OS has well moved on to Mojave as of this writing, I wonder if there’s a future in the Mac version of the app?  If you only have a Mac, be careful before purchasing this.  Apparently they’re looking into it, but for now… I hope you have Windows.

I downloaded and installed the Windows version.  The first thing I discovered when launching it was a big white screen like this:

Blank-ish screen
Yep, that’s most of it

You can’t see from the screenshot but the window is slightly larger than the available screen space, was not resizable and has its own non-standard window controls (close, minimize etc).  These window controls are glitchy and vanished after moving the window around a bit… the only way to exit the app was to Alt+F4 or right-click on the taskbar icon.

Their intention was to look cool and clean but honestly it causes some major issues.  (I have 3 monitors, so likely that was the problem.  I think they could fix it quickly by not centering the window across all monitors but only the active monitor).  TuneFab is not alone in falling into that trap, most Windows apps these days seem to want to blaze their own trail and make things look unique.

The options screen is very sparse as well – though it does have standard window decorations yay!  It’s not resizeable either.

2 options, it’s very roomy though.

Wait!  Don’t give up!

But I won’t be looking at this app all that often. I have a number of extremely valuable apps that are sadly pretty ugly.  So I’ll let the UI issues slide for now.  The TuneFab team is welcome to contact me to test some fixes for this.  The more important question is… does it work?

I click the very apparent “Add Files” button.  Ah, tells me I need to download a file through iTunes first before encoding.  Fair enough, it can’t work with what it doesn’t have.  I wonder if it would be possible through the iTunes scripting interface to get a list of movies in the library and trigger a download?  Still, another compromise I can understand and am willing to make.

So I downloaded a movie and clicked on “Add Files” again.  It tells me I need to have downloaded a movie.  But I just did?  Could they add a “Refresh list” button?  Eh whatever.  I restart the app and there it is, it shows up, along with other iTunes playlists listed in the sidebar.  At the bottom have appeared three buttons labeled “Add”, “Add All” and “Cancel”…

And… just as I was noticing this, my machine threw a BSOD.  Yikes.  Windows kinda makes it hard to cast the blame.  Is it a video driver?  Is it iTunes?  Is it this app?  All three together?  If it was Mac, it would almost certainly be the app, but I can’t exactly find out.

But BSODs happen.  We’re all friends and not looking to point fingers here, but try out an app.  I reboot and relaunch, select the movie and hit “Add”.  Nice, it comes up with video and codec information. 

Codec information
Looking good so far!

Off to the right (not shown) is a gear that gives you audio and subtitle information.  I had 4 tracks (AAC and AC3 in English and French) and all of the subtitles available.

Once I did all of that, I clicked “Convert”.  Since this is the trial version I only get the first 5 minutes, so let’s see.  It’s very quick on this machine (considerably faster than real time).  I’d guess it took about 20 seconds to run through 5 minutes of movie…  and boom the movie appears in the output folder!  Quality looks great and includes multiple audio tracks. 

I was able to take that file and run it through Handbrake for another resolution so yup the DRM is definitely gone!

The only option given in the Windows media player is to open a separate subtitle file (e.g. srt).  I ran CCExtractorGUI and the subtitles are definitely there.  I don’t use subtitles, but I know some of you really need them.

Value

Listed price is $49 USD, with a (temporary?) $5 discount, so $44 USD.  Is it worth this cost?

The polish on the app UI honestly isn’t great.  I expect more for an app this cost.  Maybe this is only an issue with the Windows version?  It appears they seem to be using Qt, and cross-platform toolkits tend to have these kind of issues.  I couldn’t test the Mac one so who knows?

But value for its functionality?  Let’s figure that out.

Well, if you figure a flexible backup of your existing iTunes movies is worth $11 each, it would pay for itself in 4 movies.  Or to look at it another way, if you can save $11 per movie by buying on iTunes instead of on disc, you’d break even in 4 movies.

Or, if you rip your movies, then maybe you can calculate the time it takes and average it out that way.  For me it takes almost 3 hours to rip and encode a Blu-Ray.  I don’t mind that much but this is much easier and faster – with the consideration that you still have to take the time to download a HD movie in advance.

So, it depends on your use and value of your time.  Personally, if it was $29 I wouldn’t even bother to calculate the time, it would be worth it.  Add another 10-20 bucks and it makes me stop and figure out things like this.

Wait.  If I get the 30% off coupon… (quick mental math).  DUDE.

Bottom line:

Once I got it installed and running, it was really impressive.  Very fast with excellent results.  The UI is very buggy but the engine works great.  If you want a DRM-free copy of your iTunes movies, this may be the way to get them.

Managing Encrypted files on Amazon Cloud Drive

I have implemented a file system on Amazon Cloud Drive for a lot of media with the great acd_cli.  To protect my privacy, I have run this through an encryption layer encfs.  My writeup will follow.

A problem I was trying to solve in my mind though, is how to manage – rename and delete files once they’re all scrambled up and I can’t discover even the path and filenames.

Ultimately this would be seamless.  Delete a local file stub and it traces back to the encrypted remote file, but it doesn’t quite work that way.  I discovered how to do this on my Linux host.

Once I realized that the filesystem for encfs has the same inode numbers for the encrypted and decrypted files, I had a clue.  First, let’s find out what that file number is:

149 is the part we want.  inode numbers are unique per partition/filesystem, and seems to persist between the encfs pairs.  Now, to find a file in the encrypted path system with inode 149… find to the rescue!

I won’t even try to copy/obfuscate the number above.  Try it if you want to see it.  It would be almost impossible to track that file without the number.  Size and date are much harder to nail down the exact file.

So, to stitch these two together first you want the inode number only:

Now this is something we can use in a delicious Linux command chain.

This is easy enough to make into a little bash script, and allow passing arguments and quoting to protect against embedded spaces, as well as including the explicit Amazon Cloud Drive working area:

Works great for specific files, not so much for directories.  You would have to change the ls command to use a -ldi parameter just for those cases.

Now that we have the filename, we can manually delete that filename on Amazon, either through the web interface or using acd_cli’s command line trash argument.

Clone a Clone

So I had yet another WD MyBook die on me a couple of days ago.  And I still went out and bought another one (what was Einstein’s definition of insanity, again?)… This one was only two years old but these things are still quite cheap and very convenient to get.  Maybe someday when I have more money I will get a proper NAS enclosure.  For now, my pattern is to buy a new one every year.  They’ve almost doubled in size every time, so I can just clone everything to the new one and go from there.

Since I had such great experience with my WD MyBook Live, I decided to get the next version, a MyCloud.  This is slightly improved, similar in appearance, also with a GB network port but this one also has a USB 3.0 host on it, so I can buy another, regular drive next and slave it off of this one.

My previous scheme didn’t seem to work well with this one, though.  I was unable to make an unattended rsync to push from that drive to this one because these drives are set up to use root as the ssh user, and it’s not set up to use PermitRootLogin without-password .  It always seems to prompt for the password, which won’t work when AFK.

Until I figure that one out, I looked for another solution.  The coolest discovery was that these drives are actually running Debian.  After some research I found out that lftp will mirror a remote directory over FTP.  Of course, lftp is not installed on these drives, however, after running…

I had them installed (on both the 2012 and 2015 vintage drives).

Next was the task of setting them up!  I found a good post on StackExchange (well, ServerFault) here that helped a lot.  I ended up using this:

With that I had some options I could use by uncommenting DELETE or TESTMODE, for example.  One additional gotcha is that it doesn’t retain the source’s ownership, but since this is such a basic setup, I just chown all the files in the LCD variable to my username.

The password for that user is in cleartext, in the file.  If you are not comfortable with that, do not use this.

It still doesn’t seem to be running in cron yet, but I need time to experiment some more.  I still much prefer the SSH method but I want to get it working reliably and repeatably.  I need to reimplement much of this each time the firmware gets updated, and copying a few files is much better than having to edit sshd config files each time.

More hack attempts

After my last experience, I checked my logs and noticed quite a load of failed attempts on my mail server.  It looks like a brute force script kiddie attack, which I’m pretty sure will fail on my machine.

Still, I want to kick out these morons.  So after some research, I found fail2ban.  The installation was simple enough, and with a little bit of configuration (in jail.local, not jail.conf!) I had it up and running… but the attacks continued.

I wrote a simple perl one-liner to parse out all of the failed login attempts, run them through sort and uniq to get the repeat offenders (twice is enough, kids) and append that to the hosts.deny.  That worked, but not ideal.  I’d rather have iptables-level blocking (using DROP instead of REJECT to waste as much of their time as possible).  But fail2ban wasn’t catching them for some reason.

I set up a secondary rule and it still failed – until I discovered fail2ban-regex! With that command you can test your rules at any time instead of waiting for the next attempt to come in.  Great!  It turns out the regex wasn’t quite right for the messages I was getting.  I simplified the regex until it caught the failures.  But it still wasn’t working live.  Grr.

fail2ban works on log files.  It scans for repeated attempts to determine if there’s an attack going on.  This would work great unless the logging daemon compresses the messages with something like “last message repeated x times”.  And this happens a lot, especially when under attack and you actually need it!  You can not turn this feature off with sysklogd.  The last key was to replace sysklogd with syslog-ng and POW, the banstick came out to play.

Debugging wasn’t very easy, because the failures are silent.  Until I found fail2ban-regex I had about 4-8 hours between tweaks to the regex to see if it worked.

At least now I have a self-setting ban trap that uses iptables-level blocking.

If you’re reading this and you’re learning to be a script kiddie,  you are learning to be a loser.  You are creating nothing of value.  You could vanish from the Internet and not only would it become a better place, but the situation would improve.  Is that really what you want?  Instead, why not keep on learning about security but do this the right way, on your own machine or a VM and learn to strengthen the Internet, not ruin it.  You might actually be actually appreciated and valued by others on the net.

Hack Attack

Someone mentioned they got a bounce from my domain’s email. I went to take a look at the error and discovered a couple of hosts trying to brute force login to my SMTP server. Some quick config changes to create a blacklist, and a fail2ban install and it has stopped now.

Lesson 1: check your logs often
Lesson 2: use SASL
Lesson 3: use complex and random passwords
Lesson 4: install and configure fail2ban or blacklist the bozos with iptables or hosts.deny or something.

I got most of these right the first try, especially the middle two.

Eternal vigilance, they say…

Canada domain name registry scam

Domain Scam
Shred this letter immediately.

I have been frustrated with this  Domain Scam for a long time now.

I have a few domain names, and I happen to be in Canada.  There is a company called “Domain Registry of Canada” that mails out official-looking envelopes (it looks like a government-issue brown windowed envelope) to everyone that has WHOIS information indicating they live in Canada.  This is an example of the letter they send.

Unless you read it quite carefully, and know what is going on, you might think you need to pay their (very expensive) domain registration fees in order to avoid losing your domain name.

This is NOT TRUE.  Consider that many people, like myself, purchase hosting from a company like 1 & 1.  Part of the package includes free domain registration for one domain.  There is very little technical know-how required to get this going.  In fact, it could be that some hotshot young web developer has set this up for you.

You need to do nothing except keep paying your web hosting amount to safely retain your domain name.  This letter conveniently omits this fact.  They do make this somewhat clear in ALL CAPS halfway through the letter, but only after the scare tactics a couple of  paragraphs above.

Is it strictly a scam?  No, I guess not, they do provide a service, and they spell out everything in this letter, but it’s very dirty.

To make this abundantly clear: There is NEVER any reason to do anything except shred this letter. 

For more information, feel free to Google “Domain Registry of Canada” and look at any link that is not their official web page (i.e. start at the second link).  Here’s a link to make it even easier.  You will find many other bloggers, most more capable than myself, that explain this quite well.

Great VPS Experience

I have been using shared hosting from 1 & 1 for a few years now. No big complaints (except their domain management page is terrible) but I needed a bit more flexibility. I suppose shared hosting with its inherent limitations helps you develop a certain way, but I wanted a bit more.

For one, I wanted a much more flexible IMAP server setup. I was getting something like 100 mailboxes at 2GB each, but what if I wanted to totally move off of gmail and use 5GB? Setting up one or two archive accounts seemed… Messy.

So I did some research and found lowendbox.com. It is a bit intimidating at first because you don’t know what to look up, or what questions to ask, like “what is OpenVZ, and does it matter for what I want to do?” Or “what happens when I lock myself out?” (Notice, that’s when, not if). “What about if I want to wipe it out and start fresh?”. “Do I need one of these control panel products?”. I have those answers now, if you want. 😀

Well, I didn’t have any very clear answers to any of these but I decided to try it out, with a great special from LEB for $56/year for a 2GB machine from ServerMania.

This. Stuff. Is. Cool.

If you have never done any admin work on Linux, set up your own server or anything, stay away. You will get little value out of it and very likely get hacked. But if you are confident with your admin and security chops, this is a crazy bargain. I chose Debian 6 because Debian. I have a long history with that.

I spent a good day figuring out, testing and setting up the firewall, and learned an amazing amount. I tightened up SSH, installed some IDS stuff and I was happy, even though the thing was hardly useful yet!

I used git during the setup so I would have some ability to track and undo dumb changes. I should have started this earlier, I know it!

First big task was IMAP and SMTP. I got that rolling and made sure that spammers couldn’t hitch a free ride. I got a free SSL certificate from StartSSL and it does the job. Test like crazy.

Next I got Apache rolling and a database server, and made sure those aren’t vulnerable either.

Finally, once I was happy, I pointed my domain to freedns.ws and configured its records. Let me stop for a second and state that of all the stuff I have done so far, FreeDNS.ws amazes me the most. I can’t see why anyone would pay for DNS server services when this thing is around (ok I notice certain record types are missing. I didn’t really need them though).

While waiting for the DNS switchover to happen (always an interminable wait) I discovered mydnscheck.com. A great tool to see how things are progressing and what you have set up or have forgotten to set up!

So far that’s it. I haven’t cancelled my shared hosting yet but I’m moving everything off it. I think I’ve outgrown it… Or more likely it just doesn’t suit me, as I never really used it to it’s potential.

Now I might even run a personal Minecraft server on my VPS, not sure yet. But hey, I can.

It was a lot of work, but a lot of fun. And I am happy in knowing there’s a little piece of the Net that I put together all by myself.

Backbone dynamic elements

I recently discovered Backbone’s setElement function. What it allowed me to do was dynamically create a view’s container in Javascript and transfer/assign the view and its events to the new element.

Using jQuery within the view initialize event:

I manipulated the this.el attribute directly, without using an interim variable, it seemed to work ok, but if there are side-effects I discover later, it should be easy enough to change. setElement will automatically assign its argument to this.el so nothing further needs to be done.

iTunes (Windows) Podcast fix

iTunes is pretty decent for Podcasts. In fact, Lifehacker readers rated it the best.  However, it has some shortcomings, to be sure.

The one that annoyed me the most is if you have a RSS feed of say, PDFs or ePubs like the magazines at jw.org, you can subscribe in iTunes whereby it will happily auto-download (and sync) them for about 3 or 4 months and then silently stop.  It expires those subscriptions because it thinks you haven’t “listened” to them for that long, so you must not be interested in them anymore.  Reading an ebook, even when synced to iTunes does not update the play count, so you have to manually re-enable them.  There is no way to disable this feature, or adjust the expiration interval.

On the Mac, you can download “Update Expired Podcasts” from the wonderful Doug’s AppleScripts site and schedule that, but on Windows, a little more complicated.

Will Perry posted a couple of decent scripts (v1 here and v2 here) that gave me an excellent start, but I ran into two big problems.  One was that it kept throwing an error implying that the track was not an object, so item.podcast was invalid (?). Another problem was that once I found the item, it refused to update it, claiming it was not modifiable.  I think something in my library must be corrupted or the metadata is incomplete… or something.  In any case, I took a hybrid of those two and added a dumb “on error resume next” statement and came up with something that works.

I also added a simple condition to update only the eBook formats, but leave the audio podcasts alone.  If you want, you can remove that condition.  Note that updating the file in this way does not affect the “Read” status – so you can still keep track of what you have read or not read!

Here it is.

REST Authentication

I did some research the other day to secure my REST API using The Slim framework.

I found a tidy little way to force HTTP authentication (basic, in this case) using this article as well as the PHP manual.

I get the client to provide the user name and password, then I can look up the (hashed) password in the database. It simply causes the call to authenticate each login with their corresponding password.

In combination with forcing the page through https (a .htaccess task) and this, I can protect API access pretty simply.

I used this in VB6 (and the great Chilkat components) to authenticate a PUT command. I can now more safely allow database inserts from over the Net. It was as simple as:

Next task is to find a more secure way to access the (read-only) API from JavaScript, without just embedding a password in the source code.  This looks like a great start.