Rescuing Encrypted files on ACD

So Amazon is shutting out Linux users.  But what if I have a bunch of encrypted files there using old encfs and acd_cli scripts?

I can copy down the encrypted files using their client at any point, but how will I know which one is which?

I did the following.  First, create a temporary directory.  I did this in my $HOME on my Mac.  Find a way that still exists to mount the drive (I used ExpanDrive).  Once that is prepared, change to the mounted and encrypted ACD directory and run this command:

Let it run for a while, it may take several minutes.  This will create in $HOME/temp the identical directory structure as on the remote drive, and the identical filenames – but they will all be zero bytes!  What good is this?

Thanks to the consistency of encfs, you can mount and decrypt this skeleton directory like this:

Now, use some other tricks to find the matching filenames and you can manually download the specific encrypted files you want.

Amazon Drive Shutouts

Amazon is in the process of revoking API keys for a number of apps used to access their Cloud Drive product.  They started with acd_cli and have now banned rclone, and apparently Stablebit CloudDrive.  As more users of these products filter into fewer and fewer available options, the API hit from those products will certainly increase, possibly leading to more bans.

Why?

Amazon is theoretically overselling cloud drive capacity.  They call their service “Unlimited” but are counting on the fact that no sane person will upload more than a few hundred gigabytes of data.  Using their client, yes, that would be true.  (To be sure, their client has improved considerably in the last 6 months or so).  The above clients, though, allow you to mount your remote drive like any regular network-attached drive, and work directly off it.  With that ability, it is worth putting terabytes of files on a cloud drive and freeing up local resources.  Some files, like video files, are ideal for this application.

Some very sane people started storing 5, 10, 50 and more terabytes on their service.  Can you imagine the accountants panicking?

But I said “theoretically” overselling.  Is that true?  Are these cloud drives really losing money by allowing large amounts of storage?  Let’s calculate the cost of storage, in a grossly oversimplified calculation.

I looked up the price of a 2 TB hard drive this morning.  It was on sale for $89.  If I wanted 2 TB of cloud storage for one year, at that price it should cost ($89/12) $7.42 per month.  Do you know any cloud services that offer that price?   Amazon’s Cloud Drive is $60/year for “unlimited” space.  Based on the price of this drive (and I know this bears almost no parallel with reality), they are expecting to split the purchase cost of this drive between ($89/5) 17.8 people, who each store a maximum of (2000/17.8) 112.3 GB each.  Would you pay $5 a month to store only 112 GB of data?  Me neither.  Amazon naturally doesn’t expect that either, so it likely means their cost is far, far cheaper than that.

Where’s the “value-line” for you at that $5/month price though?  500GB?  1TB?  2TB?  I’m guessing it’s somewhere around the 1TB mark, with options for a little more.  (Incidentally, that’s kinda the price of Office365, including 1TB of storage and, oh yes, a full office suite thrown in for “free”).  Would you pay, oh, $7.42 a month for that capacity?  Wait, that is totally coincidentally the price of a 2 TB hard drive!  And you get to keep it after a year, get a second one for replacement or expansion, every. single. year.

Their business is NOT based around cheap, small hard drives at retail prices.  They run a staggeringly huge data warehouse with storage pools.  Their pricing is a fraction of what ours are, allowing them to pay for the rest of the overhead with attaching that drive to the Internet.  They can likely afford a few hundred outliers.

So it’s not just about storage space.  It’s about reliability, availability, and convenience.  Kinda those things you lose when you shut out third party apps eh, Amazon?

So if I can’t effectively store my video backups up there, and if they close off more of their API so I can’t use Arq for backups… my practical use for ACD drops way, way down below my “value-line”.  It may be time to migrate.

A Dip in the Music Stream

Music streaming has become a big deal in the last few years.  People don’t seem to care about bandwidth, they know its a resource that seems to be basically infinite (if metered).

My problem is that it almost seems like the users are treating music the same way.

I reflect back to my discovery of music.  It started off as a social thing – you wanted to hear the new stuff that other people were talking about.  These were the days of carrying a boombox around so you could play your tunes for everyone who surely would like it as much as you, right?

After not long though, I realized there were things that others liked that I didn’t.  Why?  If it was popular, wasn’t it automatically good?  I realized that music can be something very personal, you can find some lyrics or tones that speak to your soul, but not those of the people right around you.  And, you know what, that cut both ways.  Maybe music I really enjoyed was never going to be the favourite of others 1.  I think I am still learning this, actually.

This is not to say that music lost its social aspect, sometimes it defined it.  This may have been an operation of the teen years as well, but there were times I selected or prioritized friends based on their music tastes.  I want to reject this as immature, but perhaps there is some small bit of validity to this…

Nevertheless, over this time, I began curating my own collection.  Thanks to Columbia House and BMG I gained a stack of CDs (vinyl was from record stores).  Much of it was familiar but I could afford to experiment a bit.

Music was tangible.  Music had a measurable value.  When I got a physical CD, it was an investment.  Sometimes I didn’t really like it on first listen, but since I paid for it, and I only had so many of them, I’d better get into it.  Sometimes I loved it from the first listen – yet I would usually limit myself to one listening a day, so I didn’t get tired of it.  Leaving it running all the time was absurd and wasteful.

To me, digital music downloads were the same thing, the lack of a physical disc didn’t cause a disconnect between the music and “some thing of value”.

This finally leads back to streaming.  I have tried a few services.  I ran Rdio for a long time, it was quite good, and I discovered some new artists I really enjoyed.  Then I tried Spotify for a month, it was also good, and I occasionally used some of the stations.  I ended up getting Xbox Music Pass for a year.  I tried Apple Music, and Google Play Music.  Back to Spotify for a few months.

But sometimes when I listen to stuff on any of these streaming services, I can’t shake the feeling that lots of this music isn’t worth much, and that has absolutely nothing to do with the compositions or the performance.  It seems very temporary, and that bothers me a little.  Why am I wasting my precious listening time on “throwaway” music?

I’m slowly coming around to the value of streaming.  I have discovered a few outstanding bands and songs that are valuable to me and exist only in that context.  But like air and clean water, maybe it’s something we have to stop ourselves and think about their value.

In the meantime, I’m gonna dial up some Big Country and Flock of Seagulls.  Enjoy, my public.

  1. I know, seriously?