Rescuing Encrypted files on ACD

So Amazon is shutting out Linux users.  But what if I have a bunch of encrypted files there using old encfs and acd_cli scripts?

I can copy down the encrypted files using their client at any point, but how will I know which one is which?

I did the following.  First, create a temporary directory.  I did this in my $HOME on my Mac.  Find a way that still exists to mount the drive (I used ExpanDrive).  Once that is prepared, change to the mounted and encrypted ACD directory and run this command:

Let it run for a while, it may take several minutes.  This will create in $HOME/temp the identical directory structure as on the remote drive, and the identical filenames – but they will all be zero bytes!  What good is this?

Thanks to the consistency of encfs, you can mount and decrypt this skeleton directory like this:

Now, use some other tricks to find the matching filenames and you can manually download the specific encrypted files you want.

Amazon Drive Shutouts

Amazon is in the process of revoking API keys for a number of apps used to access their Cloud Drive product.  They started with acd_cli and have now banned rclone, and apparently Stablebit CloudDrive.  As more users of these products filter into fewer and fewer available options, the API hit from those products will certainly increase, possibly leading to more bans.

Why?

Amazon is theoretically overselling cloud drive capacity.  They call their service “Unlimited” but are counting on the fact that no sane person will upload more than a few hundred gigabytes of data.  Using their client, yes, that would be true.  (To be sure, their client has improved considerably in the last 6 months or so).  The above clients, though, allow you to mount your remote drive like any regular network-attached drive, and work directly off it.  With that ability, it is worth putting terabytes of files on a cloud drive and freeing up local resources.  Some files, like video files, are ideal for this application.

Some very sane people started storing 5, 10, 50 and more terabytes on their service.  Can you imagine the accountants panicking?

But I said “theoretically” overselling.  Is that true?  Are these cloud drives really losing money by allowing large amounts of storage?  Let’s calculate the cost of storage, in a grossly oversimplified calculation.

I looked up the price of a 2 TB hard drive this morning.  It was on sale for $89.  If I wanted 2 TB of cloud storage for one year, at that price it should cost ($89/12) $7.42 per month.  Do you know any cloud services that offer that price?   Amazon’s Cloud Drive is $60/year for “unlimited” space.  Based on the price of this drive (and I know this bears almost no parallel with reality), they are expecting to split the purchase cost of this drive between ($89/5) 17.8 people, who each store a maximum of (2000/17.8) 112.3 GB each.  Would you pay $5 a month to store only 112 GB of data?  Me neither.  Amazon naturally doesn’t expect that either, so it likely means their cost is far, far cheaper than that.

Where’s the “value-line” for you at that $5/month price though?  500GB?  1TB?  2TB?  I’m guessing it’s somewhere around the 1TB mark, with options for a little more.  (Incidentally, that’s kinda the price of Office365, including 1TB of storage and, oh yes, a full office suite thrown in for “free”).  Would you pay, oh, $7.42 a month for that capacity?  Wait, that is totally coincidentally the price of a 2 TB hard drive!  And you get to keep it after a year, get a second one for replacement or expansion, every. single. year.

Their business is NOT based around cheap, small hard drives at retail prices.  They run a staggeringly huge data warehouse with storage pools.  Their pricing is a fraction of what ours are, allowing them to pay for the rest of the overhead with attaching that drive to the Internet.  They can likely afford a few hundred outliers.

So it’s not just about storage space.  It’s about reliability, availability, and convenience.  Kinda those things you lose when you shut out third party apps eh, Amazon?

So if I can’t effectively store my video backups up there, and if they close off more of their API so I can’t use Arq for backups… my practical use for ACD drops way, way down below my “value-line”.  It may be time to migrate.