Duplicacy with Wasabi Nov 22, 2017

No, this is not a recipe for a meal…

I’d like to describe the backup structure now in place here at JeeLabs, which has been running for a few months. It’s working out nicely for a personal + family setup.

There are several moving parts involved:

So that also explains the title of this post.

There are many trade-offs & considerations involved. First of all, everything is now combined into a single server machine, and hence a single point of failure:

If either the ISP link or server break down:

There are exactly four services out there which this all depends upon:

Note that all data storage resides on at least 3 disks: 2 local HDs, managed with Btrfs, plus the Wasabi backup, which is in fact a historical archive to allow retrieving old data (Nextcloud also keeps old versions of the last 30 days for its file sync areas).

Every once in a while, several months apart, I also make some backups on BluRay disks, just to have stuff on physical media I can hold in my hands. Just in case…

The beauty of Nextcloud (and Dropbox), is that you can have fast local folder copies without ever worrying about the copying and sync side of it. But there’s much more to it than that: it makes for an absolutely effortless way to manage file exchanges between your own devices as well as with “shared links”, i.e. URLs, handed out to others. Nextcloud is well integrated across macOS, iOS, and Linux (and yes: also Windows and Android).

Another very useful feature is that you can turn on or off the syncing per (sub-) folder. This makes it easy to turn the Nextcloud server into a large-scale / long-term archive and permanent repository, especially when not all of it fits on your local machine.

Duplicacy has some interesting properties in this context. First of all, it can back up to a wide range of targets:

Furthermore, it deduplicates all the data it backs up. So you can move files around and rename them at will, without leading to duplicates getting stored on the backup.

Of course the crucial feature is that the data in the backup gets encrypted before being sent to a remote system. With Duplicacy, the encryption keys never leave the system on which you perform a backup or restore.

Best of all, the core engine of Duplicacy is open source (it’s on GitHub) and written in Go (effortless command-line tool deploys).

The Windows & MacOS GUI versions (i.e. wrappers) are available for a small yearly fee - making it more probable that it won’t end up as abandonware any day soon.

The name is a bit confusing, since there’s also Duplicity, the default backup tool used in Debian-/Ubuntu-based desktop installs.

All in all, this setup is definitely not for everyone. You have to go through a lot of little details to get all the pieces right, and to be honest, I’m still tweaking it at times.

But this new setup has been delightfully effective. Nextcloud delivers all the key benefits of cloud services with a private server, inside as well as outside the home. Backups are fully managed, with proper provisions for redundancy and archival (i.e. “out-of-sight”) storage for everything that tends to end up in digital form these days.

The proof is perhaps that I’m starting to move more files from my dev machine(s) to ~/nextcloud/ - i.e. the auto-synced folder. And because of that, a future laptop is very likely to need a much smaller internal disk.

Weblog © Jean-Claude Wippler. Generated by Hugo.