Currently on the front page of the JeeLabs shop:

The benefit of doing everything yourself, is that you can make things work exactly as you want them.

The drawback of doing everything yourself, is that you have to do everything yourself…

Having become pretty independent in my work areas, my hobbies, and my income streams over the years, I know all about those trade-offs. Or at least I think I know about most aspects of this DIY-vs-outsourcing range.

It’s a bit like trying to stay on your feet with a floor covered with marbles…

Example: I used to rent a web server (a real physical one, with full root access and Linux on it). No worries about hardware outages or connectivity details. Being housed at an ISP with thousands of servers, means they’ll have round-the-clock watchdogs and support staff, and will jump into action the minute something is seriously wrong.

At the same time, I had total control over the web server software and operating system configuration. With a Linux distribution such as Debian, maintenance was delightfully simple (“apt-get update && apt-get upgrade”).

The flip side is that I had to choose and configure a web server (“lighty” / lighttpd at the time), and technologies to create dynamic database-driven websites (I built my own back then, based on Metakit – my own database).

Did it work? Sure. Did it evolve? Nope. Too busy. Didn’t want to risk breaking anything.

Only thing that setup did was track security updates (automatically). I had two break-ins over the 10 years that this went on. Learned more about rootkits than I care about (they’re evolving to amazingly sophisticated levels).

Did I learn a lot? You bet. And some of that knowledge is priceless and timeless. Big, big benefit.

But I also had to learn lots of stuff I really care very little about. For me, network routing, package installation dependencies, mail server configuration, and lighttpd configuration were a waste of time. The latter because lighttpd wasn’t really kept up to date very actively by its developer(s). Other options became more practical, meaning that all that lighttpd-specific knowledge is now useless to me.

The story is repeating itself right now. Redmine, which I use on https://jeelabs.net/ is not up to date, because I haven’t found a simple upgrade path. The difference is that it’s not just me not updating my stuff, I now have the same stagnant state with stuff from others. So what’s the point of Redmine? As far as I’m concerned, it’s a dead end (luckily, everything in there is stored in Markdown format – a solid long-term standard which I also use for the forum and the weblog).

With the forum, running on Drupal, it’s different again. Module updates are automated more or less, so I tend to track them from time to time. But Drupal itself is a little harder to update. And sure enough, it’s falling behind… With Drupal, I’m also running into not being knowledgeable enough to put it to really good use.

But the reason for writing this post is a different one – see the message at the top.

For the web shop, I use the Shopify web store service. They have the servers (at Rackspace – very good ops, I’ve used them for a couple of years). And Shopify develop and run the web store software (using Ruby on Rails).

They take care of dealing with nasty things such as possible DoS attacks, heavy data security, financial gateway interfaces – lots of important issues I no longer need to worry about. So far so good.

But they have their own agenda:

• some things don’t change, and that’s good: it works, the shop is operational
• some things don’t change, but that’s bad: years have gone by, and they still haven’t got a clue about VAT
• some things change, and that’s good: improvements to the service, new features for customers
• some things change, but that’s bad: they change their API and their XML data structures

That last one is what bites me now. I created a little scripted setup whereby I always pull information about orders from their shop database, to fill my database here with all the details, so I can generate paper invoices, and do the fulfillment of orders here. Doing this here was necessary to be able to do the Value Added Tax thing properly, as required by law and as my accountant wants it, of course.

So to summarize, the choices are:

1. do everything yourself (and pay in time)
2. outsource everything (and pay in money)
3. choose a mix (and deal with the interface changes)

Everything is a trade-off, of course. In my case, I’m moving more and more to #1 as far as operational choices are concerned (own server, own fiber connection), and #2 as far as day-to-day software use is concerned (solid, but actively developed open source software, and Apple hardware + Mac OSX for my main workplaces). These choices are optimal for me, in terms of cost and stability.

The choice to host my own servers was made a lot simpler because I’m running VM’s for the different sites, built from ready-to-run images from TurnKey Linux. What makes them (and others, like Bitnami) different, is that all VMs are automatically backed up to the cloud (Amazon S3 in my case). The way TKL does this is really clever, reducing the amount of data in incremental backups, even for all the records stored in MySQL. So not only are my VM’s pre-configured and run out of the box, they automatically self-update and they automatically self-backup – if anything goes completely wrong, I can switch to cloud-based instances and be up and running again in no time.

TurnKey Linux is an example of using third-party stuff to side-step (and in fact avoid) a massive amount of effort, while retaining maximum operational flexibiity. My Amazon S3 bill is a whopping $1.01 per month…

But the web shop setup at Shopify is far from optimal. It was supposed to be choice #2, but ended up being #3 due to the mismatch between what I need (a European shop with correct VAT handling) and what they offer (flashy stuff, aimed at the masses). In hindsight, it was a bad choice, but I really don’t want to do this myself.

Oh well, I’ll suffer the consequences – will fix my scripts and get everything going again by next Monday!

PS. My little presentation yesterday at HackersNL #5 can be found here (PDF) – for those who read Dutch.