I AM A FUCKING SYSADMIN

Don’t just apt-get install. Look before you leap.

Two words. Package versions.

Lately at my office, we’ve been iterating to a new version of our internal web hosting infrastructure, moving from, amongst other things, Ubuntu 10.04 LTS to Ubuntu 12.04 LTS. We use Ubuntu because we prefer Debian-style Linux, but with a commercial backing.

Canonical offers a certain guarantee of stability and security fixes within a certain time period, for a certain number of years, which Debian does not and cannot offer. Great! (Before I get a bunch of angry replies, I’m not claiming that Debian is unstable or insecure. That’s not my point.)

We also use the LTS releases specifically, because they are considered to be more stable, and receive “long term support”.

However, recently I’ve noticed that Canonical has made some rather questionable choices regarding to packaging. Specifically, the versions of packages that are included.

Example 1: nginx

First, it’s worth nothing that this incredibly popular package isn’t in Main but in Universe. Why is this package, a mainstay of so many web developers’, web hosts’ and sysadmins’ infrastructures not in Main? Who knows!

Secondly, the version. The version of nginx in the Ubuntu Universe repository is 1.1.19. A dev release! What’s more, a dev release with a known segfault when using the fairly common try_files parameter.

The bug has been fixed, but not in 1.1.19.

This bug might not affect you, but it does tell you one thing. nginx 1.1.19 is not to be trusted. And that’s just fine, because it’s a dev release. But if you’re running Ubuntu, you’re running 1.1.19.

Thankfully, there is an nginx-provided apt repository from which you can procure a stable version. Otherwise we’d be stuck compiling our own copies and building packages.

DEV MEANS DEV. Don’t put dev packages in your distro’s repository when there is a reasonable, stable alternative.

Example 2: The php memcache module.

Ubuntu provides version 3.0.6 of this package. An unstable release! Now it is true that at the time Ubuntu 12.04 was released, the last stable release had been in 2010. In that light, it seems somewhat reasonable to use the newer 3.0.6 beta release from 2011, right? Wrong!

Why? Because a critical piece of functionality is broken in the 3.0.6 release!

Failover!

Oh, you can still specify multiple servers in your php.ini file, and you can stick the parameter that enables failover in there, but when you test it (or worse yet, when the first listed memcached server fails in production), nothing happens! You’re down.

Anybody with any type of medium-to-large sized infrastructure needs failover. Heck even the small fries don’t want their services to go down.

Thankfully, the memcache module is available via PECL, or we would have to deal with building and packaging it on our own.

BETA MEANS BETA. Don’t put beta packages in your distro’s repository when there is a reasonable, stable alternative.

To be fair, these are both Universe packages. Yeah, I get it. Canonical doesn’t maintain universe. 

That doesn’t matter.

This kind of stuff might be fine for 11.10, or 12.10, but not for an LTS.

Obviously there’s no excuse for us as systems administrators to not be auditing the versions of packages we’re using. We do! That’s why we caught these problems in testing, and not in production.

Regardless, we need to be able to expect that the package maintainers in Universe make sane choices, even if they aren’t Canonical employees. Not choosing to package the dev or beta version of an application, module, or plug-in is the obvious sane choice. It’s as simple as that. They may not be Canonical employees, but Universe contributors still work under MOTU overseers that are meant to guide them in good packaging practices. There’s been a breakdown somewhere.

I’m not sure if this is something worth leaving Ubuntu over, but at a bare minimum it’s a good reminder that we should not just “apt-get install and forget”. Examine what you’re doing to your systems.