Add new blog post, Software Glue

1 files changed, 125 insertions, 0 deletions

diff --git a/posts/software_glue.mdwn b/posts/software_glue.mdwn
new file mode 100644
index 0000000..fa97fc8
--- /dev/null
+++ b/posts/software_glue.mdwn
@@ -0,0 +1,125 @@
+[[!meta title="Software Glue" ]]
+
+Take any software project, on its own its probably not very useful. First of
+all, you probably need a complier or interpreter, something to directly run the
+software, or convert the source form (preferred form for editing), to a form
+which can be run by the computer.
+
+In addition to this compiler or interpreter, it's very unusual to have software
+which does not use other software projects. This might require the availability
+of these other projects when compiling, or just at runtime.
+
+So say you write some software, the other bits of software that your users must
+have to build it (generate the useful form of the software, from the source
+form) are called build dependencies. Any bits of software that are required
+when your software is run are called runtime dependencies.
+
+This complexity can make trying to use software a bit difficult... You find
+some software on the web, it sounds good, so you download it. First of all, you
+need to satisfy all the build dependencies, and their dependencies, and so
+on... If you manage to make it this far, you can then actually compile/run the
+software. After this, you then need to install all the runtime dependencies,
+and their dependencies, and so on... before you can run the software.
+
+This is a rather offputting situation. Making modular software is good
+practice, but even adding one direct dependency can add many more indirect
+dependencies.
+
+Now there are systems to help with this, but unfortunately I don't think there
+is yet a perfect, or even good approach. The above description may make this
+seem easy to manage, but many of the systems around fall short.
+
+## Software Packages
+
+Software packages, or just packages for short is a term describing some
+software (normally a single software project), in some form (source, binary, or
+perhaps both), along with some metadata (information about the software, e.g.
+version or contributors).
+
+Packages are the key component of the (poor) solutions discussed below to the
+problem of distributing, and using software.
+
+## Debian Packages
+
+[Debian](https://www.debian.org/), "The universal operating system" uses
+packages (*.deb's). Debian packages are written as source packages, that can be
+built to create binary packages (one source package can make many binary
+packages). Debian packages are primarily distributed as binary packages (which
+means that the user does not have to install the build dependencies, or spend
+time building the package).
+
+Packaging the operating system from the bottom up has its advantages. This
+means that Debian can attempt to solve complex issues like bootstrapping
+(building all packages from scratch), reproducible builds (making sure
+the build process works exactly the same when the time, system name, or other
+irrelevant things are different).
+
+Using Debian's packages does have some disadvantages. They only work if you are
+installing the package into the operating system. This is quite a big deal,
+especially if you are not the owner of the system which you are using. You can
+also only install one version of a Debian package on your system. This means
+that for some software projects, there are different packages for different
+versions (normally different major versions) of the software.
+
+## npm Packages
+
+On the other end of the spectrum, you have package managers like
+[npm](https://www.npmjs.com/). This is a language specific package manager for
+the JavaScript language. It allows any user to install packages, and you can
+install one package several times on your system.
+
+However, npm has no concept of source packages, which means its difficult to
+ensure that the software you are using is secure, and that it does what it says
+it will. It is also of limited scope (although this is not necessarily bad).
+
+## Something better...
+
+I feel that there must be some middle ground between these two situations.
+Maybe involving, one, two, or more separate or interconnected bits of software
+that together can provide all the desirable properties.
+
+I think that language specific package managers are only currently good for
+development, when it comes to deployment, you often need something that can
+manage more of the system.
+
+Also, language specific package managers do not account for dependencies that
+cross language boundaries. This means that you cannot really reason about
+reproducible builds, or bootstrapping with a language specific package manager.
+
+On the other end of the scale, Debian binary packages are effectively just
+archives that you unpack in to the root directory. They assume absolute and
+relative paths, which makes them unsuitable for installing elsewhere (e.g. in a
+users home directory). This means that it is not possible to use them if you do
+not have root access on the system.
+
+## All is not yet lost...
+
+There are some signs of light in the darkness. [Debian's reproducible builds
+initiative](https://wiki.debian.org/ReproducibleBuilds) is progressing well. In
+the Debian way, this has ramifications for everyone, as an effort will be made
+to include any changes made in Debian, in the software projects themselves.
+
+I am also hearing more and more about package managers that seem to be in
+roughly the right spot. [Nix](https://nixos.org/nix/) and
+[Guix](https://www.gnu.org/software/guix/), although I have used neither both
+sound enticing, promising "atomic upgrades and rollbacks, side-by-side
+installation of multiple versions of a package, multi-user package management
+and easy setup of build environments" (from the Nix homepage). Although with
+great power comes great responsibility, performing security updates in Debian
+would probably be more complex if there could be multiple installations, of
+perhaps versions of an insecure piece of software on a system.
+
+Perhaps some semantic web technologies can play a part. URI's could prove
+useful as unique identifiers for software, and software versions. Basic package
+descriptions could be written in RDF, using URI's, allowing these to be used by
+multiple packaging systems (the ability to have sameAs properties in RDF might
+be useful).
+
+At the moment, I am working on Debian packages. I depend on these for most of
+my computers. Unfortunately, for some of the software projects I write, it is
+not really possible to just depend on Debian packages. For some I have managed
+to get by with git submodules, for others I have entered the insane world of
+shell scripts which just download the dependencies off the web, sometimes also
+using Bower and Grunt.
+
+Needless to say I am always on the look out for ways to improve this situation.