diff options
| -rw-r--r-- | posts/software_glue.mdwn | 125 |
1 files changed, 125 insertions, 0 deletions
diff --git a/posts/software_glue.mdwn b/posts/software_glue.mdwn new file mode 100644 index 0000000..fa97fc8 --- /dev/null +++ b/posts/software_glue.mdwn @@ -0,0 +1,125 @@ +[[!meta title="Software Glue" ]] + +Take any software project, on its own its probably not very useful. First of +all, you probably need a complier or interpreter, something to directly run the +software, or convert the source form (preferred form for editing), to a form +which can be run by the computer. + +In addition to this compiler or interpreter, it's very unusual to have software +which does not use other software projects. This might require the availability +of these other projects when compiling, or just at runtime. + +So say you write some software, the other bits of software that your users must +have to build it (generate the useful form of the software, from the source +form) are called build dependencies. Any bits of software that are required +when your software is run are called runtime dependencies. + +This complexity can make trying to use software a bit difficult... You find +some software on the web, it sounds good, so you download it. First of all, you +need to satisfy all the build dependencies, and their dependencies, and so +on... If you manage to make it this far, you can then actually compile/run the +software. After this, you then need to install all the runtime dependencies, +and their dependencies, and so on... before you can run the software. + +This is a rather offputting situation. Making modular software is good +practice, but even adding one direct dependency can add many more indirect +dependencies. + +Now there are systems to help with this, but unfortunately I don't think there +is yet a perfect, or even good approach. The above description may make this +seem easy to manage, but many of the systems around fall short. + +## Software Packages + +Software packages, or just packages for short is a term describing some +software (normally a single software project), in some form (source, binary, or +perhaps both), along with some metadata (information about the software, e.g. +version or contributors). + +Packages are the key component of the (poor) solutions discussed below to the +problem of distributing, and using software. + +## Debian Packages + +[Debian](https://www.debian.org/), "The universal operating system" uses +packages (*.deb's). Debian packages are written as source packages, that can be +built to create binary packages (one source package can make many binary +packages). Debian packages are primarily distributed as binary packages (which +means that the user does not have to install the build dependencies, or spend +time building the package). + +Packaging the operating system from the bottom up has its advantages. This +means that Debian can attempt to solve complex issues like bootstrapping +(building all packages from scratch), reproducible builds (making sure +the build process works exactly the same when the time, system name, or other +irrelevant things are different). + +Using Debian's packages does have some disadvantages. They only work if you are +installing the package into the operating system. This is quite a big deal, +especially if you are not the owner of the system which you are using. You can +also only install one version of a Debian package on your system. This means +that for some software projects, there are different packages for different +versions (normally different major versions) of the software. + +## npm Packages + +On the other end of the spectrum, you have package managers like +[npm](https://www.npmjs.com/). This is a language specific package manager for +the JavaScript language. It allows any user to install packages, and you can +install one package several times on your system. + +However, npm has no concept of source packages, which means its difficult to +ensure that the software you are using is secure, and that it does what it says +it will. It is also of limited scope (although this is not necessarily bad). + +## Something better... + +I feel that there must be some middle ground between these two situations. +Maybe involving, one, two, or more separate or interconnected bits of software +that together can provide all the desirable properties. + +I think that language specific package managers are only currently good for +development, when it comes to deployment, you often need something that can +manage more of the system. + +Also, language specific package managers do not account for dependencies that +cross language boundaries. This means that you cannot really reason about +reproducible builds, or bootstrapping with a language specific package manager. + +On the other end of the scale, Debian binary packages are effectively just +archives that you unpack in to the root directory. They assume absolute and +relative paths, which makes them unsuitable for installing elsewhere (e.g. in a +users home directory). This means that it is not possible to use them if you do +not have root access on the system. + +## All is not yet lost... + +There are some signs of light in the darkness. [Debian's reproducible builds +initiative](https://wiki.debian.org/ReproducibleBuilds) is progressing well. In +the Debian way, this has ramifications for everyone, as an effort will be made +to include any changes made in Debian, in the software projects themselves. + +I am also hearing more and more about package managers that seem to be in +roughly the right spot. [Nix](https://nixos.org/nix/) and +[Guix](https://www.gnu.org/software/guix/), although I have used neither both +sound enticing, promising "atomic upgrades and rollbacks, side-by-side +installation of multiple versions of a package, multi-user package management +and easy setup of build environments" (from the Nix homepage). Although with +great power comes great responsibility, performing security updates in Debian +would probably be more complex if there could be multiple installations, of +perhaps versions of an insecure piece of software on a system. + +Perhaps some semantic web technologies can play a part. URI's could prove +useful as unique identifiers for software, and software versions. Basic package +descriptions could be written in RDF, using URI's, allowing these to be used by +multiple packaging systems (the ability to have sameAs properties in RDF might +be useful). + +At the moment, I am working on Debian packages. I depend on these for most of +my computers. Unfortunately, for some of the software projects I write, it is +not really possible to just depend on Debian packages. For some I have managed +to get by with git submodules, for others I have entered the insane world of +shell scripts which just download the dependencies off the web, sometimes also +using Bower and Grunt. + +Needless to say I am always on the look out for ways to improve this situation. |