Managing software installs on Linux

June 23, 2020

If you use Linux, you’re likely going to be installing software from lots of different sources in lots of different ways. This is especially true if you develop software, as each project you work on has its own dependencies, build tools, etc. It can be difficult to keep track of all the things that are installed and manage them, so I’m going to give some of the tips and tricks I use to help.

Catch-all solution: containers/virtualisation

Before we start I just want to mention that the use of container tools like Docker or Podman, or using virtual machines, solve a lot of the problems of managing system state. They do require a fair bit of learning and setting up, and there can be limitations, but they provide a clean separation for different environments you might want to work in. Personally, I find containers too complex for my uses, but I know many who swear by them.

Keep a note of your changes

You could keep a text file or something where you can record anything you’ve installed or any system configuration you’ve made that is not easy to track or remember. It sounds crude, but as you may have hundreds of packages installed, which vary between whole applications and tiny dependency libraries, it can be useful to note what deliberate actions you took. It also helps in keeping track of whether you can get rid of things, if you note why you installed them.

Use packages where possible…

The package manager of your distro makes it easy to update and remove software. Some package managers (or tools like Debian’s deborphan) can show you all explicitly-installed packages, making it easy to remove old things you don’t need.

The main shortfall of package managers is that your distro might not have a piece of software you need, or it might be too outdated for your use. In that case, you’ll have to look for third-party package repos, like Ubuntu PPAs, or resort to installing from source.

And if you are installing from source, CheckInstall is a program which can bundle all the files that would be installed by e.g. make install into a package, so you can track/remove its files just like normal.

…and don’t mix with it when you can’t

Only files from packages should go in directories like /bin and /usr/lib. There is a whole other set of bin, lib, include, directories under /usr/local for non-packaged software, such as software you might install from source.

However, when installing multiple things, all the files are mixed together, so it’s hard to remove one from among the others. To address this, there is a really useful program called Stow which can keep each installation separate. Basically you build the software as normal, set to install under /usr/local, but then you actually install it to its own directory under /usr/local/stow, such as /usr/local/stow/thing-1.0. After, run stow -d /usr/local/stow thing-1.0 and the files will be linked into /usr/local in the usual places to they can be used as normal. The files can be unlinked with stow -d /usr/local/stow -D thing-1.0.

Isolated environments for programming languages

Programming languages often have associated tools for installing packages: Python has pip, Ruby has gem, Node has npm, etc. These tools often install to user-specific or system-wide locations. This is ok, but it means that it’s difficult to track which packages are installed for which project.

For Ruby, there is a tool called Bundler which can create project-specific environments with their own Gem packages. See the website for more info. (Personally I like to store the packages and environment in a sub-directory of my project, e.g. vendor/bundle. You can configure this by running bundle config path vendor/bundle.)

For Python, there is a tool called Pipenv which does a very similar thing for pip packages. (And you can store them in a sub-directory by setting the PIPENV_VENV_IN_PROJECT environment variable.)

← All posts