Problems of software evolution also apply to configuration files

One of the most important factors in determining how well a system, and its current shape is understood lies in how well (and where) it has been documented. This short article shows you a way to add rationale to your configuration files.

Reading the web I encountered the following article: Textual configuration has comments, GUIs don't.

In a nutshell, the article talks about how text-file configuration is better than GUI configuration because it allows for comments. With enough discipline, the maintainers of a system or application can use the comments to explain why certain configuration decisions were made.

The article, however short, makes a good point, and doesn't hit far from an area of Software Architecture in which a lot of research is still taking place. It made me wonder if some of the solutions that have been found for application code can also be put into action for more system-oriented tasks.

Another thing I started to wonder about is how, in bigger infrastructures, configuration changes that stretch between different services or even machines are recorded.

Tacit knowledge

One of the most important factors in determining how well a system, and its current shape is understood lies in how well (and perhaps where) it has been documented. Tacit knowledge is knowledge that lives in the heads of developers, or in a silent agreement between developers. It is knowledge about the "Why" of walked paths or design decisions.

Usually, if the developers that have this knowledge disappear -- change jobs, or have an unfortunate accident that leaves them impaired to speak about anything but the software product — this knowledge is lost. It can never be retrieved.

To get back to the domain of the original article. Tacit knowledge lives in all the little things. From a cronjob that for some reason needs to restart the Tomcat server every hour, to a mistifyingly large amount of PermGen space in the Catalina options environment variable. All the things that make you go "Why?!"

Known solutions

Some known solutions to minimizing the amount of tacit knowledge usually involve additional documentation systems in which all the small details and rationale for system development are stored. The unfortunate problem with this is that it creates a burden on the developers that decreases the productivity. Another, possible larger problem, is the fact that these systems are not integrated in the actual product that is being created, meaning that the documentation can become outdated, in essence, providing mis-information to users of the system.

Which is worse? No rationale, or being told a lie. Living under a blanket of false assumptions.

Archium is a project that has an interesting take on this problem. It integrates a number of new concepts into the programming language used to develop the product. Design decisions are at the heart of its implementation. The proof of concept implementation uses the Java programming language, and adds a number of constructs on top of Java allowing the developer to integrate rationale and source-code into one entity.

Perhaps something similar could be created for system maintenance.

Stacked configuration files

There already are a number of configuration provisioning systems; Puppet is one of them. Puppet allows a system maintainer to create and update configuration files on larger infrastructures from a single place. Introducing something along the lines of "design decisions" for configuration files, in which changes to the configuration are made in a stacked manner.

To keep track of the system's configuration history one could use a versioning tool like Subversion. This would allow you to record a history of changes. Each check in allows the maintainer to leave a message for the reason of his changes. Because puppet uses centralized configuration files, changes that influence multiple systems would be stored in the same revision.

A disadvantage of putting changes in a history "bin" is the fact that the changes are not visible in the configuration file itself. One has to undertake a small quest into the subversion archives to figure out the Why and What of certain configuration options. I do think that proper commenting inside the configuration files, and good tooling will help you a lot in this.

I propose a novel concept called Stacked configuration files. Not unlike Archium, this is a super-set on top of existing configuration file syntaxes. It allows for tacit knowledge to be stored right at the configuration line itself. It also contains all the historical settings with reasons of change. This allows existing, but also new maintainers to find their way through the system and its history much quicker.