NixOS contains infrastructure for building integration tests based on QEMU/KVM virtual machines running NixOS. Tests built on this infrastructure are continuously run on new nixpkgs versions to ensure that NixOS continues to install and boot and that various services continue to operate correctly. This post illustrates how one may test a simple web service using NixOS tests.

To have a simple enough example at hand, we wrote a small service in PHP—a classical guestbook in which visitors can leave a message that will then be written to a database and subsequently shown to later visitors of the same site. All example code is publicly available on GitHub at mayflower/goldenbook.

Cloning the repo and calling nix-build in the root directory will start a VM with the appropriate services loaded and run the tests provided in test.nix.

Services for Application

Our application is a typical PHP+MySQL application, so the components we will need to be able to run a modern deployment of it are:

• A web server (we chose nginx)
• A PHP FastCGI server (we chose php-fpm)
• A MySQL server (we chose MariaDB)

server.nix is a NixOS configuration module which enables and configures these services. It can be used to deploy the service on production servers by placing it in the server configuration’s imports as well as for testing.

Testing

The principal testing scaffold to run the NixOS tests for our project is provided in default.nix. It applies the nixosTest function from nixpkgs to the file containing the test specification in order to produce a test derivation.

1# default.nix
2{ pkgs ? import <nixpkgs> {} }: {
3  test = pkgs.nixosTest ./test.nix;
4}

The test specification is an attribute set that contains the following attributes:

• name: the name of the test;
• nodes: an attribute set containing a NixOS configuration module for each virtual machine used by the test;
• testScript: a Perl script that runs the actual test.

For instance in our case:

1# test.nix
2{ ... }: {
3  name = "golden-book-test";
4  nodes.server = ./server.nix;
5  testScript = "...";
6}

The Test Script

The testScript parameter for NixOS tests defines how the actual test should run. The script is written in Perl. The machines defined in nodes are exposed as objects which provide a variety of methods for controlling the machines. These range from powering machines on and off to running commands on them to making screenshots and using OCR to get text from them. All these functions are documented in the NixOS manual.

Our test script looks like this:

1startAll;
2# Wait for all services to finish starting
3$server->waitForUnit("multi-user.target");
4# Server should start out with no posts in the database and return an empty but successful response at first
5$server->succeed('[ -z "$(curl --fail localhost)" ]');
6# Making a post should succeed and the server should respond politely
7$server->succeed("curl --fail -F text=testpost localhost | grep Thanks");
8# And after that, the post should be displayed on the main page.
9$server->succeed("curl --fail localhost | grep testpost");

It ensures that the web application functions correctly from a client’s point of view. If any of these commands fail, the test will also fail.

Building the tests using nix-build yields a result directory which contains a report of the test run including logs.

Conclusion

NixOS tests are a powerful tool for integration testing, as they allow sharing configuration between the test environment and production, reducing the gap between the environments and increasing test coverage.

We’ve barely scratched the surface of what the tests allow though! NixOS' own tests include complex tests which for instance simulate bittorrent clients communicating through NATs and with changing network topologies.