553 lines
19 KiB
XML
553 lines
19 KiB
XML
<chapter xmlns="http://docbook.org/ns/docbook"
|
||
xmlns:xlink="http://www.w3.org/1999/xlink">
|
||
|
||
<title>Development</title>
|
||
|
||
<para>This chapter has some random notes on hacking on
|
||
NixOS.</para>
|
||
|
||
|
||
<!--===============================================================-->
|
||
|
||
<section>
|
||
|
||
<title>Hacking on NixOS</title>
|
||
|
||
<para>By default, NixOS’s <command>nixos-rebuild</command> command
|
||
uses the NixOS and Nixpkgs sources provided by the
|
||
<literal>nixos-unstable</literal> channel (kept in
|
||
<filename>/nix/var/nix/profiles/per-user/root/channels/nixos</filename>).
|
||
To modify NixOS, however, you should check out the latest sources from
|
||
Subversion. This is done using the following command:
|
||
|
||
<screen>
|
||
$ nixos-checkout <replaceable>/my/sources</replaceable>
|
||
</screen>
|
||
|
||
This will check out the latest NixOS sources to
|
||
<filename><replaceable>/my/sources</replaceable>/nixos</filename> and
|
||
the Nixpkgs sources to
|
||
<filename><replaceable>/my/sources</replaceable>/nixpkgs</filename>.
|
||
If you want to rebuild your system using your (modified) sources, you
|
||
need to tell <command>nixos-rebuild</command> about them using the
|
||
<option>-I</option> flag:
|
||
|
||
<screen>
|
||
$ nixos-rebuild switch -I <replaceable>/my/sources</replaceable>
|
||
</screen>
|
||
|
||
</para>
|
||
|
||
</section>
|
||
|
||
|
||
<!--===============================================================-->
|
||
|
||
<section>
|
||
|
||
<title>Extending NixOS</title>
|
||
|
||
<para>NixOS is based on a modular system for declarative configuration.
|
||
This system combines multiple <emphasis>modules</emphasis> to produce one
|
||
configuration. One of the module which compose your computer
|
||
configuration is <filename>/etc/nixos/configuration.nix</filename>. Other
|
||
modules are available under NixOS <filename>modules</filename>
|
||
directory</para>
|
||
|
||
<para>A module is a file which handles one specific part of the
|
||
configuration. This part of the configuration could correspond to an
|
||
hardware, a service, network settings, or preferences. A module
|
||
configuration does not have to handle everything from scratch, it can base
|
||
its configuration on other configurations provided by other modules. Thus
|
||
a module can <emphasis>define</emphasis> options to setup its
|
||
configuration, and it can also <emphasis>declare</emphasis> options to be
|
||
fed by other modules.</para>
|
||
|
||
<!-- module syntax -->
|
||
|
||
<para xml:id="para-module-syn">A module is a file which contains a Nix
|
||
expression. This expression should be either an expression which gets
|
||
evaluated into an attribute set or a function which returns an attribute
|
||
set.</para>
|
||
|
||
<para>When the expression is a function, it should expect only one argument
|
||
which is an attribute set containing an attribute
|
||
named <varname>config</varname> and another attribute
|
||
named <varname>pkgs</varname>. The <varname>config</varname> attribute
|
||
contains the result of the merge of all modules. This attribute is
|
||
evaluated lazily, such as any Nix expression. For more details on how
|
||
options are merged, see the details in <xref linkend="para-opt-decl"/>.
|
||
The <varname>pkgs</varname> attribute
|
||
contains <emphasis>nixpkgs</emphasis> attribute set of packages. This
|
||
attribute is necessary for declaring options.</para>
|
||
|
||
<example xml:id='module-syntax'><title>Usual module content</title>
|
||
<programlisting>
|
||
{ config, pkgs, ... }: <co xml:id='module-syntax-1' />
|
||
|
||
{
|
||
imports =
|
||
[ <co xml:id='module-syntax-2' />
|
||
];
|
||
|
||
options = {
|
||
<co xml:id='module-syntax-3' />
|
||
};
|
||
|
||
config = {
|
||
<co xml:id='module-syntax-4' />
|
||
};
|
||
}</programlisting>
|
||
</example>
|
||
|
||
<para><xref linkend='module-syntax' /> Illustrates
|
||
a <emphasis>module</emphasis> skeleton.
|
||
|
||
<calloutlist>
|
||
<callout arearefs='module-syntax-1'>
|
||
<para>This line makes the current Nix expression a function. This
|
||
line can be omitted if there is no reference to <varname>pkgs</varname>
|
||
and <varname>config</varname> inside the module.</para>
|
||
</callout>
|
||
|
||
<callout arearefs='module-syntax-2'>
|
||
<para>This list is used to enumerate path to other modules which are
|
||
declaring options used by the current module. In NixOS, default modules
|
||
are listed in the file <filename>modules/module-list.nix</filename>.
|
||
The default modules don't need to be added in the import list.</para>
|
||
</callout>
|
||
|
||
<callout arearefs='module-syntax-3'>
|
||
<para>This attribute set contains an attribute set of <emphasis>option
|
||
declaration</emphasis>.</para>
|
||
</callout>
|
||
|
||
<callout arearefs='module-syntax-4'>
|
||
<para>This attribute set contains an attribute set of <emphasis>option
|
||
definitions</emphasis>. If the module does not have any imported
|
||
modules or any option declarations, then this attribute set can be used
|
||
in place of its parent attribute set. This is a common case for simple
|
||
modules such
|
||
as <filename>/etc/nixos/configuration.nix</filename>.</para>
|
||
</callout>
|
||
</calloutlist>
|
||
|
||
</para>
|
||
|
||
<!-- option definitions -->
|
||
|
||
<para xml:id="para-opt-def">A module defines a configuration which would be
|
||
interpreted by other modules. To define a configuration, a module needs
|
||
to provide option definitions. An option definition is a simple
|
||
attribute assignment.</para>
|
||
|
||
<para>Option definitions are made in a declarative manner. Without
|
||
properties, options will always be defined with the same value. To
|
||
introduce more flexibility in the system, option definitions are guarded
|
||
by <emphasis>properties</emphasis>.</para>
|
||
|
||
<para>Properties are means to introduce conditional values inside option
|
||
definitions. This conditional values can be distinguished in two
|
||
categories. The condition which are local to the current configuration
|
||
and conditions which are dependent on others configurations. Local
|
||
properties are <varname>mkIf</varname>
|
||
and <varname>mkAssert</varname>. Global properties
|
||
are <varname>mkOverride</varname>, <varname>mkDefault</varname>
|
||
and <varname>mkOrder</varname>.</para>
|
||
|
||
<para><varname>mkIf</varname> is used to remove the option definitions which
|
||
are below it if the condition is evaluated to
|
||
false. <varname>mkAssert</varname> expects the condition to be evaluated
|
||
to true otherwise it raises an error message.</para>
|
||
|
||
<para><varname>mkOverride</varname> is used to mask previous definitions if
|
||
the current value has a lower mask number. The mask value is 100 (default)
|
||
for any option definition which does not use this property.
|
||
Thus, <varname>mkDefault</varname> is just a short-cut with a higher mask
|
||
(1000) than the default mask value. This means that a module can set an
|
||
option definition as a preference, and still let another module defining
|
||
it with a different value without using any property.</para>
|
||
|
||
<para><varname>mkOrder</varname> is used to sort definitions based on the
|
||
rank number. The rank number will sort all options definitions before
|
||
giving the sorted list of option definition to the merge function defined
|
||
in the option declaration. A lower rank will move the definition to the
|
||
beginning and a higher rank will move the option toward the end. The
|
||
default rank is 100.</para>
|
||
|
||
<!-- option declarations -->
|
||
|
||
<para xml:id="para-opt-decl">A module may declare options which are used by
|
||
other module to change the configuration provided by the current module.
|
||
Changes to the option definitions are made with properties which are using
|
||
values extracted from the result of the merge of all modules
|
||
(the <varname>config</varname> argument).</para>
|
||
|
||
<para>The <varname>config</varname> argument reproduce the same hierarchy of
|
||
all options declared in all modules. For each option, the result of the
|
||
option is available, it is either the default value or the merge of all
|
||
definitions of the option.</para>
|
||
|
||
<para>Options are declared with the
|
||
function <varname>pkgs.lib.mkOption</varname>. This function expects an
|
||
attribute set which at least provides a description. A default value, an
|
||
example, a type, a merge function and a post-process function can be
|
||
added.</para>
|
||
|
||
<para>Types are used to provide a merge strategy for options and to ensure
|
||
the type of each option definitions. They are defined
|
||
in <varname>pkgs.lib.types</varname>.</para>
|
||
|
||
<para>The merge function expects a list of option definitions and merge
|
||
them to obtain one result of the same type.</para>
|
||
|
||
<para>The post-process function (named <varname>apply</varname>) takes the
|
||
result of the merge or of the default value, and produce an output which
|
||
could have a different type than the type expected by the option.</para>
|
||
|
||
<!-- end -->
|
||
|
||
<example xml:id='locate-example'><title>Locate Module Example</title>
|
||
<programlisting>
|
||
{ config, pkgs, ... }:
|
||
|
||
with pkgs.lib;
|
||
|
||
let
|
||
cfg = config.services.locate;
|
||
locatedb = "/var/cache/locatedb";
|
||
logfile = "/var/log/updatedb";
|
||
cmd =''root updatedb --localuser=nobody --output=${locatedb} > ${logfile}'';
|
||
in
|
||
|
||
{
|
||
imports = [ /etc/nixos/nixos/modules/services/scheduling/cron.nix ];
|
||
|
||
options = {
|
||
services.locate = {
|
||
enable = mkOption {
|
||
default = false;
|
||
example = true;
|
||
type = with types; bool;
|
||
description = ''
|
||
If enabled, NixOS will periodically update the database of
|
||
files used by the <command>locate</command> command.
|
||
'';
|
||
};
|
||
|
||
period = mkOption {
|
||
default = "15 02 * * *";
|
||
type = with types; uniq string;
|
||
description = ''
|
||
This option defines (in the format used by cron) when the
|
||
locate database is updated.
|
||
The default is to update at 02:15 (at night) every day.
|
||
'';
|
||
};
|
||
};
|
||
};
|
||
|
||
config = mkIf cfg.enable {
|
||
services.cron = {
|
||
enable = true;
|
||
systemCronJobs = "${cfg.period} root ${cmd}";
|
||
};
|
||
};
|
||
}</programlisting>
|
||
</example>
|
||
|
||
<para><xref linkend='locate-example' /> illustrates a module which handles
|
||
the regular update of the database which index all files on the file
|
||
system. This modules has option definitions to rely on the cron service
|
||
to run the command at predefined dates. In addition, this modules
|
||
provides option declarations to enable the indexing and to use different
|
||
period of time to run the indexing. Properties are used to prevent
|
||
ambiguous definitions of option (enable locate service and disable cron
|
||
services) and to ensure that no options would be defined if the locate
|
||
service is not enabled.</para>
|
||
|
||
</section>
|
||
|
||
|
||
<!--===============================================================-->
|
||
|
||
<section>
|
||
|
||
<title>Building specific parts of NixOS</title>
|
||
|
||
<para>
|
||
|
||
<screen>
|
||
$ nix-build /etc/nixos/nixos -A <replaceable>attr</replaceable></screen>
|
||
|
||
where <replaceable>attr</replaceable> is an attribute in
|
||
<filename>/etc/nixos/nixos/default.nix</filename>. Attributes of interest include:
|
||
|
||
<variablelist>
|
||
|
||
<varlistentry>
|
||
<term><varname>config</varname></term>
|
||
<listitem><para>The computer configuration generated from
|
||
the <envar>NIXOS_CONFIG</envar> environment variable (default
|
||
is <filename>/etc/nixos/configuration.nix</filename>) with the NixOS
|
||
default set of modules.</para></listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term><varname>system</varname></term>
|
||
<listitem><para>The derivation which build your computer system. It is
|
||
built by the command <command>nixos-rebuild
|
||
build</command></para></listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term><varname>vm</varname></term>
|
||
<listitem><para>The derivation which build your computer system inside a
|
||
virtual machine. It is built by the command <command>nixos-rebuild
|
||
build-vm</command></para></listitem>
|
||
</varlistentry>
|
||
</variablelist>
|
||
|
||
</para>
|
||
|
||
<para>
|
||
Most parts of NixOS can be build through the <varname>config</varname>
|
||
attribute set. This attribute set allows you to have a view of the merged
|
||
option definitions and all its derivations. Important derivations are store
|
||
inside the option <option>system.build</option> and can be listed with the
|
||
command <command>nix-instantiate --xml --eval-only /etc/nixos/nixos -A
|
||
config.system.build</command>
|
||
</para>
|
||
|
||
</section>
|
||
|
||
|
||
<!--===============================================================-->
|
||
|
||
<section>
|
||
|
||
<title>Building your own NixOS CD</title>
|
||
|
||
<para>Building a NixOS CD is as easy as configuring your own computer. The
|
||
idea is to use another module which will replace
|
||
your <filename>configuration.nix</filename> to configure the system that
|
||
would be install on the CD.</para>
|
||
|
||
<para>Default CD/DVD configurations are available
|
||
inside <filename>nixos/modules/installer/cd-dvd</filename>. To build them
|
||
you have to set <envar>NIXOS_CONFIG</envar> before
|
||
running <command>nix-build</command> to build the ISO.
|
||
|
||
<screen>
|
||
$ export NIXOS_CONFIG=/etc/nixos/nixos/modules/installer/cd-dvd/installation-cd-minimal.nix
|
||
$ nix-build /etc/nixos/nixos -A config.system.build.isoImage</screen>
|
||
|
||
</para>
|
||
|
||
<para>Before burning your CD/DVD, you can check the content of the image by mounting anywhere like
|
||
suggested by the following command:
|
||
|
||
<screen>
|
||
$ mount -o loop -t iso9660 ./result/iso/cd.iso /mnt/iso</screen>
|
||
|
||
</para>
|
||
|
||
</section>
|
||
|
||
|
||
<!--===============================================================-->
|
||
|
||
<section>
|
||
|
||
<title>Testing the installer</title>
|
||
|
||
<para>Building, burning, and booting from an installation CD is rather
|
||
tedious, so here is a quick way to see if the installer works
|
||
properly:
|
||
|
||
<screen>
|
||
$ export NIXOS_CONFIG=/etc/nixos/nixos/modules/installer/cd-dvd/installation-cd-minimal.nix
|
||
$ nix-build /etc/nixos/nixos -A config.system.build.nixosInstall
|
||
$ dd if=/dev/zero of=diskimage seek=2G count=0 bs=1
|
||
$ yes | mke2fs -j diskimage
|
||
$ mount -o loop diskimage /mnt
|
||
$ ./result/bin/nixos-install</screen>
|
||
|
||
</para>
|
||
|
||
</section>
|
||
|
||
|
||
<!--===============================================================-->
|
||
|
||
<section>
|
||
|
||
<title>Testing the <literal>initrd</literal></title>
|
||
|
||
<para>A quick way to test whether the kernel and the initial ramdisk
|
||
boot correctly is to use QEMU’s <option>-kernel</option> and
|
||
<option>-initrd</option> options:
|
||
|
||
<screen>
|
||
$ nix-build /etc/nixos/nixos -A config.system.build.initialRamdisk -o initrd
|
||
$ nix-build /etc/nixos/nixos -A config.system.build.kernel -o kernel
|
||
$ qemu-system-x86_64 -kernel ./kernel/bzImage -initrd ./initrd/initrd -hda /dev/null
|
||
</screen>
|
||
|
||
</para>
|
||
|
||
</section>
|
||
|
||
<section>
|
||
|
||
<title>Whole-system testing using virtual machines</title>
|
||
|
||
<para>
|
||
Complete NixOS GNU/Linux systems can be tested in virtual machines
|
||
(VMs). This makes it possible to test a system upgrade or
|
||
configuration change before rebooting into it, using the
|
||
<command>nixos-rebuild build-vm</command> or
|
||
<command>nixos-rebuild build-vm-with-bootloader</command> command.
|
||
</para>
|
||
|
||
<para>
|
||
<!-- The following is adapted from
|
||
http://wiki.nixos.org/wiki/NixOS_VM_tests, by Eelco Dolstra. -->
|
||
|
||
The <filename>tests/</filename> directory in the NixOS source tree
|
||
contains several <emphasis>whole-system unit tests</emphasis>.
|
||
These tests can be run<footnote><para>NixOS tests can be run both from
|
||
NixOS and from a non-NixOS GNU/Linux distribution, provided the
|
||
Nix package manager is installed.</para></footnote> from the NixOS
|
||
source tree as follows:
|
||
|
||
<screen>
|
||
$ nix-build tests/ -A nfs.test
|
||
</screen>
|
||
|
||
This performs an automated test of the NFS client and server
|
||
functionality in the Linux kernel, including file locking
|
||
semantics (e.g., whether locks are maintained across server
|
||
crashes). It will first build or download all the dependencies of
|
||
the test (e.g., all packages needed to run a NixOS VM). The test
|
||
is defined in <link
|
||
xlink:href="https://nixos.org/repos/nix/nixos/trunk/tests/nfs.nix">
|
||
<filename>tests/nfs.nix</filename></link>. If the test succeeds,
|
||
<command>nix-build</command> will place a symlink
|
||
<filename>./result</filename> in the current directory pointing at
|
||
the location in the Nix store of the test results (e.g.,
|
||
screenshots, test reports, and so on). In particular, a
|
||
pretty-printed log of the test is written to
|
||
<filename>log.html</filename>, which can be viewed using a web
|
||
browser like this:
|
||
|
||
<screen>
|
||
$ icecat result/log.html
|
||
</screen>
|
||
</para>
|
||
|
||
<para>
|
||
It is also possible to run the test environment interactively,
|
||
allowing you to experiment with the VMs. For example:
|
||
|
||
<screen>
|
||
$ nix-build tests/ -A nfs.driver
|
||
$ ./result/bin/nixos-run-vms
|
||
</screen>
|
||
|
||
The script <command>nixos-run-vms</command> starts the three
|
||
virtual machines defined in the NFS test using QEMU/KVM. The root
|
||
file system of the VMs is created on the fly and kept across VM
|
||
restarts in
|
||
<filename>./</filename><varname>hostname</varname><filename>.qcow2</filename>.
|
||
</para>
|
||
|
||
<para>
|
||
Finally, the test itself can be run interactively. This is
|
||
particularly useful when developing or debugging a test:
|
||
|
||
<screen>
|
||
$ nix-build tests/ -A nfs.driver
|
||
$ ./result/bin/nixos-test-driver
|
||
starting VDE switch for network 1
|
||
>
|
||
</screen>
|
||
|
||
Perl statements can now be typed in to start or manipulate the
|
||
VMs:
|
||
|
||
<screen>
|
||
> startAll;
|
||
(the VMs start booting)
|
||
> $server->waitForJob("nfs-kernel-nfsd");
|
||
> $client1->succeed("flock -x /data/lock -c 'sleep 100000' &");
|
||
> $client2->fail("flock -n -s /data/lock true");
|
||
> $client1->shutdown;
|
||
(this releases client1's lock)
|
||
> $client2->succeed("flock -n -s /data/lock true");
|
||
</screen>
|
||
|
||
The function <command>testScript</command> executes the entire
|
||
test script and drops you back into the test driver command line
|
||
upon its completion. This allows you to inspect the state of the
|
||
VMs after the test (e.g. to debug the test script).
|
||
</para>
|
||
|
||
<para>
|
||
This and other tests are continuously run on <link
|
||
xlink:href="http://hydra.nixos.org/jobset/nixos/trunk/with-status">the
|
||
Hydra instance at <literal>nixos.org</literal></link>, which
|
||
allows developers to be notified of any regressions introduced by
|
||
a NixOS or Nixpkgs change.
|
||
</para>
|
||
|
||
<para>
|
||
The actual Nix programming interface to VM testing is in NixOS,
|
||
under <link
|
||
xlink:href="https://nixos.org/repos/nix/nixos/trunk/lib/testing.nix">
|
||
<filename>lib/testing.nix</filename></link>. This file defines a
|
||
function which takes an attribute set containing a
|
||
<literal>nixpkgs</literal> attribute (the path to a Nixpkgs
|
||
checkout), and a <literal>system</literal> attribute (the system
|
||
type). It returns an attribute set containing several utility
|
||
functions, among which the main entry point is
|
||
<literal>makeTest</literal>.
|
||
</para>
|
||
|
||
<para>
|
||
The <literal>makeTest</literal> function takes a function similar to
|
||
that found in <link
|
||
xlink:href="https://nixos.org/repos/nix/nixos/trunk/tests/nfs.nix">
|
||
<filename>tests/nfs.nix</filename></link> (discussed above). It
|
||
returns an attribute set containing (among others):
|
||
|
||
<variablelist>
|
||
|
||
<varlistentry>
|
||
<term><varname>test</varname></term>
|
||
<listitem><para>A derivation containing the test log as an HTML file,
|
||
as seen above, suitable for presentation in the Hydra continuous
|
||
build system.</para></listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term><varname>report</varname></term>
|
||
<listitem><para>A derivation containing a code coverage report, with
|
||
meta-data suitable for Hydra.</para></listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term><varname>driver</varname></term>
|
||
<listitem><para>A derivation containing scripts to run the VM test or
|
||
interact with the VM network interactively, as seen above.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
</variablelist>
|
||
</para>
|
||
|
||
|
||
</section>
|
||
|
||
</chapter>
|