Commit graph

555 commits

Author SHA1 Message Date
Robert Hensing d3f956aba3 nixos/lib: Add featureFlags, use it for minimal modules 2022-01-10 12:36:57 +01:00
Robert Hensing 25caf736d5 nixos/eval-config: Layer on top of nixos/eval-config-minimal 2022-01-10 12:26:59 +01:00
Robert Hensing e31e096b66 nixos/lib: Move evalModules into its own file 2022-01-10 12:26:59 +01:00
Robert Hensing fbd038eca2 nixos/lib: init (experimental) 2022-01-07 01:21:07 +01:00
pennae 50954ad1c5 nixos/make-options-doc: treat missing descriptions as errors by default
this partially solves the problem of "missing description" warnings of the
options doc build being lost by nix build, at the cost of failing builds that
previously ran. an option to disable this behaviour is provided.
2022-01-02 19:46:13 +01:00
pennae 1301bdb185 nixos/make-options-doc: turn relatedPackages into links
link to search.nixos.org instead of pulling package metadata out of pkgs. this
lets us cache docs of a few more modules and provides easier access to package
info from the HTML manual, but makes the manpage slightly less useful since
package description are no longer rendered.
2022-01-02 19:46:13 +01:00
pennae b92a47c87c nixos/make-options-doc: add type annotations to mergeJSON.py 2022-01-02 19:46:13 +01:00
pennae fc614c37c6 nixos/documentation: split options doc build
most modules can be evaluated for their documentation in a very
restricted environment that doesn't include all of nixpkgs. this
evaluation can then be cached and reused for subsequent builds, merging
only documentation that has changed into the cached set. since nixos
ships with a large number of modules of which only a few are used in any
given config this can save evaluation a huge percentage of nixos
options available in any given config.

in tests of this caching, despite having to copy most of nixos/, saves
about 80% of the time needed to build the system manual, or about two
second on the machine used for testing. build time for a full system
config shrank from 9.4s to 7.4s, while turning documentation off
entirely shortened the build to 7.1s.
2022-01-02 19:46:13 +01:00
Julien Moutinho 8b842173d0 nixos/make-options-doc: fix invalid ':' in XML NCName (non-colonized name) 2021-12-28 22:18:16 -05:00
pennae a70b1eb630 nixos/lib/make-options-doc: fix with nix 2.3 2021-12-14 03:41:09 +01:00
Jacek Galowicz b6bf1ca717
Merge pull request #149329 from marijanp/test-driver-restructuring
nixos/test-driver: make the test-driver a python package
2021-12-10 18:41:49 +00:00
Marijan Petričević ab693de868 nixos/test-driver: make the test-driver a python package 2021-12-10 12:27:45 +00:00
Silvan Mosberger 2f2b60dd87 lib/nixos/eval-config.nix: Fix extraArgs
Fixes a mistake in https://github.com/NixOS/nixpkgs/pull/148315 that
caused https://github.com/NixOS/nixpkgs/issues/148343#issuecomment-990881216
2021-12-10 12:55:30 +01:00
Silvan Mosberger 15c41e1d54
Merge pull request #147265 from pennae/option-docs-build
put all option docs build flavors on equal footing
2021-12-08 21:27:02 +01:00
Artturi 3fe92b45cb
Merge pull request #149665 from Artturin/squashfscores 2021-12-08 20:20:32 +02:00
Artturin eea6baad50 make-squashfs: use $NIX_BUILD_CORES or 48 cores if above 48
by default all cores are used

hoping this will fix the hydra i686 squashfs build issues as all the
failures were using 64 cores

Parallel mksquashfs: Using 64 processors
Creating 4.0 filesystem on ..., block size 1048576.
FATAL ERROR: mangle2:: xz compress failed with error code 5
2021-12-08 19:23:31 +02:00
ajs124 eee45bb295
Merge pull request #146815 from ElvishJerricco/systemd-utils-expressions
Move systemd-lib.nix and systemd-unit-options.nix into utils
2021-12-08 15:07:28 +00:00
Silvan Mosberger 490d46f044
Merge pull request #148315 from hercules-ci/nixos-evalModules-legacy-cleanup
NixOS/evalModules legacy cleanup
2021-12-07 18:30:52 +01:00
pennae 027f7e1b7f nixos/lib/make-options-doc: generate options.xml from options.json
to do this we must replace derivations with attrsets in make-options-doc, since
xml can represent derivations differently from attrset but json cannot. this
also given asciidoc and mddoc the ability to handle derivation differently,
which they previously didn't have.
2021-12-06 16:12:32 +01:00
pennae 9b97a2ea88 nix/lib/make-options-doc: remove nix-level sorting
there are no remaining users of sorted option lists except the docbook build,
which sorts its input separately.
2021-12-06 16:12:32 +01:00
pennae 4670400309 nixos/lib/make-options-doc: generate asciidoc/md in derivations
use the json file derivation we already have to also generate the asciidoc and
md options docs instead of formatting the options in nix. docbook docs are
already produced in derivations.

the new script produce the exact same output as the old in-nix generation.
2021-12-06 16:12:30 +01:00
Artturin 60422ba2ea nixos/test-driver: add 10ms delay to send_key
attempt to fix https://github.com/NixOS/nixpkgs/issues/147294
2021-12-03 20:04:56 +02:00
Robert Hensing 78ada83361 nixos/eval-config: Deprecate extraArgs and check parameters 2021-12-02 18:23:43 +00:00
Robert Hensing 0b5aea2b27 nixos/eval-config: Remove a rec 2021-12-02 18:23:43 +00:00
Robert Hensing 59c4a35aab nixos/eval-config: Avoid evalModules args and check parameters 2021-12-02 18:23:43 +00:00
Jacek Galowicz 58371472fe
Merge pull request #146512 from DeterminateSystems/better-visibility
nixos/test-driver: more context when step finishes, give more functions nested labels
2021-12-02 09:26:21 +00:00
Jacek Galowicz 57dc1085dc
Merge pull request #146271 from DeterminateSystems/better-timeouts
nixos/test-driver: add execute_with_timeout, add (functional) timeouts to more functions
2021-12-01 19:01:05 +00:00
Cole Helbling af765f3abd nixos/test-driver: give more functions nested labels
This will make it easier to trace through the test execution without
having to scroll through the entire kernel output.
2021-12-01 10:29:56 -08:00
Cole Helbling c6ee63259a nixos/test-driver: more context when step finishes
When displaying the amount of time some step took, with no other
context, it becomes nigh impossible (especially in longer tests) to see
when specific steps finished.
2021-12-01 10:29:56 -08:00
Robert Hensing 8a129f8cf0
Merge pull request #144094 from hercules-ci/nixos-specialisations-use-extendModules
nixos/specialisation: Rephrase in terms of extendModules, noUserModules
2021-12-01 11:03:36 +01:00
Martin Schwaighofer af180d554b qemu, runInLinuxVM: change default cpu to qemu64
The flag -cpu max leaves QEMU 6.1.0 stuck on some systems,
for example when /dev/kvm is not read-writable.
This does not happen with -cpu qemu64.

Getting stuck like that is a regression in 6.1.0 not yet present in 6.0.0
and should be fixed with 6.2.0 according to early testing with rc1.

We should consider reverting this change when we merge QEMU 6.2.0.
See #146526.

fixes #141596
2021-11-30 13:06:22 +00:00
talyz e8cc900eae
make-disk-image: Make additionalPaths work with Nix 2.4
The `nix` command is marked as experimental since 2.4, so an extra
flag is required to unlock it.
2021-11-29 21:11:15 +01:00
Ryan Burns 2175b157ac treewide: refactor isi686 && isx86_64 -> isx86 2021-11-20 17:50:41 -08:00
Will Fancher 851495a752 Move systemd-lib.nix and systemd-unit-options.nix into utils 2021-11-20 17:52:29 -05:00
Patrick Hilhorst 1334a62539
test-driver.py: directly import pathlib.Path 2021-11-20 01:37:08 +01:00
Patrick Hilhorst 9e71014ede
test-driver.py: always export single machine as 'machine' 2021-11-19 23:02:30 +01:00
Patrick Hilhorst a8f693ed48
test-driver.py: fix weird non-pythonism 2021-11-19 22:28:32 +01:00
Cole Helbling e62b8020f3 nixos/test-driver: add (functional) timeouts to more functions
A retry timeout doesn't really help if the thing it's retrying may block
forever.
2021-11-17 16:11:10 -08:00
Cole Helbling 363d7f3ae8 nixos/test-driver: add timeout parameter to execute 2021-11-17 16:11:03 -08:00
Artturin 83f892c511 nixos/lib: add /usr to pathsNeededForBoot
if /usr is split from root then it has to be mounted at boot
See https://github.com/NixOS/nixpkgs/issues/146115
2021-11-15 16:28:22 +02:00
Jacek Galowicz 2ba0732988
Merge pull request #142747 from helsinki-systems/fix/test-runner-execute
nixos/test-runner: Fix execute() flakiness
2021-11-02 14:51:50 +01:00
Robert Hensing b5681a7a40 nixos/specialisation: Rephrase in terms of extendModules, noUserModules
By using the new extendModules function to produce the specialisations,
we avoid reimplementing the eval-config.nix logic in reverse and fix
cross compilation support for specialisations in the process.
2021-11-01 10:59:47 +01:00
Robert Hensing 0b0d2637fc Revert "nixos/specialisation: Rephrase in terms of extendModules, noUserModules"
This reverts commit e2bea4427b.

While this commit was probably fine, I want to be conservative
with changes right before the release branch-off.
So far the extendModules commits have been adding and refactoring
expressions that did not affect derivation hashes, whereas this
commit changes the module ordering. I will open a separate PR for
it.
2021-11-01 09:41:55 +01:00
Robert Hensing e2bea4427b nixos/specialisation: Rephrase in terms of extendModules, noUserModules 2021-11-01 09:40:16 +01:00
Robert Hensing 22584ce667 nixos/eval-config.nix: Expose type
Also works for (pkgs.nixos {}).type.
2021-11-01 09:38:49 +01:00
Robert Hensing 56c4f9d005 nixosTest: Fix infinite recursion involving hasContext testScript when useNixStoreImage = true
The involved test was nixosTests.nextcloud.basic21.

It has a testScript that is strict in nodes.nextcloud.config.system.build.vm,
in assertions about imagemagick being in the system closure.

The recursion was introduced in 329a4461a7 from
https://github.com/NixOS/nixpkgs/pull/140792
2021-10-31 18:25:19 +01:00
talyz 46f7521144 make-disk-image: Add an argument to only build a Nix store image
Add the `onlyNixStore` argument which enables building images
containing the contents of the Nix store at the root, instead of a
full system.
2021-10-28 12:55:01 +02:00
talyz 329a4461a7 nixos/testing-python: Copy test script derivations to nodes
Make sure the all derivations referenced by the test script are
available on the nodes. Accessing these derivations works just fine
without this change when using 9p to mount the host's store, but when
an image is built (virtualisation.buildRootImage), the dependencies
need to be copied to the image. We don't want to copy the script
itself, though, since that would trigger unnecessary image rebuilds.
2021-10-28 12:55:01 +02:00
talyz 87d8eec069 make-disk-image: Add additionalPaths argument
Add an argument which makes it possible to add additional paths to the
image's nix store.
2021-10-28 12:55:01 +02:00
talyz 398a73ac98 make-disk-image: Add copyChannel argument
Add a copyChannel argument which controls whether the current source
tree will be made available as a nix channel in the image or
not. Previously, it always was. Making it available is useful for
interactive use of nix utils, but changes the hash of the image when
the sources are updated.
2021-10-28 12:55:01 +02:00