Commit graph

797 commits

Author SHA1 Message Date
github-actions[bot] 3e675d06f5
Merge staging-next into staging 2022-10-09 12:02:02 +00:00
Maximilian Bosch 2480532bd1
nixos/doc: fix build
Now we even have options like
`services.listmonk.database.settings."app.notify_emails"` shown
correctly (i.e. with quotes).
2022-10-09 10:13:21 +02:00
talyz 4df4d2a8ea genJqSecretsReplacementSnippet: Allow dots in attribute names...
...and escape quotation marks and backslashes.
2022-10-09 08:12:19 +02:00
Artturin 09226fffcf nixosOptionsDoc: buildInputs -> nativeBuildInputs
to make strictDepsByDefault work
2022-10-07 19:26:22 +03:00
Robert Hensing 70ec3b9f54
Merge pull request #193498 from hercules-ci/nixos-doc-disambiguate-test-options
nixos/doc: disambiguate test option ids
2022-10-04 21:35:49 +01:00
Adam Joseph c46bdcbaf2 nixos/lib/qemu-common.nix: set qemuSerialDevice for isMips64 2022-10-04 07:48:12 +00:00
figsoda a1d50eecab
Merge pull request #193132 from figsoda/clean-up
treewide: clean up
2022-10-01 17:03:11 -04:00
Robert Hensing 216c5dc10d nixos/doc: Disambiguate test option ids
Changing this later on would break external links into the manual.
2022-09-29 12:41:59 +02:00
Robert Hensing 91879ce160 make-options-doc: Make optionIdPrefix configurable ("opt-") 2022-09-29 12:34:06 +02:00
Robert Hensing aed1deab05 nixos/testing: Fix release.nix tests evaluation
Fixes the problem introduced by 12b3066aae
which caused nixos/release.nix to return the wrong attributes, while
intending to only affect nixos/lib's runTest.
This also removes callTest from the test options, because callTest is
only ever invoked by all-tests.nix.
2022-09-29 10:49:09 +02:00
Franz Pletz 417b476db7
Merge pull request #187817 from intelfx/nixos-containertarballs-fix-pixz
make-system-tarball: use `pixz -t`
2022-09-29 03:26:20 +02:00
Robert Hensing 12b3066aae lib.nixos.runTest: Return the test itself
(NB: the lib in the flake)

Correcting a mistake I made when removing the test matrix code
from https://github.com/NixOS/nixpkgs/pull/191540
2022-09-28 18:34:36 +02:00
Robert Hensing 7f0d934f9a
Merge pull request #191540 from hercules-ci/nixosTest-modular
nixosTest: make modular
2022-09-28 10:27:45 +01:00
figsoda dc4b44a2f3 nixos/lib/make-single-disk-zfs-image.nix: remove unused binding 2022-09-26 22:19:06 -04:00
figsoda 84bc9ff334 nixos/lib/make-multi-disk-zfs-image.nix: remove unused binding 2022-09-26 22:18:55 -04:00
Robert Hensing 1c0b9c4a48 nixos/testing/network.nix: Add network config to specialisations 2022-09-24 17:42:52 +01:00
Robert Hensing 7cdc9bc340 nixos/testing: Improve interactive docs 2022-09-24 17:42:52 +01:00
Robert Hensing 666e969da0 nixos/testing: Add nodes.config backcompat to nodes module argument 2022-09-24 17:38:10 +01:00
Robert Hensing 6205d37747 nixos/testing: Improve option docs 2022-09-24 17:38:10 +01:00
Robert Hensing 4ce93fbae8 nixos/testing-python: Add interactive variant support to makeTest 2022-09-24 17:38:10 +01:00
Robert Hensing 52bfa318e8 nixos/testing: Support mypy through regular mechanisms
Rebase / forward port of 2c8bbf33fd
2022-09-24 17:38:10 +01:00
Robert Hensing 0af6e6b0e5 nixos/testing/meta.nix: Add options, some optional 2022-09-24 17:38:09 +01:00
Robert Hensing f01fec4099 nixos/testing/network.nix: Fix specialisations onlyShorthand 2022-09-24 17:38:09 +01:00
Robert Hensing 9886db059a nixos/testing: Embrace callTest
My conception of its input was wrong. It is quite a useful construct,
even if its name is a bit weird.
2022-09-24 17:38:09 +01:00
Robert Hensing 5297d584bc nixos/lib/eval-config: Document the use of baseModules 2022-09-24 17:38:09 +01:00
Robert Hensing 57ac9dd3ae nixos/lib/build-vms.nix: Remove
It has been replaced by the modular test framework in nixos/lib/testing.

If you are looking for a way to produce a VM-test-like configuration
outside of the test framework, use the nixos/lib/testing/nixos-test-base.nix
NixOS module, possibly in combination with { _module.args.nodes = .....; }.
2022-09-24 17:38:09 +01:00
Robert Hensing 24d1d74e4e nixos/testing: Extract nixos-test-base.nix NixOS module 2022-09-24 17:38:09 +01:00
Robert Hensing 38fb09e427 testing-python.nix: Replace makeTest implementation 2022-09-24 17:38:08 +01:00
Robert Hensing b0c781cc41 nixos/testing: Move entrypoint to nixos/lib + doc 2022-09-24 17:38:08 +01:00
Robert Hensing 9e4277b970 nixos/testing/network.nix: Fix for specialisations and specialArgs.name
Before this, it relied on being able to `imports` from the `name`
module argument, which wasn't really necessary and required a
potentially quite breaking change. We can revert that now.
2022-09-21 10:55:12 +01:00
Robert Hensing a958a4aa00 nixos/testing: Add pkgs parameter
This parameter is for packages to use in VMs, unlike hostPkgs.
2022-09-21 10:55:11 +01:00
Robert Hensing 124b0c4abc nixos/testing/network.nix: Avoid deprecated .config 2022-09-21 10:55:11 +01:00
Robert Hensing b3de22483c nixos/testing-python.nix: Add evalTest
This is a decomposition of the testing-python.nix and build-vms.nix
files into modules.

By refactoring the glue, we accomplish the following:

 - NixOS tests can now use `imports` and other module system features.
    - Network-wide test setup can now be reusable; example:
       - A setup with all VMs configured to use a DNS server
       - Split long, slow tests into multiple tests that import a
         common module that has most of the setup.
    - Type checking for the test arguments
    - (TBD) "generated" options reference docs
 - Aspects that had to be wired through all the glue are now in their
   own files.
    - Chief example: interactive.nix.
    - Also: network.nix

In rewriting this, I've generally stuck as close as possible to the
existing code; copying pieces of logic and rewiring them, without
changing the logic itself.

I've made two exceptions to this rule

 - Introduction of `extraDriverArgs` instead of hardcoded
   interactivity logic.

 - Incorporation of https://github.com/NixOS/nixpkgs/pull/144110
   in testScript.nix.

I might revert the latter and split it into a new commit.
2022-09-21 10:55:11 +01:00
Robert Hensing 5b52d552cb nixos/make-options-doc: Explain docbook to markdown migration 2022-09-17 13:38:23 +01:00
pennae 767485a0de lib/options: deprecate docbook text and literalDocBook
deprecate literalDocBook by adding a warning (that will not fire yet) to
its uses and other docbook literal strings by adding optional warning
message to mergeJSON.
2022-09-10 18:23:13 +02:00
Jacek Galowicz 8ae3e986a7
Merge pull request #189999 from sbruder/test-driver-formatting-closed-port
nixos/test-driver: fix formatting of closed port
2022-09-06 14:48:13 +02:00
Simon Bruder 9caf5f6ba9
nixos/test-driver: fix formatting of closed port
Co-authored-by: a-kenji <aks.kenji@protonmail.com>
2022-09-06 14:32:56 +02:00
Alyssa Ross 488412a1db
nixos/test-driver: add wait_for_unit() timeout arg
For example, the wait_for_unit() call in the Moodle test times out for
myself and others[1], so it would be good to be able to increase it to
something less likely to be hit by a test that would otherwise pass.

[1]: https://github.com/NixOS/nixpkgs/pull/177052#issue-1266336706
2022-09-02 18:06:03 +00:00
pennae 3bddcf5f90
Merge branch 'master' into option-docs-md 2022-09-01 16:10:09 +02:00
pennae 9547123258 nixos/*: convert internal option descriptions to MD
we'll have to do it eventually, may as well be now.
2022-08-31 16:32:54 +02:00
pennae ef176dcf7e nixos/*: automatically convert option descriptions
conversions were done using https://github.com/pennae/nix-doc-munge
using (probably) rev f34e145 running

    nix-doc-munge nixos/**/*.nix
    nix-doc-munge --import nixos/**/*.nix

the tool ensures that only changes that could affect the generated
manual *but don't* are committed, other changes require manual review
and are discarded.
2022-08-31 16:32:53 +02:00
pennae 5a643387ec nixos/*: normalize more docbook for auto-conversion 2022-08-31 16:32:53 +02:00
pennae 9c3c13b50d nixos/make-options-doc: add inline roles for varname/envar
both of these render distinctly from plain literals in the manpage, and
manpages even semantically distinguish between the two.
2022-08-31 16:21:10 +02:00
Silvan Mosberger 6eb620ace7
Merge pull request #180222 from YorikSar/qemu-vm-darwin-pkgs
nixos/qemu-vm: Allow to build and run VMs on Darwin
2022-08-30 21:54:22 +02:00
pennae 65fd6f0774 nixos/make-options-doc: eat newlines in MD admonitions
leaving some newlines around after an admonition was closed causes the
newline rule to match, which in turn inserts literallayout newlines into
te xml output. that's not what we want.
2022-08-27 19:18:29 +02:00
Yuriy Taraday e0fd131310 nixos/qemu: Add QEMU arguments for HVF on Darwin
Only support the same architechture for now, no emulation.

Related to #108984.
2022-08-25 17:00:14 +04:00
Yuriy Taraday 8a25d56a6d nixos/qemu: nixpkgs-fmt nixos/lib/qemu-common.nix 2022-08-25 17:00:11 +04:00
Ivan Shapovalov ac18c55b56 make-system-tarball: use pixz -t
Pass `-t` to pixz to prevent it from appending an index to the end of
the uncompressed stream, confusing tools such as `machinectl import-tar`.

Fixes: #187816
2022-08-22 05:28:36 +03:00
Christian Kögler 5e7662d848
Merge pull request #167692 from peperunas/onsuccess
nixos/systemd: Add onSuccess module option for units
2022-08-19 09:17:14 +02:00
Robert Schütz 243053e521 python310Packages.mistune: 0.8.4 -> 2.0.4
Remove mistune_0_8 because it's insecure.
2022-08-15 06:53:01 +00:00
pennae af98bacbe0 Revert "nixos/docs: cache mergeJSON md conversion on baseOptionsJSON"
This reverts commit 52b0ad17e3.

we only needed this because mergeJSON was slow, but in the interim we
found a better solution to the slowness.
2022-08-05 17:13:47 +02:00
pennae 645cfa59ac nixos/make-option-docs: add xref support to markdown conversion 2022-08-03 22:01:14 +02:00
pennae 7a091b2686 nixos/make-options-doc: reuse markdown instance in mergeJSON
this doesn't construct a new (expensive) parser for every option, making
rendering about 30x faster.
2022-08-03 22:00:24 +02:00
pennae 6b13dd0e9e
Merge pull request #183491 from pennae/automatic-md-conversions
treewide: automatically md-convert option descriptions
2022-08-02 02:15:30 +02:00
Silvan Mosberger fe2c9ae814
Merge pull request #177012 from hercules-ci/nixos-modular-system
`flake.lib.nixosSystem`: Allow `nixpkgs.system` to be set modularly; improve error message
2022-08-01 15:05:16 +02:00
Alyssa Ross 5330c0a1af
treewide: use isAarch where appropriate 2022-07-30 16:18:27 +00:00
pennae 52b0ad17e3 nixos/docs: cache mergeJSON md conversion on baseOptionsJSON
with ever more options being markdown rather than docbook the conversion
time is starting to become a significant factor of doc build time.
luckily we can pre-convert all nixos option docs to MD and cache the
result of this conversion, then merge the already-converted json file
with user option docs. we leave options.json unconverted to keep it as
close to the actual nix code as possible.
2022-07-28 23:20:02 +02:00
pennae 18be724a58 nixos/make-options-doc: give MD conversion error locations
during docs conversion it can be very useful to know exactly *where* the
error the script complained about is. the name of the option should be
sufficient since option merging is rather rare, and won't merge doc
attributes anyway.
2022-07-28 20:03:02 +02:00
Philippe Schaaf 5ae6580474 add hub mode comment
Signed-off-by: Philippe Schaaf <philippe.schaaf@secunet.com>
2022-07-21 16:34:09 +02:00
Philippe Schaaf f6a290932e use vde switch in hubmode by default
Within a dual VM test-setup a strange behaviour was observed.
The two VMs are connected via one vde_switch instance
(instancevirtualisation.vlans = [ 1 ]; IMO a bad attribute name for
switch instances, has nothing to do with VLANs in sense of 802.1Q).

A ping on the base interface (eth1) works, but not on VLAN
subinterfaces (vlan1@eth1). A tcpdump of eth1 includes the ARP requests
tagged with the subinterfaces VLAN ID, but responses seems not to pass
the vde_switch. This works fine if performed on the base interface.

Putting the vde_switch in hub mode results in flooding
traffic to all vde_switch ports. This results in a expected behaviour
and a ping on a VLAN subinterface works as expected.

Signed-off-by: Philippe Schaaf <philippe.schaaf@secunet.com>
2022-07-21 11:38:03 +02:00
Robert Hensing 711e653a65 nixos/eval-config: Allow system to be set modularly when arg is null 2022-07-10 13:35:54 +02:00
Martin Weinelt c0d1de14b3
Merge pull request #179291 from dotlambda/utillinux-alias
treewide: don't use utillinux alias
2022-07-02 00:05:54 +02:00
Robert Hensing 7c81905344 nixos/make-options-doc: Support Nix-provided declaration locations
Feature was introduced in https://github.com/NixOS/nixpkgs/pull/174460,
but wasn't supported in `mergeJSON.py` yet.
2022-06-27 22:07:20 +02:00
Robert Hensing 8bff3fef40 nixos/make-options-doc: Support block quotes
Our tooling would trip without the inner <para>, despite the docbook
docs suggesting that <para> occurs in <blockquote> and vice versa.
2022-06-27 17:41:32 +02:00
Robert Hensing e04aa1bcd9 nixos/make-options-doc: Escape inline code and code blocks 2022-06-27 17:41:32 +02:00
Robert Hensing f900ed1749 nixos/make-options-doc: Support newline md node
This occurs in the ast generated for blockquotes.
2022-06-27 17:41:32 +02:00
Robert Hensing bccc3e747b nixos/make-options-doc: Fix exception handler for arity /= 1 methods 2022-06-27 17:41:32 +02:00
Robert Hensing aff2dbbc82 make-options-doc: Make variablelist id configurable
I've tried XInclude set-xml-id first, but our tooling did not pick up on it.
2022-06-27 17:41:32 +02:00
Robert Schütz 40990c3586 treewide: don't use utillinux alias 2022-06-27 04:55:23 +00:00
Robert Hensing fa9c83ca7f
Merge pull request #174460 from hercules-ci/module-docs-Nix-driven-location-links
make-options-doc: Support Nix-provided declaration links
2022-06-22 15:48:46 +02:00
Lassulus 773cbfbd28
Merge pull request #151633 from wucke13/sd-image-experiment
honor `sdImage.compressImage` also in the build process
2022-06-22 14:48:21 +02:00
Robert Hensing e2c261f2c0
Merge pull request #176146 from pennae/module-docs-markdown
treewide: markdown option docs
2022-06-21 13:16:02 +02:00
Jörg Thalheim f5ca531054
Merge pull request #158602 from hercules-ci/systemd-unit-mixed-list-error
nixos/systemd: Custom error when mixing list/non-list defs
2022-06-19 07:05:04 +01:00
Robert Hensing 2c8bbf33fd nixos/test-driver: Support mypy through regular mechanisms 2022-06-17 12:22:57 +02:00
Robert Hensing cee66a8cd5 make-options-doc: Support Nix-provided declaration links
Previously, the location logic was hardcoded, supporting only
Nixpkgs and NixOps properly, leaving other uses of the module
system without good location support.
2022-06-15 00:45:05 +02:00
pennae 320aa2a791 treewide: attempt at markdown option docs 2022-06-12 12:44:38 +02:00
Robert Hensing 9970c0b874
Merge pull request #174470 from hercules-ci/module-docs-visibility-override
make-options-doc: Allow transformOptions to override visibility
2022-06-10 14:27:58 +02:00
Robert Hensing bad676c7ed
Merge pull request #174441 from tljuniper/172325-nixostest-override-python-pkgs-additional-param
nixos/test-driver: additional Python packages in test driver
2022-06-05 20:46:01 +02:00
Bobby Rong 9d2a8e3190
nixos/test-driver: fix type hint for send_chars 2022-06-04 12:05:47 +08:00
Robert Hensing 793180cf55
Merge branch 'master' into 172325-nixostest-override-python-pkgs-additional-param 2022-06-03 14:22:13 +02:00
Michael Schneider fd4ebd8990 add opt out flag for type checking 2022-06-03 13:37:04 +02:00
Michael Schneider 814027378b nixos/test-driver: Typecheck TestScript 2022-06-02 10:05:45 +02:00
Robert Hensing 53326189ba
Merge pull request #174829 from blaggacao/fix/cleanup-nixpkgs-initial-system
nixos: remove effect-less nixpgks.initialSystem
2022-05-31 16:26:25 +02:00
tljuniper a99736e399 nixos/test-driver: add option to add extra python packages to test code 2022-05-29 11:22:23 +02:00
Alyssa Ross 1e396f9802
nixos/testing-python.nix: fix typo in warning
"pyton" -> "python"
2022-05-27 13:00:25 +00:00
David Arnold 646e214e11
nixos: remove effect-less nixpgks.initialSystem
- initialSystem was keeping track of the evaluating system
- it had been used by `nesting.children`
- since, 20.09, `nesting.children` has been replaced with named
  specializations

It appears that this option was left over and not cleand up properly.
2022-05-26 20:00:05 -05:00
Robert Hensing 75bc6da237 make-options-doc: Filter options after transformOptions
This allows the user-supplied function to change the visibility
of options.
2022-05-25 12:50:07 +02:00
Robert Hensing 90344e9f40
Merge pull request #171163 from hercules-ci/nixpkgs-config-doc
Add generated `nixpkgs.config` doc to Nixpkgs manual
2022-05-19 20:51:38 +02:00
Robert Hensing 0b02135d3b nixosOptionsDoc: refactor
Thanks to Infinisil for pointing this out.
2022-05-16 22:53:23 +02:00
Robert Hensing 062bc5e74a lib.types.functionTo: Add pseudo-attr to generated docs 2022-05-13 09:01:05 +02:00
Robert Hensing ae172a2bb4 treewide: nixosTest -> testers.nixosTest 2022-05-09 14:16:06 +02:00
Robert Hensing 28f99aad31 nixos/testing-python.nix: Set meta.mainProgram 2022-05-05 12:33:57 +02:00
Robert Hensing bb2c5a3684 nixosOptionsDoc: Make appendix tag optional 2022-05-01 21:51:19 +02:00
Patrick Hilhorst 976f00963f
nixos/test-driver: rewrite wait_before_entry to wait 2022-04-29 15:36:03 +02:00
Will Fancher 69d8047516 nixos: Fix up systemd shutdown ramfs 2022-04-28 14:12:14 -04:00
Patrick Hilhorst 1db0dcdfec
nixos/test-driver: add wait_before_entry to polling_condition 2022-04-27 16:23:33 +02:00
Patrick Hilhorst 57beeedbdf
nixos/test-driver: allow multiple entry of polling condition 2022-04-27 16:21:10 +02:00
Anna Gillert dbc95f15b8 nixos/test-driver: Avoid shell injection in machine.execute() 2022-04-21 10:54:00 +02:00
Anna Gillert f7e89a59da nixos/test-driver: fix missing shellopts in execute
Without this fix, setting the shellopts in `machine.execute` is
inconsitent.  When no timeout is used, shellopts `set -euo pipefail` are
applied to the command as expected. When a timeout is specified, the
shellopts are not applied to the command itself (which is called inside
a `sh -c` that doesn't inherit the shellopts) but rather to the
`timeout` command, leading to the following full command:

```bash
(set -euo pipefail; timeout 900 sh -c 'cmd') | (base64 --wrap 0; echo)\n
```

With this fix, this is the command we get:

```bash
timeout 900 sh -c 'set -euo pipefail; false | true') | (base64 --wrap 0; echo)\n
```
2022-04-21 10:54:00 +02:00
Graham Christensen d65eff4fb6
Merge pull request #167902 from DeterminateSystems/openstack-zfs-cleanup
openstack-image: init, make-single-disk-zfs-image: init
2022-04-16 19:44:53 -04:00