figsoda
202699c918
nixos/tests: fix typos
2023-05-19 22:31:04 -04:00
Jacek Galowicz
b7dfa5082a
Merge pull request #228220 from R-VdP/test_driver_guest_shell_timeout
...
nixos-test-driver: include a timeout for the recv call, do not assume sh == bash
2023-05-09 13:05:28 +02:00
r-vdp
4147b878bc
nixos-test-driver: include a timeout for the recv call, do not assume sh == bash
2023-04-26 01:05:53 +02:00
Alyssa Ross
d6e84a4574
nixosTest: remove hostname limitations
2023-04-25 08:55:30 +00:00
Alyssa Ross
27f97953e0
nixos/test-driver: accept non-\w* filenames
...
What the code was trying to do was helpfully add a directory and
extension if none were specified, but it did this by checking whether
the filename was composed of a very limited character set that didn't
even include dashes.
With this change, the intention of the code is clearer, and I can put
dashes in my screenshot names.
2023-03-22 15:54:06 +00:00
Astro
64a4ae6f37
nixos/test-driver: remove allow_reboot state from Machine, make it a start() kwarg
2023-03-16 01:50:15 +01:00
Astro
9fc85a85d5
nixos/test-driver: implement Machine.reboot()
2023-03-16 01:41:32 +01:00
Astro
95d121e798
nixos/test-driver: fix allow_reboot
2023-03-15 12:48:01 +01:00
rnhmjoj
067d688b16
nixos/test-driver: handle decoding errors in Machine.execute
...
The output of a command is not guaranteed to be valid UTF-8, so the
decoding can fail raising UnicodeDecodeError. If this happens during a
`succeeds` the check will be erroneously marked failed.
This changes the error handling to the "replace" mode, where invalid
codepoints are replaced with � (REPLACEMENT CHARACTER U+FFFD) and the
decoding can go on.
2023-02-07 08:47:14 +01:00
rnhmjoj
f2929eb949
nixos/test-driver: drop logging from Machine.send_monitor_command
...
Several machine operations, like `send_chars` and `send_key`, are
implemented by calling `send_monitor_command`, possibly multiple times.
This generates a huge amount of unnecessary noise in the log, because
`send_monitor_command` is a low-level operation and an implementation
detail.
Here's an excerpt from a highlighted log before and afte the change.
Before:
[1m[32msubtest: Can generate a PGP key[0m[0m
machine: [1m[32msending keys 'gpg --gen-key\n'[0m[0m
machine: [1m[32msending monitor command: sendkey g[0m[0m
machine: [1m[32mwaiting for monitor prompt[0m[0m
(finished: waiting for monitor prompt, in 0.00 seconds)
(finished: sending monitor command: sendkey g, in 0.00 seconds)
machine: [1m[32msending monitor command: sendkey p[0m[0m
machine: [1m[32mwaiting for monitor prompt[0m[0m
(finished: waiting for monitor prompt, in 0.00 seconds)
(finished: sending monitor command: sendkey p, in 0.00 seconds)
machine: [1m[32msending monitor command: sendkey g[0m[0m
machine: [1m[32mwaiting for monitor prompt[0m[0m
(finished: waiting for monitor prompt, in 0.00 seconds)
(finished: sending monitor command: sendkey g, in 0.00 seconds)
machine: [1m[32msending monitor command: sendkey spc[0m[0m
machine: [1m[32mwaiting for monitor prompt[0m[0m
(finished: waiting for monitor prompt, in 0.00 seconds)
(finished: sending monitor command: sendkey spc, in 0.00 seconds)
machine: [1m[32msending monitor command: sendkey 0x0C[0m[0m
machine: [1m[32mwaiting for monitor prompt[0m[0m
(finished: waiting for monitor prompt, in 0.00 seconds)
(finished: sending monitor command: sendkey 0x0C, in 0.00 seconds)
machine: [1m[32msending monitor command: sendkey 0x0C[0m[0m
machine: [1m[32mwaiting for monitor prompt[0m[0m
(finished: waiting for monitor prompt, in 0.00 seconds)
(finished: sending monitor command: sendkey 0x0C, in 0.00 seconds)
machine: [1m[32msending monitor command: sendkey g[0m[0m
machine: [1m[32mwaiting for monitor prompt[0m[0m
(finished: waiting for monitor prompt, in 0.00 seconds)
(finished: sending monitor command: sendkey g, in 0.00 seconds)
machine: [1m[32msending monitor command: sendkey e[0m[0m
machine: [1m[32mwaiting for monitor prompt[0m[0m
(finished: waiting for monitor prompt, in 0.00 seconds)
(finished: sending monitor command: sendkey e, in 0.00 seconds)
machine: [1m[32msending monitor command: sendkey n[0m[0m
machine: [1m[32mwaiting for monitor prompt[0m[0m
(finished: waiting for monitor prompt, in 0.00 seconds)
(finished: sending monitor command: sendkey n, in 0.00 seconds)
machine: [1m[32msending monitor command: sendkey 0x0C[0m[0m
machine: [1m[32mwaiting for monitor prompt[0m[0m
(finished: waiting for monitor prompt, in 0.00 seconds)
(finished: sending monitor command: sendkey 0x0C, in 0.00 seconds)
machine: [1m[32msending monitor command: sendkey k[0m[0m
machine: [1m[32mwaiting for monitor prompt[0m[0m
(finished: waiting for monitor prompt, in 0.00 seconds)
(finished: sending monitor command: sendkey k, in 0.00 seconds)
machine: [1m[32msending monitor command: sendkey e[0m[0m
machine: [1m[32mwaiting for monitor prompt[0m[0m
(finished: waiting for monitor prompt, in 0.00 seconds)
(finished: sending monitor command: sendkey e, in 0.00 seconds)
machine: [1m[32msending monitor command: sendkey y[0m[0m
machine: [1m[32mwaiting for monitor prompt[0m[0m
(finished: waiting for monitor prompt, in 0.00 seconds)
(finished: sending monitor command: sendkey y, in 0.00 seconds)
machine: [1m[32msending monitor command: sendkey ret[0m[0m
machine: [1m[32mwaiting for monitor prompt[0m[0m
(finished: waiting for monitor prompt, in 0.00 seconds)
(finished: sending monitor command: sendkey ret, in 0.00 seconds)
(finished: sending keys 'gpg --gen-key\n', in 0.15 seconds)
After:
[1m[32msubtest: Can generate a PGP key[0m[0m
machine: [1m[32msending keys 'gpg --gen-key\n'[0m[0m
(finished: sending keys 'gpg --gen-key\n', in 0.15 seconds)
2023-02-07 08:40:54 +01:00
Jörg Thalheim
29db54c373
nixos/tests: extend shell_interact to accept alternative socat addresses
...
`shell_interact()` is currently not nice to use. If you try to cancel
the socat process, it will also break the nixos test. Furthermore
ptpython creates it's own terminal that subprocesses are running in,
which breaks some of the terminal features of socat.
Hence this commit extends `shell_interact` to allow also to connect to
arbitrary servers i.e. tcp servers started by socat.
2023-01-27 13:09:25 +01:00
Guillaume Girol
33afbf39f6
treewide: switch to nativeCheckInputs
...
checkInputs used to be added to nativeBuildInputs. Now we have
nativeCheckInputs to do that instead. Doing this treewide change allows
to keep hashes identical to before the introduction of
nativeCheckInputs.
2023-01-21 12:00:00 +00:00
Jacek Galowicz
869545857f
Merge pull request #208354 from bjornfor/nixos-test-driver-quote-shell-args
...
nixos/test-driver: quote some shell command lines
2022-12-30 22:41:04 +01:00
Bjørn Forsman
c876f0c146
nixos/test-driver: quote some shell command lines
...
This makes them robust against paths with whitespace.
2022-12-30 13:19:59 +01:00
Bjørn Forsman
4c45c3f8f2
nixos/test-driver: use ASCII single quotes everywhere
...
A few places used Unicode U+2018/U+2019 left/right single quotes (but
not always correctly balanced). Let's just use plain ASCII single quotes
everywhere.
2022-12-30 13:00:19 +01:00
Bjørn Forsman
ce5644f658
nixos/test-driver: use f-strings instead of .format()
...
For readability.
Suggested-by: @tfc
2022-12-29 23:51:35 +01:00
Bjørn Forsman
dba49a43a0
nixos/test-driver: add optional address arg to wait_for_{open,closed}_port
...
This is useful for testing servers configured to listen on specific
addresses.
2022-12-29 13:50:17 +01:00
Jacek Galowicz
9938dec682
Merge pull request #170636 from Synthetica9/wait_before_entry
...
nixos/test-driver: add wait_before_entry
2022-12-23 19:46:23 +01:00
Zhaofeng Li
8b2d34fa5e
test-driver: Allow configuring delay for send_{key,chars}
2022-11-19 06:55:25 +01:00
Jacek Galowicz
8ae3e986a7
Merge pull request #189999 from sbruder/test-driver-formatting-closed-port
...
nixos/test-driver: fix formatting of closed port
2022-09-06 14:48:13 +02:00
Simon Bruder
9caf5f6ba9
nixos/test-driver: fix formatting of closed port
...
Co-authored-by: a-kenji <aks.kenji@protonmail.com>
2022-09-06 14:32:56 +02:00
Alyssa Ross
488412a1db
nixos/test-driver: add wait_for_unit() timeout arg
...
For example, the wait_for_unit() call in the Moodle test times out for
myself and others[1], so it would be good to be able to increase it to
something less likely to be hit by a test that would otherwise pass.
[1]: https://github.com/NixOS/nixpkgs/pull/177052#issue-1266336706
2022-09-02 18:06:03 +00:00
Philippe Schaaf
5ae6580474
add hub mode comment
...
Signed-off-by: Philippe Schaaf <philippe.schaaf@secunet.com>
2022-07-21 16:34:09 +02:00
Philippe Schaaf
f6a290932e
use vde switch in hubmode by default
...
Within a dual VM test-setup a strange behaviour was observed.
The two VMs are connected via one vde_switch instance
(instancevirtualisation.vlans = [ 1 ]; IMO a bad attribute name for
switch instances, has nothing to do with VLANs in sense of 802.1Q).
A ping on the base interface (eth1) works, but not on VLAN
subinterfaces (vlan1@eth1). A tcpdump of eth1 includes the ARP requests
tagged with the subinterfaces VLAN ID, but responses seems not to pass
the vde_switch. This works fine if performed on the base interface.
Putting the vde_switch in hub mode results in flooding
traffic to all vde_switch ports. This results in a expected behaviour
and a ping on a VLAN subinterface works as expected.
Signed-off-by: Philippe Schaaf <philippe.schaaf@secunet.com>
2022-07-21 11:38:03 +02:00
Robert Hensing
2c8bbf33fd
nixos/test-driver: Support mypy through regular mechanisms
2022-06-17 12:22:57 +02:00
Robert Hensing
bad676c7ed
Merge pull request #174441 from tljuniper/172325-nixostest-override-python-pkgs-additional-param
...
nixos/test-driver: additional Python packages in test driver
2022-06-05 20:46:01 +02:00
Bobby Rong
9d2a8e3190
nixos/test-driver: fix type hint for send_chars
2022-06-04 12:05:47 +08:00
Robert Hensing
793180cf55
Merge branch 'master' into 172325-nixostest-override-python-pkgs-additional-param
2022-06-03 14:22:13 +02:00
Michael Schneider
814027378b
nixos/test-driver: Typecheck TestScript
2022-06-02 10:05:45 +02:00
tljuniper
a99736e399
nixos/test-driver: add option to add extra python packages to test code
2022-05-29 11:22:23 +02:00
Patrick Hilhorst
976f00963f
nixos/test-driver: rewrite wait_before_entry to wait
2022-04-29 15:36:03 +02:00
Patrick Hilhorst
1db0dcdfec
nixos/test-driver: add wait_before_entry to polling_condition
2022-04-27 16:23:33 +02:00
Patrick Hilhorst
57beeedbdf
nixos/test-driver: allow multiple entry of polling condition
2022-04-27 16:21:10 +02:00
Anna Gillert
dbc95f15b8
nixos/test-driver: Avoid shell injection in machine.execute()
2022-04-21 10:54:00 +02:00
Anna Gillert
f7e89a59da
nixos/test-driver: fix missing shellopts in execute
...
Without this fix, setting the shellopts in `machine.execute` is
inconsitent. When no timeout is used, shellopts `set -euo pipefail` are
applied to the command as expected. When a timeout is specified, the
shellopts are not applied to the command itself (which is called inside
a `sh -c` that doesn't inherit the shellopts) but rather to the
`timeout` command, leading to the following full command:
```bash
(set -euo pipefail; timeout 900 sh -c 'cmd') | (base64 --wrap 0; echo)\n
```
With this fix, this is the command we get:
```bash
timeout 900 sh -c 'set -euo pipefail; false | true') | (base64 --wrap 0; echo)\n
```
2022-04-21 10:54:00 +02:00
rnhmjoj
ca1b1f6dc0
nixos/test-driver: highlight driver log lines
...
There is a whole lot of noise in a NixOS test log due to journal,
commands, and test driver messages all being mixing together.
With this commit the test driver messages are highlighted so you don't
have to squint too much to see where a subtest starts and ends or what
was the last command being run.
Here's an excerpt from a highlighted log:
[2mmachine # [ 6.647826] dhcpcd[668]: eth0: adding default route via 10.0.2.2[0m
[2mmachine # I've had a great day![0m
(finished: must succeed: curl -sfL https://example.com/how-are-you ? | tee /dev/stderr, in 0.19 seconds)
(finished: subtest: Privoxy can filter https requests, in 0.22 seconds)
[1m[32msubtest: Blocks are working[0m
machine: [1m[32mwaiting for TCP port 443[0m
[2mmachine # Connection to localhost (::1) 443 port [tcp/https] succeeded![0m
(finished: waiting for TCP port 443, in 0.03 seconds)
machine: [1m[32mmust fail: curl -f https://example.com/ads 1>&2[0m
[2mmachine # [ 6.730608] nscd[742]: 742 monitored file `/etc/resolv.conf` was written to[0m
[2mmachine # [ 6.733071] privoxy[814]: 2022-04-14 09:41:01.695 7fa995c39640 Actions: +block{Fake ads} ...
[2mmachine # curl: (22) The requested URL returned error: 403[0m
[2mmachine # [ 6.753098] systemd[1]: Stopping Name Service Cache Daemon...[0m
[2mmachine # [ 6.755632] systemd[1]: nscd.service: Deactivated successfully.[0m
[2mmachine # [ 6.757069] systemd[1]: Stopped Name Service Cache Daemon.[0m
(finished: must fail: curl -f https://example.com/ads 1>&2, in 0.07 seconds)
machine: [1m[32mmust succeed: curl -f https://example.com/PRIVOXY-FORCE/ads 1>&2[0m
[2mmachine # [ 6.765099] systemd[1]: Starting Name Service Cache Daemon...[0m
[2mmachine # [ 6.781519] nscd[925]: 925 monitoring file `/etc/passwd` (1)[0m
[2mmachine # [ 6.844105] systemd[1]: Reached target Network is Online.[0m
[2mmachine # [ 6.848860] privoxy[814]: 2022-04-14 09:41:01.805 7fa995c39640 Actions: +block{Fake ads} ...
[2mmachine # [ 6.856068] nscd[925]: 925 monitoring file `/etc/nsswitch.conf` (8)[0m
[2mmachine # [ 6.858229] systemd[1]: Reached target Multi-User System.[0m
[2mmachine # [ 6.858934] nscd[925]: 925 monitoring directory `/etc` (2)[0m
[2mmachine # [ 6.861315] systemd[1]: Startup finished in 2.374s (kernel) + 4.451s (userspace) = 6.826s.[0m
[2mmachine # Hot Nixpkgs PRs in your area. Click here![0m
[2mmachine # [ 6.873055] nscd[925]: 925 monitoring directory `/etc` (2)[0m
[2mmachine # [ 6.874489] nscd[925]: 925 monitoring file `/etc/nsswitch.conf` (8)[0m
[2mmachine # [ 6.875599] nscd[925]: 925 monitoring directory `/etc` (2)[0m
(finished: must succeed: curl -f https://example.com/PRIVOXY-FORCE/ads 1>&2, in 0.12 seconds)
(finished: subtest: Blocks are working, in 0.22 seconds)
[1m[32msubtest: Temporary certificates are cleaned[0m
machine: [1m[32mmust succeed: test $(ls /run/privoxy/certs | wc -l) -gt 0[0m
(finished: must succeed: test $(ls /run/privoxy/certs | wc -l) -gt 0, in 0.02 seconds)
machine: [1m[32mmust succeed: date -s "$(date --date '12 days')"[0m
(finished: must succeed: date -s "$(date --date '12 days')", in 0.02 seconds)
[2mmachine # [ 6.959589] systemd[1]: Started Logrotate Service.[0m
[2mmachine # [ 6.966685] systemd[1]: Starting Cleanup of Temporary Directories...[0m
[2mmachine # [ 6.974783] systemd[1]: logrotate.service: Deactivated successfully.[0m
[2mmachine # [ 7.004493] systemd[1]: systemd-tmpfiles-clean.service: Deactivated successfully.[0m
[2mmachine # [ 7.005457] systemd[1]: Finished Cleanup of Temporary Directories.[0m
machine: [1m[32mmust succeed: test $(ls /run/privoxy/certs | wc -l) -eq 0[0m
(finished: must succeed: test $(ls /run/privoxy/certs | wc -l) -eq 0, in 0.01 seconds)
(finished: subtest: Temporary certificates are cleaned, in 0.15 seconds)
(finished: run the VM test script, in 7.44 seconds)
test script finished in 7.48s
[1m[32mcleanup[0m
kill machine (pid 8)
2022-04-14 11:54:21 +02:00
Janne Heß
7d0e21c77e
nixos/test-runner: Allow writing to qemu stdin
2022-03-25 12:08:21 +01:00
Viktor Kleen
6d766ae8b7
nixos/test-driver: deduplicate VLANs
...
Naively deduplicate VLANs in the python driver for NixOS tests. The
current implementation accidentally works, since the VLan class mutates
the environment. On construction it sets QEMU_VDE_SOCKET_${id} and this
environment variable gets overwritten once a second VLAN with the same
id is constructed. Because the NIC flags passed to qemu just use the
QEMU_VDE_SOCKET_${id} environment variable, this implicitly chooses a
single vde_switch process for each VLAN.
However, this leads to unusable vde_switch processes being spawned in
each test run and as a side effect makes it impossible to access the
correct VLan objects in the interactive test driver. It also makes it
remarkably hard to understand why the current implementation ever
worked.
2022-03-22 13:16:28 +00:00
Daniel Fullmer
24fb872079
nixos/test-driver: fix regression when using legacy start command
...
The aarch64-linux versions of the boot.uefiUsb and boot.uefiCdrom tests
were broken by b0fc9da879
.
That commit was a refactor which omitted the qemuBinary option, which was
previously available in the legacy start command. This restores that
option and fixes the tests previously mentioned.
2022-02-17 18:12:24 -08:00
Lassulus
b80607af67
Merge pull request #154804 from Mic92/tests-prompt
...
nixos/tests: add prompt to shell_interact()
2022-02-02 18:50:46 +01:00
Marijan Petričević
411bb3c35f
add --no-interactive flag
2022-01-26 11:27:01 +01:00
Marijan Petričević
446c21fdc7
factor out tmp-dir checks
2022-01-25 11:31:25 +01:00
Marijan Petričević
03c90974a7
add tmp_dir access check
2022-01-25 11:14:57 +01:00
Marijan Petričević
e17fcbc966
introduce writeable_dir argparse type
2022-01-25 10:21:33 +01:00
Marijan Petričević
7765670c8a
make output_directory an optional parameter
2022-01-25 01:18:52 +01:00
Marijan Petričević
f9b5f9dba7
nixos/test-driver: use an argument instead of the out env-var
2022-01-25 01:17:29 +01:00
Jörg Thalheim
f472e52aa6
nixos/tests: add prompt to shell_interact()
...
Example session:
>>> start_all()
>>> machines[0].shell_interact()
client: waiting for the VM to finish booting
client: connected to guest root shell
client: (connecting took 0.00 seconds)
(finished: waiting for the VM to finish booting, in 0.05 seconds)
client: Terminal is ready (there is no initial prompt):
$ ls -la
total 39
drwxrwxrwt 11 root root 4096 Jan 15 06:06 .
drwxr-xr-x 17 root root 4096 Jan 15 06:06 ..
drwxrwxrwt 2 root root 4096 Jan 15 06:06 .font-unix
drwxrwxrwt 2 root root 4096 Jan 15 06:06 .ICE-unix
drwx------ 2 root root 2 Jan 12 20:19 shared
drwx------ 3 root root 4096 Jan 15 06:06 systemd-private-b8f21699ea684491926859758de41975-nscd.service-txgYVZ
drwx------ 3 root root 4096 Jan 15 06:06 systemd-private-b8f21699ea684491926859758de41975-systemd-logind.service-U3GmlL
drwxrwxrwt 2 root root 4096 Jan 15 06:06 .Test-unix
drwxrwxrwt 2 root root 4096 Jan 15 06:06 .X11-unix
drwxr-xr-x 2 root root 2 Jan 15 06:06 xchg
drwxrwxrwt 2 root root 4096 Jan 15 06:06 .XIM-unix
2022-01-15 07:07:06 +01:00
Patrick Hilhorst
793a2f50f1
nixos/test-driver: remove unused imports, add pylint unused-import check
2022-01-02 23:12:21 +01:00
Patrick Hilhorst
a2f5092867
nixos/test-driver: simplify logic, reduce interaction surface
2022-01-02 22:52:17 +01:00
Patrick Hilhorst
7830f000c5
nixos/test-driver: simplify coopmulti
2022-01-02 22:20:04 +01:00