From d33e52b2539c5b36a5a876df9006a7145efd42ea Mon Sep 17 00:00:00 2001
From: Klemens Nanni <klemens@posteo.de>
Date: Sun, 15 May 2022 01:00:37 +0200
Subject: [PATCH] nixos/stage-1: Fix library path in libraries also

`extra-utils` composes the set of programs and libraries needed by

1. copying over all programs
2. copying over all libraries any program directly links against
3. set the runtime path for every program to the library directory

It seems that this approach misses the case where a library itself links
against another library.  That is to say, `extra-utils` assumes that
either only progams link against libraries or that every library linked
to by a library is already linked to by a program.

`mount.zfs` linking against `libcrypto`, in turn linking against `libdl`
shows how the current approach falls short:

```
$ objdump -p $(which mount.zfs) | grep NEEDED | grep -e libdl -e libcrypto
  NEEDED               libcrypto.so.1.1
$ ldd (which mount.zfs) | grep libdl
        libdl.so.2 => /nix/store/ybkkrhdwdj227kr20vk8qnzqnmj7a06x-glibc-2.34-115/lib/libdl.so.2 (0x00007f9967a9a000
```

Using `mount.zfs` directly in stage 1 init still works since
`LD_LIBRARY_PATH` overrides this (as intended).

util-linux's `mount` however executes `mount.zfs` with LD_LIBRARY_PATH
removed from its environment as can be seen with strace(1) in an
interactive stage 1 init shell (`boot.shell_on_fail` kernel parameter):

```
 # env -i LD_LIBRARY_PATH=$LD_LIBRARY_PATH $(which strace) -ff -e trace=/exec -v -qqq $(which mount) /mnt-root
execve("/nix/store/3gqbb3swgiy749fxd5a4k6kirkr2jr9n-extra-utils/bin/mount", ["/nix/store/3gqbb3swgiy749fxd5a4k"..., "/mnt-root"], ["LD_LIBRARY_PATH=/nix/store/3gqbb"...]) = 0
[pid  1026] execve("/sbin/mount.zfs", ["/sbin/mount.zfs", "<redacted>", "/mnt-root", "-o", "rw,zfsutil"], []) = 0
/sbin/mount.zfs: error while loading shared libraries: libdl.so.2: cannot open shared object file: No such file or directory
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=1026, si_uid=0, si_status=127, si_utime=0, si_stime=0} ---
```

env(1) is used for clarity (hence subshells for absoloute paths).

While `mount` uses the right library path, `mount.zfs` is stripped of
it, so ld.so(8) fails resolve `libdl` (as required by `libcrypto`).

To fix this and not rely on `LD_LIBRARY_PATH` to be set, fix the library
path inside libraries as well.

This finally mounts all ZFS filesystems using `zfsutil` with correct and
intended mount options.
---
 nixos/modules/system/boot/stage-1.nix | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/nixos/modules/system/boot/stage-1.nix b/nixos/modules/system/boot/stage-1.nix
index ec2bd5ef352..2900919b4b6 100644
--- a/nixos/modules/system/boot/stage-1.nix
+++ b/nixos/modules/system/boot/stage-1.nix
@@ -231,6 +231,11 @@ let
         patchelf --set-interpreter $out/lib/ld*.so.? --set-rpath $out/lib $i || true
       done
 
+      find $out/lib -type f \! -name 'ld*.so.?' | while read i; do
+        echo "patching $i..."
+        patchelf --set-rpath $out/lib $i
+      done
+
       if [ -z "${toString (pkgs.stdenv.hostPlatform != pkgs.stdenv.buildPlatform)}" ]; then
       # Make sure that the patchelf'ed binaries still work.
       echo "testing patched programs..."