These features are internal-only, have special characters that bash
doesn't support in variable names, and aren't normally given
environment variables by cargo as far as I can tell.
This commit fixes precise dependency ignorance by converting the
environment variable `autoPatchelfIgnoreMissingDeps` into a bash array
`ignoreMissingDepsArray`, passing `"${ignoreMissingDepsArray[@]}"`
instead of `"${autoPatchelfIgnoreMissingDeps[@]}"` to the python
script.
The original implementation does not work when
`autoPatchelfIgnoreMissingDeps` contains multiple dependency names.
Because it mistakenly passes `"${autoPatchelfIgnoreMissingDeps[@]}"`
to the python script. According to the Nix manual
(https://nixos.org/manual/nix/stable/expressions/derivations.html),
lists of strings are concatenated into whitespace-separated strings,
then passed to the builder as environment variables. So, if
`autoPatchelfIgnoreMissingDeps = [ "dep1" "dep2" "dep3" ]`,
`"${autoPatchelfIgnoreMissingDeps[@]}"` will be expanded to a single
argument `"dep1 dep2 dep3"`, which is not the intended behavior,
because the python script takes the long argument as a dependency
name.
With this commit, `"${ignoreMissingDepsArray[@]}"` will be expanded to
three arguments `"dep1" "dep2" "dep3"` arguments as expected, fixing
the issue.
cpio includes the number of directory hard links in archives it creates.
Some filesystems, like btrfs, do not count directory hard links the same
way as more common filesystems like ext4 or tmpfs, so archives built
when /tmp is on such a filesystem do not reproduce. This patch replaces
cpio with bsdtar, which does not have this issue. The specific
invocation is from this page:
https://reproducible-builds.org/docs/archives/
before
$ nix build ".#whipper.tests.version"
whipper> File "/nix/store/2iiyy58pmm1ys6dy8ycbmmmfm67iakv1-whipper-0.10.0/bin/.whipper-wrapped", line 6, in <module>
whipper> File "/nix/store/2iiyy58pmm1ys6dy8ycbmmmfm67iakv1-whipper-0.10.0/lib/python3.9/site-packages/whipper/__init__.py", line 5, in <module>
note: keeping build directory '/tmp/nix-build-whipper-0.10.0-test-version.drv-7'
after
$ nix build ".#whipper.tests.version"
whipper> Traceback (most recent call last):
whipper> File "/nix/store/2iiyy58pmm1ys6dy8ycbmmmfm67iakv1-whipper-0.10.0/bin/.whipper-wrapped", line 6, in <module>
whipper> from whipper.command.main import main
whipper> File "/nix/store/2iiyy58pmm1ys6dy8ycbmmmfm67iakv1-whipper-0.10.0/lib/python3.9/site-packages/whipper/__init__.py", line 5, in <module>
whipper> from pkg_resources import (get_distribution,
whipper> ModuleNotFoundError: No module named 'pkg_resources'
note: keeping build directory '/tmp/nix-build-whipper-0.10.0-test-version.drv-34'
error: builder for '/nix/store/5lxjicdhwgmjcz9ddlxgq3s3gyaa6lz4-whipper-0.10.0-test-version.drv' failed with exit code 1;
Desktop files are only useful when accompanied by the binaries they
specify. So it makes more sense to put them next to the binaries rather
than `$out` which only usually contains the binaries.
This simplifies usages and makes the default value consistent.
In a few cases, the default value was interpreted to be `false`,
but this is useless, because virtually nobody will explicitly
set `allowAliases = true;`.
Similar to the implementation of the `%f` and `%u` field codes. In this
case the amount of arguments passed poses no problem but the position
could, at least in theory.
This finishes the implementation of all the non-deprecated field codes.
As a part of that, repetitions of field codes are left alone. Originally
all field codes were removed. Now we replace only the first occurence.
This is correct for at least `%f`, `%u`, `%F` and `%U` because at most
one of them is permitted.
Shortcomings:
1. We replace `%[cfFikuU]` patterns one at a time. This means if the
right field code appears as part of the rest of the `Exec` field or
in a field code that was substituted earlier.
2. If any field code is repeated, only the first occurence is
substituted.
`%f` and `%u` are used to signal the program only accepts a single file
or URI argument. I do not believe there's a way to signal this
information to macOS but it is possible the program really won't work if
multiple files are passed and it's possible the relative position of
`%i`, `%c` or `%k` matters. So we replace `%f` or `%u` with `$1`. That
way we only pass one file in the (possibly significant) position of the
field code.
`ls -1 "$iconsdir/"*` listed the source directory for me when the glob
had no matches. Switching to `-A` circumvents this problem and has the
added advantage that it cannot run into argument list length limits.
Checked the desktop entry spec, there's other field codes than `%[fFuU]`
and those can in fact occur more than once, hence dropping '$' and
adding `/g`.
The "Exec" key in desktop items sometimes has one of the `%f`, `%F`,
`%u` and `%U` suffixes, which specify whether the command takes a file,
multiple files or a generalized URL or URLs. Darwin application bundles
do no understand this syntax so we do the next best thing, which is
simply dropping it.
Allows restricting patches to a specific subdirectory, à la
`git diff --relative=subdir`.
This cannot be done (cleanly) currently because the `includes` logic
happens *after* `stripLen` is applied, so we can't match on `subdir/*`.
This change adds a `relative` argument that makes this possible by
filtering files before doing any processing, and setting `stripLen` and
`extraPrefix` accordingly.
This reverts commit 848091a52b, reversing
changes made to ab0e692ac7.
It caused issues with elfutils tests,
probably through over-stripping of glibc parts.
After being rewritten in https://github.com/NixOS/nixpkgs/pull/149731, this hook
can fail on Mach-O binaries. Since patching ELF files on Darwin doesn't make
much sense anyway, we'll mark this as Linux-exclusive.
Sometimes scalable icons or icons within the thresholds from the desired
resolutions aren't available. In this case it's still nicer to end up
with a blocky scaled icon rather than the generic default.
In order to compose a `.icns` file containing multiple icon sizes I had
to pass `--toc` to `icnsutil`. This did not seem to have a negative
effect on `.icns` containing only a single icon size.
On macOS 10.13 the 48x48 icon size is not supported. It results in a
corrupted image being displayed. I suspect the image data is being
truncated to what it expects for 32x32 or maybe data is read for
128x128, which would be a buffer overflow.
Older macOS cannot interpret the `CFBundleIconFiles` key so we cannot
rewrite the `CFBundleIconFile` entry without special consideration. I
opted to fix this by inverting the squircle logic. We always add both
the `CFBundleIconFile` and `CFBundleIconFiles` keys. The former is
necessary for at least macOS 10.13 and probably 10.12. The latter seems
to be ignored on those versions and overrides the former on newer
versions of macOS.
Inverting the logic also allows us to rely on the `toPlist` generator to
generate the XML syntax, which is a nice bonus.
Some crates do not have a Cargo.toml at the top-level, but only in
nested directories. Before this change importCargoLock used to fail with:
error: manifest path `/nix/store/some-store-path/Cargo.toml` does not exist
Conflict in pkgs/development/libraries/libvirt/default.nix
required manual adjustments. The fetched patch is already in src.
I checked that libvirt builds.
As discussed in https://github.com/NixOS/nixpkgs/issues/163590, it's not really required and has a side effect of adding refeferences to packages from nativeBuildInputs that aren't really required
It is the package list output that should be sorted. The current
output sequence is not logical because '.' is sorted in front of '/'
with input sorting and is not deterministic for different language
environments.
To get a deterministic and logical sorting in environments with
different language configured, env "LC_ALL=C" is set and the produced
strings is that is sorted. To get alphabetic sorting and not a pure
ASCII value sort "--ignore-case" is added.
This reverts commit 0b1856bfe3.
This broke one of our clients projects where a local libary could no
longer be found. Since there is no easy way to disable it's better
if this flag is set per project using `dotnetRestoreFlags`.
Uses fsharp interactive (fsi) to run fsx script
Expose mkNugetSource and mkNugetDeps functions
Use them in writers.writeFSharp and buildDotnetModule
Add tests
pkgs.writeTextFile doesn't like spaces in filenames.
This is fixed in #162246, but other tools will probably dislike
spaces in .desktop file names as well, and that PR will take forever,
so let's fix this anyway.
If bash is executed within an environment where PATH is not set, it uses
the DEFAULT_PATH_VALUE compiled into bash to set PATH. In nixpkgs we set
this to /no-such-path by default. This makes sense in a nixpkgs/NixOS
environment since paths like /bin or /usr/bin should not be used.
However, when bash is used inside an FHS environment, this produces
results that differ from distributions which follow the FHS standard.
Before this change:
$ steam-run env -i /bin/bash -c 'echo $PATH'
/no-such-path
After this change:
$ steam-run env -i /bin/bash -c 'echo $PATH'
/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
This updates makeDesktopItem to explicitly support all the fields in the spec,
converts list-like fields to native Nix lists instead of semicolon-separated strings,
and allows automatically generating [Desktop Action] sections from Nix code
instead of hardcoding them as extraConfig strings.
To keep this for the future we also strictDeps where possible, including for janePackages, topkg, oasis and ocamlbuild.
This makes some closures significantly smaller and makes cross compilation easier
- Convert icons to a single .icns file; and
- Provide an opt-out via X-macOS-Squircle in the desktop item to
override the squircle behavior when the source icons look bad when
converted automatically.
In https://github.com/NixOS/nixpkgs/issues/158042 I noticed order
mismatch as a bootstrap build failure when building x86_64-linux
against glibc-2.35 in nixpkgs (bootstrap libs has glibc-2.27):
expand-response-params> ld: /nix/store/p4s4jf7aq6v6z9iazll1aiqwb34aqxq9-bootstrap-tools/lib/crt1.o: in function `_start':
expand-response-params> /build/glibc-2.27/csu/../sysdeps/x86_64/start.S:101: undefined reference to `__libc_csu_fini'
expand-response-params> ld: /build/glibc-2.27/csu/../sysdeps/x86_64/start.S:102: undefined reference to `__libc_csu_init'
expand-response-params> collect2: error: ld returned 1 exit status
Here crt1.o from glibc-2.27 links against libc.so.6 from glibc-2.35.
This happens because ordering of `-L` (influences `libc.so` lookup) and
`-B` (influences `crt1.o` lookup) flags differs:
expand-response-params> -B/...-bootstrap-tools/lib
expand-response-params> -B/...-glibc-2.35/lib/
...
expand-response-params> -L/...-glibc-2.35/lib
expand-response-params> -L/...-bootstrap-tools/lib
The change makes consistent ordering of `-L`/`-B` and allows getting to
stage4 for `glibc-2.35` target.
According to https://stackoverflow.com/q/46197810/115030,
--only-keep-debug preserves all the information stripped by
--strip-unneeded. This reduces the size of the webkitgtk output by 22%
(123 MB → 96 MB).
Inspired by #159612.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Disable file globbing in --prefix/--suffix, since bash will otherwise
try to find filenames matching the the value to be prefixed/suffixed
if it contains characters considered wildcards, such as `?` and
`*`. We want the value as is, except we also want to split it on on
the separator; hence we can't quote it.
This should speed up restore times a fair bit, especially for bigger
projects. Roslyn also has it enabled by default already, so I don't
expect any breakages from it.
This is a much more flexible way of doing things, as we can adopt and
reuse these hooks for various tasks. Syntax highlighting now also works
way better for me, which is a nice bonus :)
* rewrite autoPatchelfHook in python
* Update pkgs/build-support/setup-hooks/auto-patchelf.py
Co-authored-by: aszlig <aszlig@redmoonstudios.org>
* Update pkgs/build-support/setup-hooks/auto-patchelf.py
Co-authored-by: aszlig <aszlig@redmoonstudios.org>
* Apply suggestions from code review
Co-authored-by: aszlig <aszlig@redmoonstudios.org>
* Fix issues discovered during tests
* Apply suggestions from code review
Co-authored-by: aszlig <aszlig@redmoonstudios.org>
* fixup line wrapping
* autoPatchelfHook: Improve compatibility with bash version
* autoPatchelfHook: Fix symlink-reated issues
* autoPatchelfHook: Revert dubious patchelf invocation test
* autoPatchelfHook: Untangle the executable detection logic
* fixup! autoPatchelfHook: Untangle the executable detection logic
* autoPatchelfHook: Fix invalid borrow issue
* autoPatchelfHook: Handle runtimeDependencies as the bare string it is
* autoPatchelfHook: add bintools dependency
For the very rare cases where it is not included by default.
* autoPatchelfHook: replace old hook with the rewrite
* autoPatchelfHook: get rid of the old hook content
* autoPatchelfHook: fix wrong ordering of debug info
* autoPatchelfHook: persist extra search path across incovations
* autoPatchelfHook: fix wrong usage of global variables
* Update auto-patchelf.py
PEP8: ignoreMissing -> ignore_missing
* Apply suggestions from code review
Co-authored-by: aszlig <aszlig@redmoonstudios.org>
* autoPatchelfHook: remove imprecise and incorrect warning
* Apply explicit types from code review
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
* Complement and polish types and snake_casing
Co-authored-by: aszlig <aszlig@redmoonstudios.org>
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
When prefixing or suffixing list variables, check that the value or
values aren't already part of the list. If this is the case when
suffixing, the list won't be touched at all. When prefixing, however,
the last matching instance of the value will be moved to the beginning
of the list. Any remaining duplicates of the value will be left as-is.
Co-authored-by: Vincenzo Mantova <xworld21@users.sf.net>
Previously, when sha256 either wasn't defined or set to an empty string
fetchpatch would error out as follows:
'''
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
...
/nix/store/agwlk2bcfvz2ggrsbvwd7696qj55frbi-stdenv-linux/setup: line 96: /build/: Is a directory
sed: couldn't flush stdout: Broken pipe
'''
This patch makes it show fetchurl's error message instead:
'''
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
...
error: hash mismatch in fixed-output derivation:
specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
got: sha256-NWGWoyEgT/ztCwbhNgGPvG+nqX4bxtFnD+wds6fklbs=
'''
This is very convenient for TOFU.
Co-Authored-By: Ivar Scholten <ivar.scholten@protonmail.com>
This is useful for a use-case we have with a Nix-based CI system that
specifies things like deploy steps as passthru attributes[0].
Previously the only way to do this would have been to concatenate
attributes onto the resulting derivation, but passing them in and
actually treating them as proper passthru attributes is cleaner.
[0]: https://cs.tvl.fyi/depot@f7d7da6aceb407b719cf4683a75878fd3aca319e/-/blob/nix/buildkite/default.nix?L222-226
This was pretty straightforward, note that go1.17 is explicitly required
by v0.5.1, and one of the tests requires git, so I added it to the
checkInputs.
Also the tests now pass and don't need the mangle, so I removed the
patch. I left the darwin/aarch64 patch in.
The `nix.*` options, apart from options for setting up the
daemon itself, currently provide a lot of setting mappings
for the Nix daemon configuration. The scope of the mapping yields
convience, but the line where an option is considered essential
is blurry. For instance, the `extra-sandbox-paths` mapping is
provided without its primary consumer, and the corresponding
`sandbox-paths` option is also not mapped.
The current system increases the maintenance burden as maintainers have to
closely follow upstream changes. In this case, there are two state versions
of Nix which have to be maintained collectively, with different options
avaliable.
This commit aims to following the standard outlined in RFC 42[1] to
implement a structural setting pattern. The Nix configuration is encoded
at its core as key-value pairs which maps nicely to attribute sets, making
it feasible to express in the Nix language itself. Some existing options are
kept such as `buildMachines` and `registry` which present a simplified interface
to managing the respective settings. The interface is exposed as `nix.settings`.
Legacy configurations are mapped to their corresponding options under `nix.settings`
for backwards compatibility.
Various options settings in other nixos modules and relevant tests have been
updated to use structural setting for consistency.
The generation and validation of the configration file has been modified to
use `writeTextFile` instead of `runCommand` for clarity. Note that validation
is now mandatory as strict checking of options has been pushed down to the
derivation level due to freeformType consuming unmatched options. Furthermore,
validation can not occur when cross-compiling due to current limitations.
A new option `publicHostKey` was added to the `buildMachines`
submodule corresponding to the base64 encoded public host key settings
exposed in the builder syntax. The build machine generation was subsequently
rewritten to use `concatStringsSep` for better performance by grouping
concatenations.
[1] - https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md
When I designed `mkShell`, I didn't have a good idea of what the output
should look like and so decided to make the build fail. In practice,
this causes quite a bit of confusion and complications because now the
shell cannot be part of a normal package set without failing the CI as
well.
This commit changes that build phase to record all the build inputs in a
file. That way it becomes possible to build it, makes sure that all the
build inputs get built as well, and also can be used as a GC root.
(by applying the same trick as #95536).
The documentation has also been improved to better describe what mkShell
does and how to use it.
macOS's dyld can be rather picky as to what dylib it accepts. This
even changes across macOS versions. Therefore we now build a fat
dylib with all three architectures (x86_64, arm64, arm64e). This
should then be compatible with pretty much any macOS's dyld.
The tools built with buildGraalVmNativeImage have broken UTF-8 support
when not properly setting the locale, e.g.
$ bb -e '(prn "bépo àê")'
"b??po ????"
This commit sets the locale to en_US.UTF-8 by default, which fixes that.
