Commit graph

5850 commits

Author SHA1 Message Date
github-actions[bot] f3a93620b1
Merge master into staging-next 2022-11-10 12:01:27 +00:00
Shamrock Lee 6386d79326 patchRcPath hooks: use the passthru argument 2022-11-10 09:27:40 +08:00
Shamrock Lee da31bd5673 patchRcPathBash, patchRcPathCsh, patchRcPathFish, patchRcPathPosix: init
Init patchRcPath hooks, which provides
utilities to patch shell scripts to be sourced by users.

Add test cases and documentation.
2022-11-10 09:23:52 +08:00
github-actions[bot] 020ea91183
Merge master into staging-next 2022-11-10 00:02:27 +00:00
John Ericson e94d54dd86 build-support/rust/lib: Add toTargetFamily
Taken from https://github.com/kolloch/crate2nix/pull/255/files, it
belongs in Nixpkgs not crate2nix.

I have been using that P.R. for a few months without incident.
2022-11-09 15:52:01 -05:00
github-actions[bot] d99020653d
Merge master into staging-next 2022-11-09 18:01:11 +00:00
Winter 16722903aa buildNpmPackage: init 2022-11-09 10:35:59 -05:00
github-actions[bot] e51ba60580
Merge master into staging-next 2022-11-07 00:03:30 +00:00
Yarny0 335a9083b0 patchPpdFilesHook: new setup hook for absolute executable paths
PostScript Printer Description (ppd) files
describe printer features and capabilities.
They are usually evaluated by CUPS to convert
print jobs into a format suitable for a printer.
The conversion is often accomplished by commands
or even short shell scripts inside the ppd files.

ppd files are included in many printer driver packages.
Their scripts sometimes refer to other executables;
some of them are more common (like `perl`),
others are more exotic (like `rastertohp`).
If an executable is called with its name alone,
the effects of the ppd file depend on whether
the executable is in the PATH of CUPS,
and on the executable's version.
If an executable is called with an absolut path
(like `/usr/bin/perl`), it won't work at all in NixOS.

The commit at hand adds a setup hook that uses
the `fixupPhase` to substitute certain executable's
invocations in pdd files with absolute paths.
To use it, add the hook to `nativeBuildInputs` and
provide a list of executable names in `ppdFileCommands`.
Each executable must be available in the
package that is being built, or in `buildInputs`.
The setup hook's script then looks for ppd files in
`share/cups/model` and `share/ppds` in each output,
and replaces executable names with their absolute paths.
If ppd files need to be patched in unorthodox locations or
the setup hook needs to be invoked manually for other reasons,
one may leave the list `ppdFileCommands` empty to
avoid automatic processing of ppd files, then call
the shell function `patchPpdFileCommands` directly.
Details are described in the file `patch-ppd-hook.sh`.

Notes on the motivation for this setup hook:

Most packages in nixpkgs that provide
ppd files do not patch those ppd files at all.
This is not fatal when the executables are just called
with their names since the user can add packages
with the executables to `services.printing.drivers`.
E.g. if the user adds `pkgs.perl`, then all ppd
files that invoke `perl` will work as expected.
Nevertheless, to make these ppd files independent of
their execution environment, command invocations should
be substituted with absolut paths into the nix store.
This is similar to patching shebang lines so scripts can be
called independently of having the interpreter in the PATH.

The hook script in this commit is meant to support new packages
`foomatic-db*` which will generate several thousands of
ppd files referencing a plethora of different executables.
During development of these packages, I realized that
it's quite hard to patch ppd files in a robust way.
While binary names like `rastertokpsl` seem to be sufficiently
unique to be patched with `sed`, names like `date` or `gs`
are hard to patch without producing "false positives",
i.e., coincidental occurences of the executable's name that do
*not* refer to the executable and should not be patched at all.
As this problem also affects other packages,
it seems reasonable to put a robust implementation
in its own setup hook so that other
packages can use it without much effort.

Notes on the implementation:

The ppd file format is far from trivial.
The basic structure are key-value pairs;
keys may occur multiple times.
Only a small subset of keys may contain
executable names or shell scripts in their values.
Some values may span multiple lines;
a linebreak might even occur in the middle of a token.
Some executable names also occur in other keys by accident
where they must not be substituted (e.g. `gs` or `date`).

It is necessary to provide the list of command
names that will be patched for two reasons:
ppd files often contain "tokens" that might look
like commands (e.g. "file" or "host") but aren't;
these would erroneously get patched.
Also, looking for everything that might be a command
would slow down the patching process considerably.

The implementation uses `awk` to detect
keys that might contain executable names;
only their values are treated for substitution.
This avoids most cases of "overzealous" substitutions.
Since values may span multiple lines,
`sed` alone (while faster than `awk`) cannot focus
its substitution capabilities on relevant keys.
An elaborate set of regular expressions further helps
to minimize the probability of "false positives".
Several tricks are employed to speed up `awk`.
Notably, relevant files are identified with
`grep` before `awk` is applied to those files only.
Note that the script probably cannot handle fancy command
names (like spaces or backslashes as part of the name).
Also, there are still edge cases that the script would
mistakenly skip, e.g. if a shell script contains a
line break in the middle of an executable's name;
although ppd files permit such constellations,
I have yet to see one.

ppd files may be gzipped.
The setup hook accepts gzipped ppd files:
It decompresses them, substitutes paths, then recompresses them.
However, Nix cannot detect substituted paths as
runtime dependencies in compressed ppd files.
To ensure substituted paths are propagated as
runtime dependencies, the script adds each substituted
path to the variable `propagatedBuildInputs`.
Since this might not be enough for multi-output packages,
those paths are also written directly to
`nix-support/propagated-build-inputs`.
See the comment in `patch-ppd-hook.sh` for details.

Finally, the setup hook comes with a small test that
probes some edge cases with an artificial ppd file.

References:

* https://www.cups.org/doc/spec-ppd.html
  * general ppd file specification
  * lists some keys that may contain
    executable names or shell scripts

* https://refspecs.linuxfoundation.org/LSB_4.0.0/LSB-Printing/LSB-Printing/ppdext.html
  * lists some keys that may contain
    executable names or shell scripts

* https://en.wikipedia.org/wiki/PostScript_Printer_Description#CUPS
  * lists the usual locations of ppd files
2022-11-06 16:19:22 +01:00
github-actions[bot] cfd5f0bddc
Merge staging-next into staging 2022-11-05 00:02:45 +00:00
Artturi 1cdecacb79
Merge pull request #198489 from Artturin/fetchgitlabremoteavvt
fetchFromGitLab: don't passthruAttrs fetchGit specific attrs to fetchUrl
2022-11-04 20:29:38 +02:00
Yureka c2eaaae50d cargoSetupHook: pass host config flags 2022-10-30 18:55:47 +01:00
Artturin 19d2bb4863 fetchFromGitLab: don't passthruAttrs fetchGit specific attrs to fetchUrl
if `fetchSubmodules = false` to 'fetchFromGitLab' then theres the
following error

error: anonymous function at /nix/store/9m8drnpifyl5qsx93g6ll2xw6wkps03z-source/pkgs/build-support/fetchurl/default.nix:41:1 called with unexpected argument 'fetchSubmodules'

       at /nix/store/9m8drnpifyl5qsx93g6ll2xw6wkps03z-source/pkgs/build-support/fetchzip/default.nix:36:1:

           35|
           36| fetchurl ((
             | ^
           37|   if (pname != "" && version != "") then
2022-10-29 19:12:07 +03:00
github-actions[bot] c45b1c2226
Merge staging-next into staging 2022-10-29 00:02:59 +00:00
github-actions[bot] f349a716a3
Merge master into staging-next 2022-10-29 00:02:26 +00:00
Anderson Torres f367ac90aa
Merge pull request #197997 from atorres1985-contrib/hummingbird
skawarePackages: reformat
2022-10-28 17:10:53 -03:00
John Ericson 0c09b3d17c
Merge pull request #197937 from danielbarter/cc-wrapper-hook
cc-wrapper: adding a cc-wrapper-hook to the cc-wrapper
2022-10-28 17:30:03 +01:00
AndersonTorres a76bd55090 skawarePackages: move all of them to development/skaware-packages 2022-10-28 11:30:49 -03:00
Vladimír Čunát 6890557c17
Revert "Merge #191724: cc-wrapper: remove duplicate include flags"
Let's redo this later, with more testing and less bugs.
This reverts commits 4f6e99870b and d700d8e8a2.
2022-10-28 10:28:12 +02:00
Daniel Barter 77bd639c4c cc-wrapper: adding a cc-wrapper-hook to the cc-wrapper 2022-10-26 09:33:43 -07:00
github-actions[bot] 81a6c5a54c
Merge staging-next into staging 2022-10-25 18:16:24 +00:00
Sandro Jäckel 0f386859f5
build-dotnet-module: don't end with exit code 1 when update was sucessfull
otherwise scripts might end prematurely
2022-10-25 16:12:22 +02:00
github-actions[bot] f332a666fb
Merge staging-next into staging 2022-10-25 12:02:09 +00:00
Vincent Laporte 8b11f0a513 ocamlPackages.buildOcaml: remove 2022-10-25 09:40:57 +02:00
github-actions[bot] 683aa50df6
Merge staging-next into staging 2022-10-23 06:08:06 +00:00
Anderson Torres 749231c9a2
Merge pull request #196681 from corngood/fix-nuget-to-nix
nuget-to-nix: fix warning when package is installed from sdk
2022-10-23 00:58:56 -03:00
github-actions[bot] d86d8a1a3d
Merge staging-next into staging 2022-10-20 00:05:50 +00:00
github-actions[bot] aac580f88f
Merge master into staging-next 2022-10-20 00:05:13 +00:00
Sandro 23d0060889
Merge pull request #192639 from ylecornec/ylecornec/buildBazelPackage-tests 2022-10-19 23:36:53 +02:00
github-actions[bot] e9d710f780
Merge staging-next into staging 2022-10-19 06:25:17 +00:00
github-actions[bot] cea1c3f0cd
Merge master into staging-next 2022-10-19 06:24:39 +00:00
Daniel Poelzleithner 9331c0c772 FHSEnv: export /etc/shells
/etc/shells is often read by programs to determine which shells should be
provided. Fixes problems with extensions in vscode-fhs
2022-10-18 22:59:26 -07:00
Stella fbf83cda37 build-fhs-userenv-bubblewrap: use -m not -f for readlink
This change will let more inexistent paths be passed along, avoiding extremely weird and hard to debug behavior. See https://github.com/containers/bubblewrap/issues/520 for what I personally encountered.
2022-10-18 22:50:59 -07:00
David McFarland abebf5f591 nuget-to-nix: fix warning when package is installed from sdk 2022-10-18 23:42:46 -03:00
github-actions[bot] 262bce30e8
Merge staging-next into staging 2022-10-17 00:04:53 +00:00
github-actions[bot] 18a6423900
Merge master into staging-next 2022-10-17 00:04:21 +00:00
Sergei Trofimovich 8480f40236
Merge pull request #191134 from Et7f3/remove-duplicate-code-cc-wrapper
cc-wrapper: remove duplicate C{C,XX}${role_post}
2022-10-16 21:02:04 +01:00
Maximilian Bosch a914b9460d
Merge pull request #193075 from Ma27/nextcloud-pkg-fix
fetchNextcloudApp: rewrite with fetchzip & applyPatches
2022-10-16 20:07:57 +02:00
Vladimír Čunát d700d8e8a2
stdenv cc-wrapper: revert a problematic change for non-Linux
The change from 0bea4a194f (PR #191724) broke stdenv bootstap
on *-darwin, and I didn't want to drop the *-linux binaries
that we have already.
2022-10-14 09:40:48 +02:00
Martin Weinelt c728598b84 Merge remote-tracking branch 'origin/staging-next' into staging 2022-10-13 23:29:04 +02:00
Robert Hensing 8fca2fd4bf
Merge pull request #141050 from koenw/dockertools_too_many_arguments
build-support: Fix error when building images with many layers
2022-10-13 21:45:10 +01:00
github-actions[bot] 7df9a7a84c
Merge staging-next into staging 2022-10-13 18:07:50 +00:00
Koen Wilde 2f5fcda329 build-support: Fix error when building images with many layers
When building a docker image using `dockertools.buildLayeredImage`, the
resulting image layers are passed to `jq` through the command line. When
building an image with too many layers this would exceed the maximum
command line argument length.

Hence, we store the list of layers in the Nix store and pass them to
`jq` as a file argument using `--slurpfile`.

Fixes #140908.
2022-10-13 16:27:13 +02:00
Shea Levy ee7722a7de
Merge pull request #100634 from fzakaria/faridzakaria/buildmaven-nix
buildMaven: Update buildMaven to pure Nix
2022-10-13 08:06:26 -04:00
Vladimír Čunát 00a757ed3f
Merge branch 'master' into staging 2022-10-13 08:27:55 +02:00
Artturi e66d2fd89d
Merge pull request #194256 from Artturin/treewides2 2022-10-13 00:08:01 +03:00
lassulus 85a36f9146 writers.writeJS: pass arguments to script 2022-10-11 20:16:37 +02:00
github-actions[bot] 5d957f3dba
Merge staging-next into staging 2022-10-11 00:05:26 +00:00
Martin Weinelt 85713b7e37 Merge remote-tracking branch 'origin/master' into staging-next 2022-10-11 00:18:04 +02:00
pacien 0495b0217b writers.writeFish: avoid loading user config files
This makes the Fish interpreter ignore user configuration files.
This was not desirable for running scripts, especially system ones.
2022-10-10 23:10:36 +02:00