Commit graph

52 commits

Author SHA1 Message Date
pennae dc895fb281 lib: make extendDerivation lighter on eval
the fix to extendDerivation in #140051 unwittingly worsened eval performance by
quite a bit. set elements alone needed over 1GB extra after the change, which
seems disproportionate to how small it was. if we flip the logic used to
determine which outputs to install around and keep a "this one exactly" flag in
the specific outputs instead of a "all of them" in the root we can avoid most
of that cost.
2021-10-15 16:39:10 +02:00
Peter Ferenczy 1ec54d2fba
buildenv: fix regression introduced by #134215 2021-08-20 15:47:47 +03:00
Peter Ferenczy f8be06a311 buildEnv: better warning and error messages 2021-08-17 18:16:20 +02:00
sternenseemann b398d00903 trivial-builders: merge passAsFile with env in runCommand'
Previously it was awkward to use the runCommand-variants with
passAsFile as a double definition of passAsFile would potentially
break runCommand: passAsFile would overwrite the previous definition,
defeating the purpose of setting it in runCommand in the first place.
This is now fixed by concatenating the [ "buildCommand" ] list with
one the one from env, if present.

Adjust buildEnv where passAsFile = null; was passed in some cases,
breaking evaluation since it'd evaluate to [ "buildCommand" ] ++ null.
2021-03-18 19:56:40 +01:00
John Ericson 4f6ec19dbc buildEnv: Support nativeBuildInputs too
Since #112276, we should always put `makeWrapper` in
`nativeBuildInputs`. But `buildEnv` was saying put it in `buildInputs`.
That's wrong!

Fix the instructions, and make the right thing possible.
2021-03-02 22:34:54 +00:00
José Romildo Malaquias 4a1ef63c55 buildenv: paths to link resolving to the same absolute path is not a conflict
When building an environment if two paths conflict but one or both are
symbolic links and they resolve to the same real path, the conflict is
discarded because the contents of both paths are the same. One of them
is chosen and there is no need to recur into them in order to build
deeper symbolic links.
2020-06-10 06:42:48 -03:00
Linus Heckemann 3125ab3afe buildEnv: check pathsToLink before checking collisions
Fixes #32683
2019-09-18 15:13:50 +02:00
Vladimír Čunát 2adf954e84
Merge #55420: buildEnv, substituteAll: disable binary cache
... into staging.
2019-02-22 13:57:01 +01:00
Maximilian Bosch 9fcd3bffc2 buildEnv: improve file check to avoid false-positives
The original change in #55372 was supposed to fix the case where a store
path which is a file should be placed into `buildEnv` which broke with a
fairly misleading Perl error.

Unfortunately this introduced a regression, `findFiles` can have targets
that are files if the file isn't a store path. Rather than adding more
obscure checks with probably further regressions, I figured that it's
better to replicate the behavior of `lib.isStorePath` and explicitly
check if the store path is a file and break in this case only.

This should also fix recent staging issues.
2019-02-21 10:37:42 +01:00
Maximilian Bosch 1cab56e01a buildEnv: break with a proper error if one path is actually a file
I noticed by creating `buildEnv` where I accidentally put a derivation
from `pkgs.writeText` into `paths` and got a broken build with the
following misleading error message:

```
Use of uninitialized value $stat1 in numeric ne (!=) at /nix/store/9g4wc31j7a2xp22xpgwr0qssfxahxdzl-builder.pl line 74.
Use of uninitialized value $stat1 in bitwise and (&) at /nix/store/9g4wc31j7a2xp22xpgwr0qssfxahxdzl-builder.pl line 75.
different permissions in `' and `/nix/store/0vy5ss91laxvwkyvrbld5hv27i88qk5w-noise': 0000 <-> 0444 at /nix/store/9g4wc31j7a2xp22xpgwr0qssfxahxdzl-builder.pl line 75.
```

It can be reproduced with an expression like this:

``` nix
{ pkgs ? import <nixpkgs> { } }:
let
  file = pkgs.writeText "test" ''
    content
  '';
in
  pkgs.buildEnv {
    name = "test-env";
    paths = [ /* ... */ file ];
  }
```
2019-02-19 07:56:44 +01:00
volth 09665fee2b
buildEnv: allowSubstitutes = false
trivial derivation
2019-02-07 22:19:26 +00:00
Frederik Rietdijk 87f757e834 Merge pull request #27780 from tilpner/overridable-buildenv
Make buildEnv overridable
2017-09-14 17:26:33 +02:00
tilpner 77f1c7d5bc
buildenv: Make buildEnv overridable
callPackage already calls makeOverridable, but that just
makes the function that evaluates to buildEnv overridable,
not buildEnv itself.
If no overridable version of buildEnv is used during construction,
users can't override e.g. `paths` at all
2017-08-02 11:56:37 +02:00
Thomas Tuegel dce958ac39
buildenv: read propagated-user-env-packages line-by-line
Since 3cb745d5a6, the format of
propagated-user-env-packages has changed and propagated packages have not been
included by buildenv, including in the system environment.

The buildenv builder is modified to read propagated-user-env-packages
line-by-line, instead of expecting all packages on one line.
2017-07-16 16:06:06 -05:00
Eric Litak bd45ec6416 buildEnv: use buildPackages.perl so crossDrv works 2017-04-14 02:00:08 -07:00
Alexey Shmalko 0172558e82 buildEnv: build the whole tree of directories to pathsToLink
This patch fixes #16614 and #16741.

The first issue was caused by the fact that both `/share` and
`/share/fish/vendor_completions.d` end in the `pathsToLink`. The
`pkgs/build-support/buildenv/builder.pl` creates `/share`, then links
`/share/fish` under `/share` and then tries to create the directory
`/share/fish/vendor_completions.d` and fails because it already exists.

The simplest way to reproduce the issue is to build the next Nix
expression:

```nix
let pkgs = import <nixpkgs> { };
in pkgs.buildEnv {
  name = "buildenv-issue";

  paths = [
    pkgs.fish
    pkgs.vim
  ];

  pathsToLink = [
    "/share"
    "/share/fish/vendor_completions.d"
  ];
}
```

The second issue is more critical and was caused by the fact findFiles
doesn't recurse deep enough. It stops at first unique directory for the
package (e.g., "/share" or even "/") and later the scripts decides it
shouldn't link it as it doesn't match pathsToLink (e.g., "/share/fish"),
so the result is empty.

The test:
```nix
let pkgs = import <nixpkgs> { };
in pkgs.buildEnv {
  name = "buildenv-issue";

  paths = [
    pkgs.fish
    pkgs.vim
  ];

  pathsToLink = [
    "/share/fish/functions"
  ];
}
```

or

```nix
let pkgs = import <nixpkgs> { };
in pkgs.buildEnv {
  name = "buildenv-issue";

  paths = [
    pkgs.vim
  ];

  pathsToLink = [
    "/share"
  ];
}
```
2016-07-13 03:54:11 +03:00
Vladimír Čunát f57c6449dc buildEnv: fix #14682 evaluation in some edge cases
I supplied meta.outputsToInstall automatically in all
mkDerivation products, but some packages still don't use it.
The reported case: jekyll -> bundlerEnv -> buildEnv -> runCommand.
2016-04-17 08:57:17 +02:00
Vladimír Čunát 9a824f2f1d treewide: rename extraOutputs{ToLink,ToInstall}
This is to get more consistent with `meta.outputsToInstall`.
2016-04-07 15:59:44 +02:00
Vladimír Čunát 2995439003 buildEnv: respect meta.outputsToInstall
As a result `systemPackages` now also respect it.
Only nix-env remains and that has a PR filed:
    https://github.com/NixOS/nix/pull/815
2016-04-07 15:59:44 +02:00
Vladimír Čunát d6b46ecb30 Merge branch 'closure-size' into p/default-outputs 2016-03-14 11:27:15 +01:00
Vladimír Čunát ae74c356d9 Merge recent 'staging' into closure-size
Let's get rid of those merge conflicts.
2016-02-03 16:57:19 +01:00
aszlig ab84149c99
buildEnv: Pass the right number of args to addPkg
Regression introduced by 4529ed1259.

I've missed this in #5096, not because of a messed up rebase as I have
guessed from a comment on #12635 but missed this in the first place.

The testing I did while working on the pull request weren't exhaustive
enough to cover this, because I haven't tested with packages that use
the propagatedUserEnvPkgs attribute.

In order to make the test a bit more exhaustive this time, let's test it
using:

nix-build -E 'with import ./. {}; buildEnv {
  name = "testenv";
  paths = [
    pkgs.hello pkgs.binutils pkgs.libsoup pkgs.gnome3.yelp
    pkgs.gnome3.totem
  ];
}'

And with this commit the errors no longer show up and the environment is
built correctly.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Fixes: #12635
2016-01-28 14:50:44 +01:00
Vladimír Čunát ab8a691d05 nixos systemPackages: rework default outputs
- Now `pkg.outputUnspecified = true` but this attribute is missing in
  every output, so we can recognize whether the user chose or not.
  If (s)he didn't choose, we put `pkg.bin or pkg.out or pkg` into
  `systemPackages`.
- `outputsToLink` is replaced by `extraOutputsToLink`.
  We add extra outputs *regardless* of whether the user chose anything.
  It's mainly meant for outputs with docs and debug symbols.
- Note that as a result, some libraries will disappear from system path.
2016-01-28 11:24:18 +01:00
aszlig bfb11fd030
buildEnv: Skip content check on ignoreCollisions.
Checking file contents is redundant in this case, because we will go
ahead anyway, regardless of whether the content is the same.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-01-18 03:56:09 +01:00
aszlig 4529ed1259
buildEnv: Check the content of colliding paths.
Originally wanted to include ignoreCollisions in cups-progs, but I think
it's better if we use ignoreCollisions only if there are _real_
collisions between files with different contents.

Of course, we also check whether the file permissions match, so you get
a collision if contents are the same but the permissions are different.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-01-18 03:54:22 +01:00
Luca Bruno 5b0352a6a4 Merge branch 'master' into closure-size 2015-12-11 18:31:00 +01:00
Luca Bruno 31ed92f65f Fix system-path with multiout 2015-12-01 15:09:41 +01:00
Tuomas Tynkkynen 74c09a6e64 buildEnv: Allow setting meta attributes 2015-11-29 19:13:19 +02:00
William A. Kennington III 1496b0f6a1 Fix buildenv 2015-10-30 18:09:45 -07:00
William A. Kennington III dd2de66d61 Merge branch 'master.upstream' into staging.upstream 2015-10-30 17:16:07 -07:00
Eelco Dolstra 58e9440b89 Add option to link additional package outputs into system.path
This is necessary to get stuff like separate manpages, info files,
debug symbols, etc.
2015-10-30 15:21:02 +01:00
Nikolay Amiantov d49140f3e4 buildenv: support very long paths 2015-10-22 15:35:39 +03:00
Vladimír Čunát fa33b083eb buildEnv: add new parameters: extraPrefix and buildInputs 2015-09-17 17:43:18 +02:00
Eelco Dolstra e2a9541efc buildEnv: Fix handling of empty list of packages 2015-08-25 11:13:34 +02:00
Eelco Dolstra e4610f2965 buildEnv: Support package priorities like nix-env
This gets rid of a bunch of collision warnings.
2015-08-25 00:40:40 +02:00
Florian Friesdorf 84030ee1f0 buildEnv to support passthru 2014-07-10 16:41:02 +02:00
Eelco Dolstra 03f9026ac5 buildEnv: don't warn about collisions in propagated packages
This mimicks buildenv in Nix more closely.
2012-07-25 23:18:39 -04:00
Eelco Dolstra fa276c45dd * Set preferLocalBuild on a few more trivial builders.
svn path=/nixpkgs/trunk/; revision=33935
2012-04-26 15:17:43 +00:00
Eelco Dolstra a2102af6df * buildEnv: use $SHELL (i.e. bash) rather than /bin/sh. This allows
NixOS VM tests to be built on Ubuntu, where /bin/sh is dash rather
  than bash.

svn path=/nixpkgs/trunk/; revision=24592
2010-11-03 22:37:00 +00:00
Eelco Dolstra e875ec2524 * Make sure that if pathsToLink contains an element such as
"/share/info" that it doesn't match "/share/information.nix".

svn path=/nixpkgs/trunk/; revision=23059
2010-08-09 16:08:02 +00:00
Eelco Dolstra 4ed96dfbc2 * Ignore non-existant directories.
svn path=/nixpkgs/trunk/; revision=23026
2010-08-07 18:41:43 +00:00
Eelco Dolstra 3c824666fd * Remove the call to update-mime-database, which doesn't belong here
but in the postBuild hook.

svn path=/nixpkgs/trunk/; revision=22527
2010-07-08 12:55:03 +00:00
Eelco Dolstra a6fcf45726 * First figure out what symlinks need to be created, then create
them.  This prevents unnecessary unlink operations to resolve
  collisions between directories.

svn path=/nixpkgs/trunk/; revision=22526
2010-07-08 12:52:17 +00:00
Yury G. Kudryashov 99e825a42c Build MIME database from all available packages
svn path=/nixpkgs/trunk/; revision=21076
2010-04-14 19:26:50 +00:00
Yury G. Kudryashov 295af6ab17 Ignore fake collisions
svn path=/nixpkgs/trunk/; revision=21075
2010-04-14 19:26:31 +00:00
Yury G. Kudryashov 31d047a83f Remove tabs
svn path=/nixpkgs/trunk/; revision=21004
2010-04-10 20:27:09 +00:00
Marc Weber 349de9bafe revrting my accident only keeping rubygem updates and ruby libs
This partially reverts commit -r 16100

svn path=/nixpkgs/trunk/; revision=16107
2009-06-30 15:58:02 +00:00
Marc Weber d1e39e78f9 adding rubygems support to nix - implemented target libraries/apps: sup mail client
svn path=/nixpkgs/trunk/; revision=16100
2009-06-30 14:29:20 +00:00
Eelco Dolstra 2d89fb4fee * Support a post-install hook.
svn path=/nixpkgs/trunk/; revision=14142
2009-02-20 15:40:11 +00:00
Wouter den Breejen 5200276c3b nixpkgs dir: nvidia and libsrvg
svn path=/nixpkgs/trunk/; revision=8785
2007-05-28 14:10:46 +00:00