When expanding path variables, kpathsea uses a trick to speed up identifying
leaf directories. The manual says
> The trick is that in every real Unix implementation (as opposed to the POSIX
> specification), a directory which contains no subdirectories will have
> exactly two links (namely, one for . and one for ..). That is to say, the
> st_nlink field in the ‘stat’ structure will be two. Thus, we don’t have to
> stat everything in the bottom-level (leaf) directories—we can just check
> st_nlink, notice it’s two, and do no more work.
>
> But if you have a directory that contains a single subdirectory and 500
> regular files, st_nlink will be 3, and Kpathsea has to stat every one of those
> 501 entries. Therein lies slowness.
>
> You can disable the trick by undefining ST_NLINK_TRICK in
> kpathsea/config.h. (It is undefined by default except under Unix.)
This does not work as expected with nixpkgs symlink trees and programs that rely
on kpathsea path expansion do not work properly.
One example is luaotfload's font database, which is populated by the font files
in the directories obtained by path-expanding the value of the `OPENTYPEFONTS`
configuration variable with kpathsea. The expanded value can be checked with
`kpsewhich --show-path="opentype fonts"`. Before this change, the expanded
value does not include the various font directories symlinked into
`/texmf/fonts/opentype/public`, since kpathsea considers this a leaf
directory (every child is a symlink, not a directory). Hence luaotfload does
not find the fonts in the texlive installation.
This patch disables this trick.
Co-authored-by: Robin Gloster <mail@glob.in>
stdenv: print message if structuredAttrs is enabled
stdenv: add _append
reduces the chance of a user doing it wrong
fix nix develop issue
output hooks don't work yet in nix develop though
making $outputs be the same on non-structuredAttrs and structuredAttrs
is too much trouble.
lets instead make a function that gets the output names
reading environment file '/nix/store/2x7m69a2sm2kh0r6v0q5s9z1dh41m4xf-xz-5.2.5-env-bin'
nix: src/nix/develop.cc:299: std::string Common::makeRcScript(nix::ref<nix::Store>, const BuildEnvironment&, const Path&): Assertion `outputs != buildEnvironment.vars.end()' failed.
use a function to get all output names instead of using $outputs
copy env functionality from https://github.com/NixOS/nixpkgs/pull/76732/commits
The code and typical use case suggest that ghostscript is not used
graphically. Removing this dependency reduces the build closure of
many tools that use texlive for documentation.
If this needs to be reverted, please state the graphical use case
in a comment, so we don't make this mistake again.
Passing `-l$NIX_BUILD_CORES` improperly limits the overall system load.
For a build machine which is configured to run `$B` builds where each
build gets `total cores / B` cores (`$C`), passing `-l $C` to make will
improperly limit the load to `$C` instead of `$B * $C`.
This effect becomes quite pronounced on machines with 80 cores, with
40 simultaneous builds and a cores limit of 2. On a machine with this
configuration, Nix will run 40 builds and make will limit the overall
system load to approximately 2. A build machine with this many cores
can happily run with a load approaching 80.
A non-solution is to oversubscribe the machine, by picking a larger
`$C`. However, there is no way to divide the number of cores in a way
which fairly subdivides the available cores when `$B` is greater than
1.
There has been exploration of passing a jobserver in to the sandbox,
or sharing a jobserver between all the builds. This is one option, but
relatively complicated and only supports make. Lots of other software
uses its own implementation of `-j` and doesn't support either `-l` or
the Make jobserver.
For the case of an interactive user machine, the user should limit
overall system load using `$B`, `$C`, and optionally systemd's
cpu/network/io limiting features.
Making this change should significantly improve the utilization of our
build farm, and improve the throughput of Hydra.
depsBuildBuild for 'ar: command not found'
BUILDCC for failure during 'checking whether the C compiler works...'
there's still a issue but it maybe wont cause reverse dependency issues?
so overall its still an improvement
```
cd texk/texlive/tl_support && make run-texlinks
make[1]: Entering directory '/build/texlive-20210325-source/WorkDir/texk/texlive/tl_support'
if test -s /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/texlinks && test -s /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/share/texmf-dist/web2c/fmtutil.cnf; then \
echo "running texlinks in `pwd` via: /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/texlinks -v -f /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/share/texmf-dist/web2c/fmtutil.cnf -e "" /nix/stor>
PATH="/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin:/nix/store/146229z6ingpiz0g1yxs2jpq1h546ms3-gcc-wrapper-11.3.0/bin:/nix/store/sc50c3qvzlw6dmfbhxw96j9yrj47pv1v-gcc-11.3.0/bin:/nix/store/20navlfxxadg2qvkqh3vc55z27divi40-glibc-2.35-163>
else :; fi
running texlinks in /build/texlive-20210325-source/WorkDir/texk/texlive/tl_support via: /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/texlinks -v -f /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/s>
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpsewhich: line 1: syntax error: unexpected word (expecting ")")
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 1: ^?ELF^B^A^A^B<B7>^A<80>: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 1: ^]: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 1: ^]A<B0>^B<B8>^B^A^B^Fp^]p^]Ap^]A^P^B^P^B^H^D^D<A8>^B<A8>^B@<A8>^B@: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 1: ^]A: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 2: @@<E0><9A>@8: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 5: syntax error: unterminated quoted string
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin: no write permissions. Skipping...
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 1: ^?ELF^B^A^A^B<B7>^A<80>: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 1: ^]: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 1: ^]A<B0>^B<B8>^B^A^B^Fp^]p^]Ap^]A^P^B^P^B^H^D^D<A8>^B<A8>^B@<A8>^B@: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 1: ^]A: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 2: @@<E0><9A>@8: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 5: syntax error: unterminated quoted string
make[1]: Leaving directory '/build/texlive-20210325-source/WorkDir/texk/texlive/tl_support'
/nix/store/j00iibma3wqn88f8bi9x2c9dns3ww98v-texlinks/bin/texlinks: line 252: /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpsewhich: cannot execute binary file: Exec format error
/nix/store/j00iibma3wqn88f8bi9x2c9dns3ww98v-texlinks/bin/texlinks: line 298: /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: cannot execute binary file: Exec format error
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin: no write permissions. Skipping...
/nix/store/j00iibma3wqn88f8bi9x2c9dns3ww98v-texlinks/bin/texlinks: line 192: /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: cannot execute binary file: Exec format error
@nix { "action": "setPhase", "phase": "fixupPhase" }
```
post-installation fixup
strip is /nix/store/falx4gakl6k1q64aqr4f8mvy8vfqcqaq-cctools-binutils-darwin-949.0.1/bin/strip
patching script interpreter paths in /nix/store/q1haqqnh6i2677ihad4673gc3slna3sw-texlinks.sh
/nix/store/q1haqqnh6i2677ihad4673gc3slna3sw-texlinks.sh: interpreter directive changed from "#!/bin/sh" to "/nix/store/715y5lnw12vdcc3hh9sc9vbkdi6c83x6-bash-4.4-p23/bin/sh"
sed: couldn't open temporary file /nix/store/sedqPf2C0: Operation not permitted
In https://github.com/NixOS/nix/issues/5333 I noticed that adding
a minor change to `bash` derivation triggers rebuilds in CA for
`bash` (expected) and `texlive.combine` (unexpected).
regnat debugged derivation instability down to sort on outPath.
The change avoid sorting on outPaths and uses pname of the derivations.
Tested on `R` derivation.
Closes: https://github.com/NixOS/nix/issues/5333
Co-authored-by: Vincenzo Mantova <1962985+xworld21@users.noreply.github.com>
Motivations
- not well reproducible as they disable hashed mirrors (tarballs.nixos.org)
- only reachable by modifying the file pkgs.nix, thus unused
- the space saving is analogous to using substitutes
Rust 1.50.0 incorporated a Cargo change (rust-lang/cargo#8937) in
which cargo vendor erroneously changed permissions of vendored
crates. This was fixed in Rust
1.51.0 (rust-lang/cargo#9131). Unfortunately, this means that all
cargoSha256/cargoHashes produced during the Rust 1.50.0 cycle are
potentially broken.
This change updates cargoSha256/cargoHash tree-wide.
Fixes#121994.
The dvisvgm version provided by TeXLive now includes the fixes
mentioned in the attached issue and thus can be used again instead of
relying on upstream sources.
This is done because the xpdf provided by Nixpkgs is marked as
insecure. Granted, the version provided by TeXLive also has these
vulnerabilities, but it might be okay to treat them as irrelevant in
the context of TeX.
Building a simple LaTeX document without bibTeX with a table of contents yielded a document with an empty table of contents since LaTeX was only run once
In 85605c8a29 we got a really nice
performance improvement.
By using this implementation for texlive instantiation RSS is cut by
~2.6x with a similar wall time speedup.
Also change default package version from texlive year to a
revision. The problem with using texlive year is that it will change
on each of our update even if the package itself was not updated. The
texlive SVN revision number should not change unless the package is
updated. This will allow to share more store paths between different
texlive versions.
chktex binary doesn't work without its global resource file which
comes in its tlType="run" package. This commit moves chktex into a
separate tlType="bin" package so it is only used along with its
tlType="run" counterpart.
Without it, building a document fails with the following error:
pdflatex failed
index.tex: File `pdflscape.sty' not found.
index.tex:47: Emergency stop.
