5.2 KiB
Notes on maintaining/upgrading
Upgrading texlive.bin
texlive
contains a few binaries, defined in bin.nix
and released once a year.
In order to reduce closure size for users who just need a few of them, we split it into
packages such as core
, core-big
, xdvi
, etc. This requires making assumptions
about dependencies between the projects that may change between releases; if
you upgrade you may have to do some work here.
Updating the package set
texlive
contains several thousand packages from CTAN, defined in tlpdb.nix
.
The CTAN mirrors are not version-controlled and continuously moving, with more than 100 updates per month.
To create a consistent and reproducible package set in nixpkgs, we generate nix expressions for all packages in TeX Live at a certain day.
To upgrade the package snapshot, follow this process.
Upgrade package information from texlive package database
Update version
in default.nix
with the day of the new snapshot, the new TeX
Live year, and the final status of the snapshot. Then update
texlive.tlpdbxz.hash
to match the new hash of texlive.tlpdb.xz
and run
nix-build ../../../../.. -A texlive.tlpdb.nix --no-out-link
This will download either the daily or the final snapshot of the TeX Live
package database texlive.tlpdb.xz
and extract the relevant package info
(including version numbers and sha512 hashes) for the selected upstream
distribution.
Finally, replace tlpdb.nix
with the generated file. Note that if the
version
info does not match the metadata of tlpdb.nix
(as found in the
00texlive.config
package), TeX Live packages will not evaluate.
The test pkgs.tests.texlive.tlpdbNix
verifies that the file tlpdb.nix
in Nixpkgs matches the one that generated from texlive.tlpdb.xz
.
Build packages locally and generate fix hashes
To prevent unnecessary rebuilds, texlive packages are built as fixed-output
derivations whose hashes are contained in fixed-hashes.nix
.
Updating the list of fixed hashes requires a local build of all new packages,
which is a resource-intensive process. First build the hashes for the new
packages. Consider tweaking the -j
option to maximise core usage.
nix-build generate-fixed-hashes.nix -A newHashes -j 8
Then build the Nix expression containing all the hashes, old and new. This step cannot be parallelized because it relies on 'import from derivation'.
nix-build generate-fixed-hashes.nix -A fixedHashesNix
Finally, copy the result to fixed-hashes.nix
.
Warning. The expression fixedHashesNix
reuses the previous fixed hashes
when possible. This is based on two assumptions: that .tar.xz
archives with
the same names remain identical in time (which is the intended behaviour of
CTAN and the various mirrors) and that the build recipe continues to produce
the same output. Should those assumptions not hold, remove the previous fixed
hashes for the relevant package, or for all packages.
Updating the licensing information
The license of each package in texlive is automatically extracted from texlive's
texlive.tlpdb into tlpdb.nix. The combined licenses of the schemes is stored
separately in default.nix
and must be kept in sync with the licenses of the
actual contents of these schemes. Whether this is the case can be verified with the
pkgs.tests.texlive.licenses
test. In case of a mismatch, copy the “correct”
license lists reported by the test into default.nix
.
Running the testsuite
There are a some other useful tests that haven't been mentioned before. Build them with
nix-build ../../../../.. -A tests.texlive --no-out-link
Commit changes
Commit the updated tlpdb.nix
and fixed-hashes.nix
to the repository with
a message like
texlive: 2022-final -> 2023.20230401
Please make sure to follow the CONTRIBUTING guidelines.
Reviewing the bin containers
Most tlType == "bin"
containers consist of links to scripts distributed in
$TEXMFDIST/scripts
with a number of patches applied within default.nix
.
At each upgrade, please run the tests tests.texlive.shebangs
to verify that
all shebangs have been patched and in case add the relevant interpreters, and
use tests.texlive.binaries
to check if basic execution of all binaries works.
Please review manually all binaries in the broken
and ignored
lists of
tests.texlive.binaries
at least once for major TeX Live release.
Since the tests cannot catch all runtime dependencies, you should grep the
$TEXMFDIST/scripts
folder for common cases, for instance (where $scripts
points to the relevant folder of scheme-full
):
- Calls to
exec $interpreter
grep -IRS 'exec ' "$TEXMFDIST/scripts" | cut -d: -f2 | sort -u | less -S
- Calls to Ghostscripts (see
needsGhostscript
incombine.nix
)grep -IR '\([^a-zA-Z]\|^\)gs\( \|$\|"\)' "$TEXMFDIST"/scripts grep -IR 'rungs' "$TEXMFDIST"
As a general rule, if a runtime dependency as above is essential for the core
functionality of the package, then it should be made available in the bin
containers (by patching PATH
), or in texlive.combine
(as we do for
Ghostscript). Non-essential runtime dependencies should be ignored if they
increase the closure substantially.