Commit graph

69 commits

Author SHA1 Message Date
Rick van Schijndel a31a3eca58
Merge pull request #251066 from lilyinstarlight/feature/prefetch-npm-deps-tokens
prefetch-npm-deps: add support for NIX_NPM_TOKENS env var
2023-09-12 17:31:26 +02:00
Lily Foster 7f76ac6e09
fetchNpmDeps: pass NIX_NPM_TOKENS as an impure env var 2023-09-11 16:50:17 -04:00
Lily Foster e271266179
prefetch-npm-deps: add support for NIX_NPM_TOKENS env var 2023-09-11 16:49:36 -04:00
sternenseemann bd374243c0 npmHooks: use adjacent packages, not buildPackages
Hooks are essentially implemented as special shell packages that run on
their respective host platform. When they are used, they appear as
nativeBuildInputs (as they need to be executed as part of the build of a
package using them) so are taken from buildPackages relative to the
derivation using them.

Since the override in buildNpmPackage nullifies splicing, we take
npmHooks from buildPackages manually.

Fixes pkgsCross.ghcjs.buildPackages.emscripten and thus
pkgsCross.ghcjs.haskellPackages.ghc.
2023-09-09 17:55:37 +02:00
Robert Schütz 96e5dce777 buildNpmPackage: use installManPage 2023-08-19 10:04:00 -07:00
Pierre Bourdon 28693fb1c3
buildNpmPackage: symlink manpages to the correct output directory 2023-08-18 12:41:05 +02:00
Lily Foster 785ed11d0a
prefetch-npm-deps: fix error typo and unnecessary name qualifier 2023-08-09 19:13:26 -04:00
MicroBlock c2b8bf7f4e
feat: implement redirect for fetch-yarn-deps
This would fix NixOS/nixpkgs#245415
2023-07-26 15:13:37 +08:00
Lily Foster 249efd2271
Merge pull request #223457 from lilyinstarlight/fix/fetch-yarn-deps-pulsar
fetchYarnDeps: account for more yarn.lock spec edge cases
2023-07-16 20:24:16 -04:00
Stanislav Asunkin 38ca04ed64 buildNpmPackage: remove redundant passthru.npmDeps 2023-07-15 10:29:55 +03:00
Lily Foster bee82b47b5
Merge pull request #240419 from lilyinstarlight/feature/prefetch-npm-deps-isahc
prefetch-npm-deps: use isahc instead of ureq
2023-07-08 14:18:07 +02:00
Lily Foster fb9252fa48
prefetch-npm-deps: use exponential backoff 2023-07-04 14:47:13 -04:00
Lily Foster 86f94c3e45
prefetch-yarn-deps: add tests passthru 2023-06-29 11:11:41 -04:00
Lily Foster ba7a869a9a
prefetch-npm-deps: add env_logger 2023-06-29 09:51:37 -04:00
Lily Foster d2897e463d
prefetch-npm-deps: use isahc instead of ureq 2023-06-29 09:46:35 -04:00
Lily Foster aa2f51f0d2
prefetch-npm-deps: update deps and base64 usage 2023-06-29 09:46:35 -04:00
Lily Foster 9af0383296
prefetch-npm-deps: support NIX_BUILD_CORES 2023-06-29 09:46:33 -04:00
Lily Foster 19e48f0da9
buildNpmPackage: add npmWorkspace and npmPruneFlags args 2023-06-29 07:42:25 -04:00
Lily Foster 33898b9ad7
Merge pull request #238452 from XYenon/master
prefetch-npm-deps: add retry
2023-06-28 14:01:57 -04:00
XYenon 4915de2600 prefetch-npm-deps: add retry 2023-06-29 00:20:25 +08:00
Winter c9abff9027 buildNpmPackage: add lilyinstarlight as maintainer/code owner
This is long overdue, Lily has done more than me for Node.js in Nixpkgs
by this point.
2023-06-27 14:59:40 -04:00
Lily Foster 5c32e0ba7c
prefetch-yarn-deps: add --fixup-lockfile flag to fixup a yarn.lock (#214062)
The flag iterates through the lockfile entries, rewrites `resolved` URLs
to those that will be in the cache (like `fixup_yarn_lock` from
yarn2nix), removes `integrity` for git deps whose hash won't match the
reproducible repacking that the fetcher does, writes the amended
lockfile, and exits.
2023-06-27 20:27:33 +02:00
Winter f9138c5ad6 npmHooks.npmInstallHook: allow disabling npm prune invocation
In some odd scenarios, `npm prune` either fails, or hangs. I have no idea
what could possibly be wrong at the moment, but let's provide an escape
hatch for packages that can still use the rest of the install hook's
functionality.
2023-05-28 00:47:54 -04:00
Winter 9117f01272
npmHooks.npmConfigHook: add npmRoot option 2023-05-21 17:26:22 -04:00
Winter 9f264f69f8
buildNpmPackage: use nodejs patch flag to work around roadblocks 2023-05-20 18:29:43 -04:00
Winter ac35d7ea86
prefetch-npm-deps: look up hashes from cache when fixing up lockfiles 2023-04-30 10:29:46 -04:00
Winter 7efebca89c
prefetch-npm-deps: fix reproducibility
v1 lockfiles can contain multiple references to the same version of a
package, and these references can contain different `integrity` values,
such as one having SHA-1 and SHA-512, while another just has SHA-512.

Given that HashMap iteration order isn't defined, this causes
reproducibility issues, as a different integrity value could be chosen
each time.

Thanks to @lilyinstarlight for discovering this issue originally, as well
as the idea for the sorting-based implementation.
2023-04-28 18:09:39 -04:00
Winter faa3de1bf5
prefetch-npm-deps: fix clippy lint 2023-04-06 11:32:47 -04:00
Winter e1d64c1941
prefetch-npm-deps: throw better error when unsupported git service is used
See https://github.com/NixOS/nixpkgs/pull/206476#discussion_r1058689383.
2023-04-06 11:32:47 -04:00
Winter 7e247e6fec
prefetch-npm-deps: download dev deps for git deps with install scripts
Git dependencies with install scripts are built isolated from the main
package, so their development dependencies are required.

To take advantage of this, #206477 is needed.
2023-04-06 11:32:39 -04:00
Winter 3e68a27822
prefetch-npm-deps: refactor
This splits prefetch-npm-deps into multiple files, as well as making a
few small changes along the way, such as going from a `HashMap` to a `Vec`
as the container for packages, to deduplicate them more efficently.
2023-04-06 11:32:23 -04:00
Lily Foster a777753eb2
fetchYarnDeps: add more git url tests 2023-03-27 19:40:08 -04:00
Lily Foster 88375f1b74
fetchYarnDeps: fix test hashes 2023-03-27 19:40:08 -04:00
Lily Foster e7cd275cd8
fetchYarnDeps: account for more yarn.lock spec edge cases
* Ignore relative `file:` paths.
* Support github codeload URLs with `refs/tags/tag` in addition to just `tag`.
* Support https://github.com/owner/repo/archive/ref.tar.gz URLs for git download.
2023-03-27 19:40:05 -04:00
github-actions[bot] 507feca606
Merge master into staging-next 2023-02-20 12:01:32 +00:00
Yureka 20539ac23d prefetch-npm-deps: add nix to PATH
Fixes "No such file or directory" error when running prefetch-npm-deps
in its default mode (print hashes, for update scripts etc),
in an environment that doesn't have a `nix` binary (for `nix hash`).
2023-02-20 01:20:56 -05:00
Artturin 4e3dcf364e treewide: makeSetupHook deps -> propagatedBuildInputs 2023-02-07 21:02:00 +02:00
Artturin b36d9696c5 treewide: fix error: attribute 'runtimeShell' missing 2022-12-22 20:50:27 +02:00
Winter 865387061f npmHooks.npmBuildHook: fix diagnostic
Single quotes must be used when using graves in a quote, else Bash will interpret it as a command to run.

Thanks to Dimitri for catching this (8e651111b7 (commitcomment-92100762)).
2022-12-19 18:47:09 -05:00
Lily Foster 669f019379 npmHooks.npmConfigHook: make diagnostic match prefetch-npm-deps
This updates the diagnostic message for a missing package-lock.json to
match what fetchNpmDeps currently says
2022-12-19 18:47:09 -05:00
Winter 7ab5323cfb npmHooks.npmInstallHook: pass --no-save to prune 2022-11-25 15:47:05 -05:00
Sandro Jäckel fde0382efd buildNpmPackage: forward pre/postPatch to fetchNpmDeps 2022-11-25 15:47:05 -05:00
Sandro 84477691f7 fetchNpmDeps: allow package-json.lock symlinks, update hint 2022-11-25 15:47:05 -05:00
Lily Foster b023946d2b prefetch-npm-deps: fix hash stability
GNU tar will apparently silently include mtime of files if --mtime is
passed with an unrecognized date format. This led to hash instability
from those mtimes and this fixes it to force all mtimes to epoch
timestamp 0.
2022-11-23 20:52:08 -05:00
Winter 009a234bdd prefetch-npm-deps: repack hosted git deps
Previously, we stored the tarballs from the hosted Git providers directly in the cache. However, as we've seen with `fetchFromGitHub` etc, these files may change subtly.

Given this, this commit repacks the dependencies before storing them in the cache.
2022-11-21 15:00:08 -05:00
Winter 091d039b12 prefetch-npm-deps: deduplicate dependencies when parsing lockfile
A `package-lock.json` file can contain multiple instances of the same dependency, which caused unnecessary downloads and duplicate index entries in the generated cache.
2022-11-21 15:00:08 -05:00
Winter b117b359db prefetch-npm-deps: move tests to separate file 2022-11-21 15:00:08 -05:00
Winter 3d5f77f496 npmHooks.npmConfigHook: sugget specifying legacy peer deps option for all commands 2022-11-21 15:00:08 -05:00
Winter 93334f5234 npmHooks.npmInstallHook: pass install flags to prune 2022-11-21 15:00:08 -05:00
Winter 80ecb954e9 npmHooks.npmConfigHook: add diagnostic for when dependencies aren't provided 2022-11-21 15:00:08 -05:00