Daniël de Kok
|
2c5f7b241c
|
python3Packages.tokenizers: canonicalize tests
Make handling of tests more like other Python derivations (and they
actually run again).
|
2021-05-25 09:15:05 -07:00 |
|
|
Daniël de Kok
|
1202bf4227
|
python3Packages.tokenizers: 0.10.1 -> 0.10.3
Changelog:
https://github.com/huggingface/tokenizers/releases/tag/python-v0.10.2
https://github.com/huggingface/tokenizers/releases/tag/python-v0.10.3
|
2021-05-25 09:15:05 -07:00 |
|
|
Daniël de Kok
|
da45ee7202
|
python3Packages.tokenizers: 0.10.0 -> 0.10.1
Changelog:
https://github.com/huggingface/tokenizers/releases/tag/python-v0.10.1
|
2021-03-06 08:54:17 +01:00 |
|
|
Daniël de Kok
|
4a4da22674
|
python3Packages.tokenizers: switch to buildPythonPackage
Use the new cargoSetupHook to set up Cargo vendoring, so that we do
not need buildRustPackage anymore.
|
2021-02-10 07:01:24 +01:00 |
|
Pavol Rusnak
|
a6ce00c50c
|
treewide: remove stdenv where not needed
|
2021-01-25 18:31:47 +01:00 |
|
|
Daniël de Kok
|
cca0b0117b
|
python3Packages.tokenizers: 0.9.4 -> 0.10.0
Changelog:
https://github.com/huggingface/tokenizers/releases/tag/python-v0.10.0
|
2021-01-23 11:22:34 +01:00 |
|
Profpatsch
|
4a7f99d55d
|
treewide: with stdenv.lib; in meta -> with lib;
Part of: https://github.com/NixOS/nixpkgs/issues/108938
meta = with stdenv.lib;
is a widely used pattern. We want to slowly remove
the `stdenv.lib` indirection and encourage people
to use `lib` directly. Thus let’s start with the meta
field.
This used a rewriting script to mostly automatically
replace all occurances of this pattern, and add the
`lib` argument to the package header if it doesn’t
exist yet.
The script in its current form is available at
https://cs.tvl.fyi/depot@2f807d7f141068d2d60676a89213eaa5353ca6e0/-/blob/users/Profpatsch/nixpkgs-rewriter/default.nix
|
2021-01-11 10:38:22 +01:00 |
|
|
Daniël de Kok
|
c67382b02f
|
python3Packages.tokenizers: 0.9.2 -> 0.9.4
Changelog:
https://github.com/huggingface/tokenizers/releases/tag/python-v0.9.3
https://github.com/huggingface/tokenizers/releases/tag/python-v0.9.4
|
2020-11-10 14:52:03 -08:00 |
|
|
Daniël de Kok
|
d2e918cc12
|
python3Packages.tokenizers: 0.8.1 -> 0.9.2
Changelog:
https://github.com/huggingface/tokenizers/releases/tag/python-v0.9.0
https://github.com/huggingface/tokenizers/releases/tag/python-v0.9.1
https://github.com/huggingface/tokenizers/releases/tag/python-v0.9.2
Changes in the derivation:
* Tokenizers uses a newer version of pyo3 that does not require Rust
nightly anymore. So, we do not have to use any cheat codes anymore.
* Tokenizers is now a mixed Rust/Python project. The way it is set
up does not work with Maturin, so switch to setuptools-rust instead.
* Add additional data files needed for tests.
* Use `pytestCheckHook`.
|
2020-10-28 10:41:27 -07:00 |
|
|
Daniël de Kok
|
403e2506af
|
python3Packages.tokenizers: 0.8.1rc1 -> 0.8.1
|
2020-08-05 08:23:11 +02:00 |
|
|
Daniël de Kok
|
b0a6bd4281
|
python3Packages.tokenizers: fix build
Update the parking_lot dependency to be compatible with recent Rust
versions that replace asm! by llvm_asm!:
https://github.com/Amanieu/parking_lot/pull/223
Fixes #94682
|
2020-08-05 08:19:01 +02:00 |
|
|
Daniël de Kok
|
918d5cfb33
|
python3Packages.tokenizers: 0.8.0 -> 0.8.1.rc1
Changelog:
https://github.com/huggingface/tokenizers/releases/tag/python-v0.8.1.rc1
|
2020-07-07 07:25:13 +02:00 |
|
|
Daniël de Kok
|
66d0b2a0d0
|
python3Packages.tokenizers: init at 0.8.0
|
2020-07-02 21:53:51 +02:00 |
|