This reverts commit bddafba6a5.
Running without pytest-xdist leads to reliable crashes on both
aarch64-linux (Ampere Q80-30) and x86_64-linux (Zen 3 Epyc) on hydra,
which prevents the package from being cached.
We revert this change to prevent the churn on downstream maintainers
and consumers, that would otherwise have to build this expensive
package locally, which does not work reliably.
bfd5c2606a