c2b898da76
Passing `-l$NIX_BUILD_CORES` improperly limits the overall system load. For a build machine which is configured to run `$B` builds where each build gets `total cores / B` cores (`$C`), passing `-l $C` to make will improperly limit the load to `$C` instead of `$B * $C`. This effect becomes quite pronounced on machines with 80 cores, with 40 simultaneous builds and a cores limit of 2. On a machine with this configuration, Nix will run 40 builds and make will limit the overall system load to approximately 2. A build machine with this many cores can happily run with a load approaching 80. A non-solution is to oversubscribe the machine, by picking a larger `$C`. However, there is no way to divide the number of cores in a way which fairly subdivides the available cores when `$B` is greater than 1. There has been exploration of passing a jobserver in to the sandbox, or sharing a jobserver between all the builds. This is one option, but relatively complicated and only supports make. Lots of other software uses its own implementation of `-j` and doesn't support either `-l` or the Make jobserver. For the case of an interactive user machine, the user should limit overall system load using `$B`, `$C`, and optionally systemd's cpu/network/io limiting features. Making this change should significantly improve the utilization of our build farm, and improve the throughput of Hydra.
53 lines
1.5 KiB
Bash
Executable file
53 lines
1.5 KiB
Bash
Executable file
################################################################################
|
|
# Build all of the platforms manually since the `all_platforms' target
|
|
# doesn't preserve all of the build outputs and overrides CFLAGS.
|
|
set -e
|
|
set -u
|
|
|
|
################################################################################
|
|
# Prevent a warning from shellcheck:
|
|
out=${out:-/tmp}
|
|
|
|
################################################################################
|
|
export CFLAGS=$NIX_CFLAGS_COMPILE
|
|
export MAKEFLAGS="\
|
|
${enableParallelBuilding:+-j${NIX_BUILD_CORES}}"
|
|
|
|
################################################################################
|
|
PRODUCTS="blackmagic.bin blackmagic.hex blackmagic_dfu.bin blackmagic_dfu.hex"
|
|
|
|
################################################################################
|
|
make_platform() {
|
|
echo "Building for hardware platform $1"
|
|
|
|
make clean
|
|
make PROBE_HOST="$1"
|
|
|
|
if [ "$1" = "hosted" ]; then
|
|
install -m 0555 blackmagic "$out/bin"
|
|
fi
|
|
|
|
for f in $PRODUCTS; do
|
|
if [ -r "$f" ]; then
|
|
mkdir -p "$out/firmware/$1"
|
|
install -m 0444 "$f" "$out/firmware/$1"
|
|
fi
|
|
done
|
|
|
|
}
|
|
|
|
################################################################################
|
|
# Start by building libopencm3:
|
|
make -C libopencm3
|
|
|
|
################################################################################
|
|
# And now all of the platforms:
|
|
cd src
|
|
|
|
mkdir -p "$out/bin"
|
|
|
|
for platform in platforms/*/Makefile.inc; do
|
|
probe=$(basename "$(dirname "$platform")")
|
|
make_platform "$probe"
|
|
done
|