b12f/nixpkgs
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity

Merge pull request #14858 from ericbmerritt/elixir-support

Browse source 
Elixir support
This commit is contained in:
Joachim Fasting 2016-04-25 23:04:14 +02:00
parent ede605e4f1 cd27abe751
commit 2dbfe512db
28 changed files with 30732 additions and 3498 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

376
doc/beam-users-guide.xml Normal file
View file

@ -0,0 +1,376 @@
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xml:id="users-guide-to-the-erlang-infrastructure">
<title>User's Guide to the Beam Infrastructure</title>
<section xml:id="beam-introduction">
<title>Beam Languages (Erlang &amp; Elixir) on Nix</title>
<para>
In this document and related Nix expressions we use the term
<emphasis>Beam</emphasis> to describe the environment. Beam is
the name of the Erlang Virtial Machine and, as far as we know,
from a packaging perspective all languages that run on Beam are
interchangable. The things that do change, like the build
system, are transperant to the users of the package. So we make
no distinction.
</para>
</section>
<section xml:id="build-tools">
<title>Build Tools</title>
<section xml:id="build-tools-rebar3">
<title>Rebar3</title>
<para>
By default Rebar3 wants to manage it's own dependencies. In the
normal non-Nix, this is perfectly acceptable. In the Nix world it
is not. To support this we have created two versions of rebar3,
<literal>rebar3</literal> and <literal>rebar3-open</literal>. The
<literal>rebar3</literal> version has been patched to remove the
ability to download anything from it. If you are not running it a
nix-shell or a nix-build then its probably not going to work for
you. <literal>rebar3-open</literal> is the normal, un-modified
rebar3. It should work exactly as would any other version of
rebar3. Any Erlang package should rely on
<literal>rebar3</literal> and thats really what you should be
using too.
</para>
</section>
<section xml:id="build-tools-other">
<title>Mix &amp; Erlang.mk</title>
<para>
Both Mix and Erlang.mk work exactly as you would expect. There
is a bootstrap process that needs to be run for both of
them. However, that is supported by the
<literal>buildMix</literal> and <literal>buildErlangMk</literal> derivations.
</para>
</section>
</section>
<section xml:id="how-to-install-beam-packages">
<title>How to install Beam packages</title>
<para>
Beam packages are not registered in the top level simply because
they are not relevant to the vast majority of Nix users. They are
installable using the <literal>beamPackages</literal> attribute
set.
You can list the avialable packages in the
<literal>beamPackages</literal> with the following command:
</para>
<programlisting>
$ nix-env -f &quot;&lt;nixpkgs&gt;&quot; -qaP -A beamPackages
beamPackages.esqlite esqlite-0.2.1
beamPackages.goldrush goldrush-0.1.7
beamPackages.ibrowse ibrowse-4.2.2
beamPackages.jiffy jiffy-0.14.5
beamPackages.lager lager-3.0.2
beamPackages.meck meck-0.8.3
beamPackages.rebar3-pc pc-1.1.0
</programlisting>
<para>
To install any of those packages into your profile, refer to them by
their attribute path (first column):
</para>
<programlisting>
$ nix-env -f &quot;&lt;nixpkgs&gt;&quot; -iA beamPackages.ibrowse
</programlisting>
<para>
The attribute path of any Beam packages corresponds to the name
of that particular package in Hex or its OTP Application/Release name.
</para>
</section>
<section xml:id="packaging-beam-applications">
<title>Packaging Beam Applications</title>
<section xml:id="packaging-erlang-applications">
<title>Erlang Applications</title>
<section xml:id="rebar3-packages">
<title>Rebar3 Packages</title>
<para>
There is a Nix functional called
<literal>buildRebar3</literal>. We use this function to make a
derivation that understands how to build the rebar3 project. For
example, the epression we use to build the <link
xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link>
project follows.
</para>
<programlisting>
{stdenv, fetchFromGitHub, buildRebar3, ibrowse, jsx, erlware_commons }:
buildRebar3 rec {
name = "hex2nix";
version = "0.0.1";
src = fetchFromGitHub {
owner = "ericbmerritt";
repo = "hex2nix";
rev = "${version}";
sha256 = "1w7xjidz1l5yjmhlplfx7kphmnpvqm67w99hd2m7kdixwdxq0zqg";
};
beamDeps = [ ibrowse jsx erlware_commons ];
}
</programlisting>
<para>
The only visible difference between this derivation and
something like <literal>stdenv.mkDerivation</literal> is that we
have added <literal>erlangDeps</literal> to the derivation. If
you add your Beam dependencies here they will be correctly
handled by the system.
</para>
<para>
If your package needs to compile native code via Rebar's port
compilation mechenism. You should add <literal>compilePort =
true;</literal> to the derivation.
</para>
</section>
<section xml:id="erlang-mk-packages">
<title>Erlang.mk Packages</title>
<para>
Erlang.mk functions almost identically to Rebar. The only real
difference is that <literal>buildErlangMk</literal> is called
instead of <literal>buildRebar3</literal>
</para>
<programlisting>
{ buildErlangMk, fetchHex, cowlib, ranch }:
buildErlangMk {
name = "cowboy";
version = "1.0.4";
src = fetchHex {
pkg = "cowboy";
version = "1.0.4";
sha256 =
"6a0edee96885fae3a8dd0ac1f333538a42e807db638a9453064ccfdaa6b9fdac";
};
beamDeps = [ cowlib ranch ];
meta = {
description = ''Small, fast, modular HTTP server written in
Erlang.'';
license = stdenv.lib.licenses.isc;
homepage = "https://github.com/ninenines/cowboy";
};
}
</programlisting>
</section>
<section xml:id="mix-packages">
<title>Mix Packages</title>
<para>
Mix functions almost identically to Rebar. The only real
difference is that <literal>buildMix</literal> is called
instead of <literal>buildRebar3</literal>
</para>
<programlisting>
{ buildMix, fetchHex, plug, absinthe }:
buildMix {
name = "absinthe_plug";
version = "1.0.0";
src = fetchHex {
pkg = "absinthe_plug";
version = "1.0.0";
sha256 =
"08459823fe1fd4f0325a8bf0c937a4520583a5a26d73b193040ab30a1dfc0b33";
};
beamDeps = [ plug absinthe];
meta = {
description = ''A plug for Absinthe, an experimental GraphQL
toolkit'';
license = stdenv.lib.licenses.bsd3;
homepage = "https://github.com/CargoSense/absinthe_plug";
};
}
</programlisting>
</section>
</section>
</section>
<section xml:id="how-to-develop">
<title>How to develop</title>
<section xml:id="accessing-an-environment">
<title>Accessing an Environment</title>
<para>
Often, all you want to do is be able to access a valid
environment that contains a specific package and its
dependencies. we can do that with the <literal>env</literal>
part of a derivation. For example, lets say we want to access an
erlang repl with ibrowse loaded up. We could do the following.
</para>
<programlisting>
~/w/nixpkgs ❯❯❯ nix-shell -A beamPackages.ibrowse.env --run "erl"
Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V7.0 (abort with ^G)
1> m(ibrowse).
Module: ibrowse
MD5: 3b3e0137d0cbb28070146978a3392945
Compiled: January 10 2016, 23:34
Object file: /nix/store/g1rlf65rdgjs4abbyj4grp37ry7ywivj-ibrowse-4.2.2/lib/erlang/lib/ibrowse-4.2.2/ebin/ibrowse.beam
Compiler options: [{outdir,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/ebin"},
debug_info,debug_info,nowarn_shadow_vars,
warn_unused_import,warn_unused_vars,warnings_as_errors,
{i,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/include"}]
Exports:
add_config/1 send_req_direct/7
all_trace_off/0 set_dest/3
code_change/3 set_max_attempts/3
get_config_value/1 set_max_pipeline_size/3
get_config_value/2 set_max_sessions/3
get_metrics/0 show_dest_status/0
get_metrics/2 show_dest_status/1
handle_call/3 show_dest_status/2
handle_cast/2 spawn_link_worker_process/1
handle_info/2 spawn_link_worker_process/2
init/1 spawn_worker_process/1
module_info/0 spawn_worker_process/2
module_info/1 start/0
rescan_config/0 start_link/0
rescan_config/1 stop/0
send_req/3 stop_worker_process/1
send_req/4 stream_close/1
send_req/5 stream_next/1
send_req/6 terminate/2
send_req_direct/4 trace_off/0
send_req_direct/5 trace_off/2
send_req_direct/6 trace_on/0
trace_on/2
ok
2>
</programlisting>
<para>
Notice the <literal>-A beamPackages.ibrowse.env</literal>.That
is the key to this functionality.
</para>
</section>
<section xml:id="creating-a-shell">
<title>Creating a Shell</title>
<para>
Getting access to an environment often isn't enough to do real
development. Many times we need to create a
<literal>shell.nix</literal> file and do our development inside
of the environment specified by that file. This file looks a lot
like the packageing described above. The main difference is that
<literal>src</literal> points to project root and we call the
package directly.
</para>
<programlisting>
{ pkgs ? import &quot;&lt;nixpkgs&quot;&gt; {} }:
with pkgs;
let
f = { buildRebar3, ibrowse, jsx, erlware_commons }:
buildRebar3 {
name = "hex2nix";
version = "0.1.0";
src = ./.;
erlangDeps = [ ibrowse jsx erlware_commons ];
};
drv = beamPackages.callPackage f {};
in
drv
</programlisting>
<section xml:id="building-in-a-shell">
<title>Building in a shell</title>
<para>
We can leveral the support of the Derivation, regardless of
which build Derivation is called by calling the commands themselv.s
</para>
<programlisting>
# =============================================================================
# Variables
# =============================================================================
NIX_TEMPLATES := "$(CURDIR)/nix-templates"
TARGET := "$(PREFIX)"
PROJECT_NAME := thorndyke
NIXPKGS=../nixpkgs
NIX_PATH=nixpkgs=$(NIXPKGS)
NIX_SHELL=nix-shell -I "$(NIX_PATH)" --pure
# =============================================================================
# Rules
# =============================================================================
.PHONY= all test clean repl shell build test analyze configure install \
test-nix-install publish plt analyze
all: build
guard-%:
@ if [ "${${*}}" == "" ]; then \
echo "Environment variable $* not set"; \
exit 1; \
fi
clean:
rm -rf _build
rm -rf .cache
repl:
$(NIX_SHELL) --run "iex -pa './_build/prod/lib/*/ebin'"
shell:
$(NIX_SHELL)
configure:
$(NIX_SHELL) --command 'eval "$$configurePhase"'
build: configure
$(NIX_SHELL) --command 'eval "$$buildPhase"'
install:
$(NIX_SHELL) --command 'eval "$$installPhase"'
test:
$(NIX_SHELL) --command 'mix test --no-start --no-deps-check'
plt:
$(NIX_SHELL) --run "mix dialyzer.plt --no-deps-check"
analyze: build plt
$(NIX_SHELL) --run "mix dialyzer --no-compile"
</programlisting>
<para>
If you add the <literal>shell.nix</literal> as described and
user rebar as follows things should simply work. Aside from the
<literal>test</literal>, <literal>plt</literal>, and
<literal>analyze</literal> the talks work just fine for all of
the build Derivations.
</para>
</section>
</section>
</section>
<section xml:id="generating-packages-from-hex-with-hex2nix">
<title>Generating Packages from Hex with Hex2Nix</title>
<para>
Updating the Hex packages requires the use of the
<literal>hex2nix</literal> tool. Given the path to the Erlang
modules (usually
<literal>pkgs/development/erlang-modules</literal>). It will
happily dump a file called
<literal>hex-packages.nix</literal>. That file will contain all
the packages that use a recognized build system in Hex. However,
it can't know whether or not all those packages are buildable.
</para>
<para>
To make life easier for our users, it makes good sense to go
ahead and attempt to build all those packages and remove the
ones that don't build. To do that, simply run the command (in
the root of your <literal>nixpkgs</literal> repository). that follows.
</para>
<programlisting>
$ nix-build -A beamPackages
</programlisting>
<para>
That will build every package in
<literal>beamPackages</literal>. Then you can go through and
manually remove the ones that fail. Hopefully, someone will
improve <literal>hex2nix</literal> in the future to automate
that.
</para>
</section>
</chapter>

305
doc/erlang-users-guide.xml
View file

@ -1,305 +0,0 @@
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xml:id="users-guide-to-the-erlang-infrastructure">
<title>User's Guide to the Erlang Infrastructure</title>
<section xml:id="build-tools">
<title>Build Tools</title>
<para>
By default Rebar3 wants to manage it's own dependencies. In the
normal non-Nix, this is perfectly acceptable. In the Nix world it
is not. To support this we have created two versions of rebar3,
<literal>rebar3</literal> and <literal>rebar3-open</literal>. The
<literal>rebar3</literal> version has been patched to remove the
ability to download anything from it. If you are not running it a
nix-shell or a nix-build then its probably not going to work for
you. <literal>rebar3-open</literal> is the normal, un-modified
rebar3. It should work exactly as would any other version of
rebar3. Any Erlang package should rely on
<literal>rebar3</literal> and thats really what you should be
using too.
</para>
</section>
<section xml:id="how-to-install-erlang-packages">
<title>How to install Erlang packages</title>
<para>
Erlang packages are not registered in the top level simply because
they are not relevant to the vast majority of Nix users. They are
installable using the <literal>erlangPackages</literal> attribute set.
You can list the avialable packages in the
<literal>erlangPackages</literal> with the following command:
</para>
<programlisting>
$ nix-env -f &quot;&lt;nixpkgs&gt;&quot; -qaP -A erlangPackages
erlangPackages.esqlite esqlite-0.2.1
erlangPackages.goldrush goldrush-0.1.7
erlangPackages.ibrowse ibrowse-4.2.2
erlangPackages.jiffy jiffy-0.14.5
erlangPackages.lager lager-3.0.2
erlangPackages.meck meck-0.8.3
erlangPackages.rebar3-pc pc-1.1.0
</programlisting>
<para>
To install any of those packages into your profile, refer to them by
their attribute path (first column):
</para>
<programlisting>
$ nix-env -f &quot;&lt;nixpkgs&gt;&quot; -iA erlangPackages.ibrowse
</programlisting>
<para>
The attribute path of any Erlang packages corresponds to the name
of that particular package in Hex or its OTP Application/Release name.
</para>
</section>
<section xml:id="packaging-erlang-applications">
<title>Packaging Erlang Applications</title>
<section xml:id="rebar3-packages">
<title>Rebar3 Packages</title>
<para>
There is a Nix functional called
<literal>buildRebar3</literal>. We use this function to make a
derivation that understands how to build the rebar3 project. For
example, the epression we use to build the <link
xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link>
project follows.
</para>
<programlisting>
{stdenv, fetchFromGitHub, buildRebar3, ibrowse, jsx, erlware_commons }:
buildRebar3 rec {
name = "hex2nix";
version = "0.0.1";
src = fetchFromGitHub {
owner = "ericbmerritt";
repo = "hex2nix";
rev = "${version}";
sha256 = "1w7xjidz1l5yjmhlplfx7kphmnpvqm67w99hd2m7kdixwdxq0zqg";
};
erlangDeps = [ ibrowse jsx erlware_commons ];
}
</programlisting>
<para>
The only visible difference between this derivation and
something like <literal>stdenv.mkDerivation</literal> is that we
have added <literal>erlangDeps</literal> to the derivation. If
you add your Erlang dependencies here they will be correctly
handled by the system.
</para>
<para>
If your package needs to compile native code via Rebar's port
compilation mechenism. You should add <literal>compilePort =
true;</literal> to the derivation.
</para>
</section>
<section xml:id="hex-packages">
<title>Hex Packages</title>
<para>
Hex packages are based on Rebar packages. In fact, at the moment
we can only compile Hex packages that are buildable with
Rebar3. Packages that use Mix and other build systems are not
supported. That being said, we know a lot more about Hex and can
do more for you.
</para>
<programlisting>
{ buildHex }:
buildHex {
name = "esqlite";
version = "0.2.1";
sha256 = "1296fn1lz4lz4zqzn4dwc3flgkh0i6n4sydg501faabfbv8d3wkr";
compilePort = true;
}
</programlisting>
<para>
For Hex packages you need to provide the name, the version, and
the Sha 256 digest of the package and use
<literal>buildHex</literal> to build it. Obviously, the package
needs to have already been published to Hex.
</para>
</section>
</section>
<section xml:id="how-to-develop">
<title>How to develop</title>
<section xml:id="accessing-an-environment">
<title>Accessing an Environment</title>
<para>
Often, all you want to do is be able to access a valid
environment that contains a specific package and its
dependencies. we can do that with the <literal>env</literal>
part of a derivation. For example, lets say we want to access an
erlang repl with ibrowse loaded up. We could do the following.
</para>
<programlisting>
~/w/nixpkgs ❯❯❯ nix-shell -A erlangPackages.ibrowse.env --run "erl"
Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V7.0 (abort with ^G)
1> m(ibrowse).
Module: ibrowse
MD5: 3b3e0137d0cbb28070146978a3392945
Compiled: January 10 2016, 23:34
Object file: /nix/store/g1rlf65rdgjs4abbyj4grp37ry7ywivj-ibrowse-4.2.2/lib/erlang/lib/ibrowse-4.2.2/ebin/ibrowse.beam
Compiler options: [{outdir,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/ebin"},
debug_info,debug_info,nowarn_shadow_vars,
warn_unused_import,warn_unused_vars,warnings_as_errors,
{i,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/include"}]
Exports:
add_config/1 send_req_direct/7
all_trace_off/0 set_dest/3
code_change/3 set_max_attempts/3
get_config_value/1 set_max_pipeline_size/3
get_config_value/2 set_max_sessions/3
get_metrics/0 show_dest_status/0
get_metrics/2 show_dest_status/1
handle_call/3 show_dest_status/2
handle_cast/2 spawn_link_worker_process/1
handle_info/2 spawn_link_worker_process/2
init/1 spawn_worker_process/1
module_info/0 spawn_worker_process/2
module_info/1 start/0
rescan_config/0 start_link/0
rescan_config/1 stop/0
send_req/3 stop_worker_process/1
send_req/4 stream_close/1
send_req/5 stream_next/1
send_req/6 terminate/2
send_req_direct/4 trace_off/0
send_req_direct/5 trace_off/2
send_req_direct/6 trace_on/0
trace_on/2
ok
2>
</programlisting>
<para>
Notice the <literal>-A erlangPackages.ibrowse.env</literal>.That
is the key to this functionality.
</para>
</section>
<section xml:id="creating-a-shell">
<title>Creating a Shell</title>
<para>
Getting access to an environment often isn't enough to do real
development. Many times we need to create a
<literal>shell.nix</literal> file and do our development inside
of the environment specified by that file. This file looks a lot
like the packageing described above. The main difference is that
<literal>src</literal> points to project root and we call the
package directly.
</para>
<programlisting>
{ pkgs ? import &quot;&lt;nixpkgs&quot;&gt; {} }:
with pkgs;
let
f = { buildHex, ibrowse, jsx, erlware_commons }:
buildHex {
name = "hex2nix";
version = "0.1.0";
src = ./.;
erlangDeps = [ ibrowse jsx erlware_commons ];
};
drv = erlangPackages.callPackage f {};
in
drv
</programlisting>
<section xml:id="building-in-a-shell">
<title>Building in a shell</title>
<para>
Unfortunatly for us users of Nix, Rebar isn't very cooperative
with us from the standpoint of building a hermetic
environment. When building the rebar3 support we had to do some
sneaky things to get it not to go out and pull packages on its
own. Also unfortunately, you have to do some of the same things
when building a project inside of a Nix shell.
<orderedlist numeration="arabic">
<listitem>
<para>Run <literal>rebar3-nix-bootstrap</literal> every time
dependencies change</para>
</listitem>
<listitem>
<para>Set Home to the current directory.</para>
</listitem>
</orderedlist>
If you do these two things then Rebar will be happy with you. I
codify these into a makefile. Forunately, rebar3-nix-bootstrap
is idempotent and fairly quick. so you can run it as often as
you like.
</para>
<programlisting>
# =============================================================================
# Rules
# =============================================================================
.PHONY= all test clean repl shell build test analyze bootstrap
all: test
clean:
rm -rf _build
rm -rf .cache
repl:
nix-shell --run "erl"
shell:
nix-shell --run "bash"
bootstrap:
nix-shell --pure --run "rebar3-nix-bootstrap"
build: bootstrap
nix-shell --pure --run "HOME=$(CURDIR) rebar3 compile"
analyze: bootstrap
nix-shell --pure --run "HOME=$(CURDIR) rebar3 do compile,dialyzer"
test: bootstrap
nix-shell --pure --run "HOME=$(CURDIR) rebar3 do compile,dialyzer,eunit"
</programlisting>
<para>
If you add the <literal>shell.nix</literal> as described and
user rebar as follows things should simply work.
</para>
</section>
</section>
</section>
<section xml:id="generating-packages-from-hex-with-hex2nix">
<title>Generating Packages from Hex with Hex2Nix</title>
<para>
Updating the Hex packages requires the use of the
<literal>hex2nix</literal> tool. Given the path to the Erlang
modules (usually
<literal>pkgs/development/erlang-modules</literal>). It will
happily dump a file called
<literal>hex-packages.nix</literal>. That file will contain all
the packages that use a recognized build system in Hex. However,
it can't know whether or not all those packages are buildable.
</para>
<para>
To make life easier for our users, it makes good sense to go
ahead and attempt to build all those packages and remove the
ones that don't build. To do that, simply run the command (in
the root of your <literal>nixpkgs</literal> repository). that follows.
</para>
<programlisting>
$ nix-build -A erlangPackages
</programlisting>
<para>
That will build every package in
<literal>erlangPackages</literal>. Then you can go through and
manually remove the ones that fail. Hopefully, someone will
improve <literal>hex2nix</literal> in the future to automate
that.
</para>
</section>
</chapter>

2
doc/manual.xml
View file

@ -21,7 +21,7 @@
<xi:include href="coding-conventions.xml" />
<xi:include href="submitting-changes.xml" />
<xi:include href="haskell-users-guide.xml" />
<xi:include href="erlang-users-guide.xml" />
<xi:include href="beam-users-guide.xml" />
<xi:include href="contributing.xml" />
</book>

86
pkgs/development/beam-modules/build-erlang-mk.nix Normal file
View file

@ -0,0 +1,86 @@
{ stdenv, writeText, erlang, perl, which, gitMinimal, wget }:
{ name, version
, src
, setupHook ? null
, buildInputs ? []
, beamDeps ? []
, postPatch ? ""
, compilePorts ? false
, installPhase ? null
, meta ? {}
, ... }@attrs:
with stdenv.lib;
let
shell = drv: stdenv.mkDerivation {
name = "interactive-shell-${drv.name}";
buildInputs = [ drv ];
};
pkg = self: stdenv.mkDerivation ( attrs // {
app_name = "${name}";
name = "${name}-${version}";
inherit version;
dontStrip = true;
inherit src;
setupHook = if setupHook == null
then writeText "setupHook.sh" ''
addToSearchPath ERL_LIBS "$1/lib/erlang/lib"
''
else setupHook;
buildInputs = [ erlang perl which gitMinimal wget ];
propagatedBuildInputs = beamDeps;
configurePhase = ''
runHook preConfigure
# We shouldnt need to do this, but it seems at times there is a *.app in
# the repo/package. This ensures we start from a clean slate
make SKIP_DEPS=1 clean
runHook postConfigure
'';
buildPhase = ''
runHook preBuild
make SKIP_DEPS=1
runHook postBuild
'';
installPhase = ''
runHook preInstall
mkdir -p $out/lib/erlang/lib/${name}
cp -r ebin $out/lib/erlang/lib/${name}/
cp -r src $out/lib/erlang/lib/${name}/
if [ -d include ]; then
cp -r include $out/lib/erlang/lib/${name}/
fi
if [ -d priv ]; then
cp -r priv $out/lib/erlang/lib/${name}/
fi
if [ -d doc ]; then
cp -r doc $out/lib/erlang/lib/${name}/
fi
runHook postInstall
'';
passthru = {
packageName = name;
env = shell self;
inherit beamDeps;
};
});
in fix pkg

0
pkgs/development/erlang-modules/build-hex.nix → pkgs/development/beam-modules/build-hex.nix
View file

85
pkgs/development/beam-modules/build-mix.nix Normal file
View file

@ -0,0 +1,85 @@
{ stdenv, writeText, elixir, erlang, hexRegistrySnapshot, hex }:
{ name
, version
, src
, setupHook ? null
, buildInputs ? []
, beamDeps ? []
, postPatch ? ""
, compilePorts ? false
, meta ? {}
, ... }@attrs:
with stdenv.lib;
let
shell = drv: stdenv.mkDerivation {
name = "interactive-shell-${drv.name}";
buildInputs = [ drv ];
};
bootstrapper = ./mix-bootstrap;
pkg = self: stdenv.mkDerivation ( attrs // {
name = "${name}-${version}";
inherit version;
dontStrip = true;
inherit src;
setupHook = if setupHook == null
then writeText "setupHook.sh" ''
addToSearchPath ERL_LIBS "$1/lib/erlang/lib"
''
else setupHook;
inherit buildInputs;
propagatedBuildInputs = [ hexRegistrySnapshot hex elixir ] ++ beamDeps;
configurePhase = ''
runHook preConfigure
${erlang}/bin/escript ${bootstrapper}
runHook postConfigure
'';
buildPhase = ''
runHook preBuild
export HEX_OFFLINE=1
export HEX_HOME=`pwd`
export MIX_ENV=prod
MIX_ENV=prod mix compile --debug-info --no-deps-check
runHook postBuild
'';
installPhase = ''
runHook preInstall
MIXENV=prod
if [ -d "_build/shared" ]; then
MIXENV=shared
fi
mkdir -p "$out/lib/erlang/lib/${name}-${version}"
for reldir in src ebin priv include; do
fd="_build/$MIXENV/lib/${name}/$reldir"
[ -d "$fd" ] || continue
cp -Hrt "$out/lib/erlang/lib/${name}-${version}" "$fd"
success=1
done
runHook postInstall
'';
passthru = {
packageName = name;
env = shell self;
inherit beamDeps;
};
});
in fix pkg

11
pkgs/development/erlang-modules/build-rebar3.nix → pkgs/development/beam-modules/build-rebar3.nix
View file

@ -1,10 +1,10 @@
{ stdenv, writeText, erlang, rebar3, openssl, libyaml, fetchHex, fetchFromGitHub,
{ stdenv, writeText, erlang, rebar3, openssl, libyaml,
pc, buildEnv }:
{ name, version
, src
, setupHook ? null
, buildInputs ? [], erlangDeps ? [], buildPlugins ? []
, buildInputs ? [], beamDeps ? [], buildPlugins ? []
, postPatch ? ""
, compilePorts ? false
, installPhase ? null
@ -27,8 +27,9 @@ let
inherit version;
buildInputs = buildInputs ++ [ erlang rebar3 openssl libyaml ];
propagatedBuildInputs = unique (erlangDeps ++ ownPlugins);
propagatedBuildInputs = unique (beamDeps ++ ownPlugins);
dontStrip = true;
# The following are used by rebar3-nix-bootstrap
inherit compilePorts;
buildPlugins = ownPlugins;
@ -47,7 +48,7 @@ let
configurePhase = ''
runHook preConfigure
rebar3-nix-bootstrap
${erlang}/bin/escript ${rebar3.bootstrapper}
runHook postConfigure
'';
@ -81,7 +82,7 @@ let
passthru = {
packageName = name;
env = shell self;
inherit erlangDeps;
inherit beamDeps;
};
});
in

16
pkgs/development/beam-modules/default.nix Normal file
View file

@ -0,0 +1,16 @@
{ stdenv, pkgs }:
let
self = rec {
hexPackages = import ./hex-packages.nix { stdenv = stdenv; callPackage = self.callPackage; pkgs = pkgs; };
callPackage = pkgs.lib.callPackageWith (pkgs // self // hexPackages);
buildRebar3 = callPackage ./build-rebar3.nix {};
buildHex = callPackage ./build-hex.nix {};
buildErlangMk = callPackage ./build-erlang-mk.nix {};
buildMix = callPackage ./build-mix.nix {};
## Non hex packages
hex = callPackage ./hex {};
webdriver = callPackage ./webdriver {};
};
in self // self.hexPackages

0
pkgs/development/tools/build-managers/rebar3/fetch-hex.nix → pkgs/development/beam-modules/fetch-hex.nix
View file

29449
pkgs/development/beam-modules/hex-packages.nix Normal file
View file

File diff suppressed because it is too large Load diff

23
pkgs/development/beam-modules/hex-registry-snapshot.nix Normal file
View file

@ -0,0 +1,23 @@
{stdenv, writeText, fetchFromGitHub }:
stdenv.mkDerivation rec {
name = "hex-registry";
rev = "59b836d";
version = "0.0.0+build.${rev}";
src = fetchFromGitHub {
owner = "erlang-nix";
repo = "hex-pm-registry-snapshots";
inherit rev;
sha256 = "1l8m6ckn5ivhfiv3k4dymi6b7wg511fwymnpxd6ymfd39dq0n5b0";
};
installPhase = ''
mkdir -p "$out/var/hex"
zcat "registry.ets.gz" > "$out/var/hex/registry.ets"
'';
setupHook = writeText "setupHook.sh" ''
export HEX_REGISTRY_SNAPSHOT="$1/var/hex/registry.ets"
'';
}

58
pkgs/development/beam-modules/hex/default.nix Normal file
View file

@ -0,0 +1,58 @@
{stdenv, fetchFromGitHub, writeText, elixir }:
let
shell = drv: stdenv.mkDerivation {
name = "interactive-shell-${drv.name}";
buildInputs = [ drv ];
};
pkg = self: stdenv.mkDerivation rec {
name = "hex";
version = "v0.11.3";
src = fetchFromGitHub {
owner = "hexpm";
repo = "hex";
rev = "f5e200ad95f030f0a7ab88a86545dd0dde1ee521";
sha256 = "0n4cgmnbmglarydls9pmxznbzp49pv85ncbd4f2lp1fm7qr08xfw";
};
setupHook = writeText "setupHook.sh" ''
addToSearchPath ERL_LIBS "$1/lib/erlang/lib/"
'';
dontStrip = true;
buildInputs = [ elixir ];
buildPhase = ''
runHook preBuild
export HEX_OFFLINE=1
export HEX_HOME=./
export MIX_ENV=prod
mix compile
runHook postBuild
'';
installPhase = ''
runHook preInstall
mkdir -p $out/lib/erlang/lib
cp -r ./_build/prod/lib/hex $out/lib/erlang/lib/
runHook postInstall
'';
meta = {
description = "Package manager for the Erlang VM https://hex.pm";
license = stdenv.lib.licenses.mit;
homepage = "https://github.com/hexpm/hex";
maintainers = with stdenv.lib.maintainers; [ ericbmerritt ];
};
passthru = {
env = shell self;
};
};
in stdenv.lib.fix pkg

112
pkgs/development/beam-modules/mix-bootstrap Executable file
View file

@ -0,0 +1,112 @@
#!/usr/bin/env escript
%% -*- erlang-indent-level: 4;indent-tabs-mode: nil -*-
%%! -smp enable
%%% ---------------------------------------------------------------------------
%%% @doc
%%% The purpose of this command is to prepare a rebar3 project so that
%%% rebar3 understands that the dependencies are all already
%%% installed. If you want a hygienic build on nix then you must run
%%% this command before running rebar3. I suggest that you add a
%%% `Makefile` to your project and have the bootstrap command be a
%%% dependency of the build commands. See the nix documentation for
%%% more information.
%%%
%%% This command designed to have as few dependencies as possible so
%%% that it can be a dependency of root level packages like rebar3. To
%%% that end it does many things in a fairly simplistic way. That is
%%% by design.
%%%
%%% ### Assumptions
%%%
%%% This command makes the following assumptions:
%%%
%%% * It is run in a nix-shell or nix-build environment
%%% * that all dependencies have been added to the ERL_LIBS
%%% Environment Variable
-record(data, {version
, erl_libs
, root
, name
, registry_snapshot}).
-define(LOCAL_HEX_REGISTRY, "registry.ets").
main(Args) ->
{ok, RequiredData} = gather_required_data_from_the_environment(Args),
ok = bootstrap_libs(RequiredData).
%% @doc
%% This takes an app name in the standard OTP <name>-<version> format
%% and returns just the app name. Why? because rebar is doesn't
%% respect OTP conventions in some cases.
-spec fixup_app_name(file:name()) -> string().
fixup_app_name(Path) ->
BaseName = filename:basename(Path),
case string:tokens(BaseName, "-") of
[Name, _Version] -> Name;
Name -> Name
end.
-spec gather_required_data_from_the_environment([string()]) -> {ok, #data{}}.
gather_required_data_from_the_environment(_) ->
{ok, #data{ version = guard_env("version")
, erl_libs = os:getenv("ERL_LIBS", [])
, root = code:root_dir()
, name = guard_env("name")
, registry_snapshot = guard_env("HEX_REGISTRY_SNAPSHOT")}}.
-spec guard_env(string()) -> string().
guard_env(Name) ->
case os:getenv(Name) of
false ->
stderr("Expected Environment variable ~s! Are you sure you are "
"running in a Nix environment? Either a nix-build, "
"nix-shell, etc?~n", [Name]),
erlang:halt(1);
Variable ->
Variable
end.
-spec bootstrap_libs(#data{}) -> ok.
bootstrap_libs(#data{erl_libs = ErlLibs}) ->
io:format("Bootstrapping dependent libraries~n"),
Target = "_build/prod/lib/",
Paths = string:tokens(ErlLibs, ":"),
CopiableFiles =
lists:foldl(fun(Path, Acc) ->
gather_directory_contents(Path) ++ Acc
end, [], Paths),
lists:foreach(fun (Path) ->
ok = link_app(Path, Target)
end, CopiableFiles).
-spec gather_directory_contents(string()) -> [{string(), string()}].
gather_directory_contents(Path) ->
{ok, Names} = file:list_dir(Path),
lists:map(fun(AppName) ->
{filename:join(Path, AppName), fixup_app_name(AppName)}
end, Names).
%% @doc
%% Makes a symlink from the directory pointed at by Path to a
%% directory of the same name in Target. So if we had a Path of
%% {`foo/bar/baz/bash`, `baz`} and a Target of `faz/foo/foos`, the symlink
%% would be `faz/foo/foos/baz`.
-spec link_app({string(), string()}, string()) -> ok.
link_app({Path, TargetFile}, TargetDir) ->
Target = filename:join(TargetDir, TargetFile),
ok = make_symlink(Path, Target).
-spec make_symlink(string(), string()) -> ok.
make_symlink(Path, TargetFile) ->
file:delete(TargetFile),
ok = filelib:ensure_dir(TargetFile),
io:format("Making symlink from ~s to ~s~n", [Path, TargetFile]),
ok = file:make_symlink(Path, TargetFile).
%% @doc
%% Write the result of the format string out to stderr.
-spec stderr(string(), [term()]) -> ok.
stderr(FormatStr, Args) ->
io:put_chars(standard_error, io_lib:format(FormatStr, Args)).

34
pkgs/development/beam-modules/pgsql/default.nix Normal file
View file

@ -0,0 +1,34 @@
{stdenv, fetchFromGitHub, buildRebar3 }:
let
shell = drv: stdenv.mkDerivation {
name = "interactive-shell-${drv.name}";
buildInputs = [ drv ];
};
pkg = self: buildRebar3 rec {
name = "pgsql";
version = "25+beta.2";
src = fetchFromGitHub {
owner = "semiocast";
repo = "pgsql";
rev = "14f632bc89e464d82ce3ef12a67ed8c2adb5b60c";
sha256 = "17dcahiwlw61zhy8aq9rn46lwb35fb9q3372s4wmz01czm8c348w";
};
dontStrip = true;
meta = {
description = "Erlang PostgreSQL Driver";
license = stdenv.lib.licenses.mit;
homepage = "https://github.com/semiocast/pgsql";
maintainers = with stdenv.lib.maintainers; [ ericbmerritt ];
};
passthru = {
env = shell self;
};
};
in stdenv.lib.fix pkg

0
pkgs/development/erlang-modules/webdriver/default.nix → pkgs/development/beam-modules/webdriver/default.nix
View file

14
pkgs/development/erlang-modules/default.nix
View file

@ -1,14 +0,0 @@
{ stdenv, pkgs }: #? import <nixpkgs> {} }:
let
self = rec {
hex = import ./hex-packages.nix { stdenv = stdenv; callPackage = self.callPackage; };
callPackage = pkgs.lib.callPackageWith (pkgs // self // hex);
buildRebar3 = callPackage ./build-rebar3.nix {};
buildHex = callPackage ./build-hex.nix {};
## Non hex packages
webdriver = callPackage ./webdriver {};
};
in self // self.hex

3027
pkgs/development/erlang-modules/hex-packages.nix
View file

File diff suppressed because it is too large Load diff

9
pkgs/development/interpreters/elixir/default.nix
View file

@ -1,4 +1,5 @@
{ stdenv, fetchurl, erlang, rebar, makeWrapper, coreutils, curl, bash }:
{ stdenv, fetchurl, erlang, rebar, makeWrapper, coreutils, curl, bash,
debugInfo ? false }:
stdenv.mkDerivation rec {
name = "elixir-${version}";
@ -16,6 +17,12 @@ stdenv.mkDerivation rec {
LANG = "en_US.UTF-8";
LC_TYPE = "en_US.UTF-8";
setupHook = ./setup-hook.sh;
buildFlags = if debugInfo
then "ERL_COMPILER_OPTIONS=debug_info"
else "";
preBuild = ''
# The build process uses ./rebar. Link it to the nixpkgs rebar
rm -v rebar

5
pkgs/development/interpreters/elixir/setup-hook.sh Normal file
View file

@ -0,0 +1,5 @@
addErlLibPath() {
addToSearchPath ERL_LIBS $1/lib/elixir/lib
}
envHooks+=(addErlLibPath)

53
pkgs/development/tools/build-managers/rebar3/default.nix
View file

@ -1,73 +1,62 @@
{ stdenv, writeText, callPackage, fetchurl,
fetchHex, erlang, hermeticRebar3 ? true, rebar3-nix-bootstrap, tree, fetchFromGitHub }:
fetchHex, erlang, hermeticRebar3 ? true,
tree, fetchFromGitHub, hexRegistrySnapshot }:
let
version = "3.0.0-beta.4";
registrySnapshot = callPackage ./registrySnapshot.nix { };
version = "3.1.0";
bootstrapper = ./rebar3-nix-bootstrap;
# TODO: all these below probably should go into nixpkgs.erlangModules.sources.*
# {erlware_commons, "0.16.0"},
erlware_commons = fetchHex {
pkg = "erlware_commons";
version = "0.16.0";
sha256 = "0kh24d0001390wfx28d0xa874vrsfvjgj41g315vg4hac632krxx";
version = "0.19.0";
sha256 = "1gfsy9bbhjb94c5ghff2niamn93x2x08lnklh6pp7sfr5i0gkgsv";
};
# {ssl_verify_hostname, "1.0.5"},
ssl_verify_hostname = fetchHex {
pkg = "ssl_verify_hostname";
version = "1.0.5";
sha256 = "1gzavzqzljywx4l59gvhkjbr1dip4kxzjjz1s4wsn42f2kk13jzj";
};
# {certifi, "0.1.1"},
certifi = fetchHex {
pkg = "certifi";
version = "0.1.1";
sha256 = "0afylwqg74gprbg116asz0my2nipmki0512c8mdiq6xdiyjdvlg6";
version = "0.4.0";
sha256 = "04bnvsbssdcf6b9h9bfglflds7j0gx6q5igl1xxhx6fnwaz37hhw";
};
# {providers, "1.5.0"},
providers = fetchHex {
pkg = "providers";
version = "1.5.0";
sha256 = "1hc8sp2l1mmx9dfpmh1f8j9hayfg7541rmx05wb9cmvxvih7zyvf";
version = "1.6.0";
sha256 = "0byfa1h57n46jilz4q132j0vk3iqc0v1vip89li38gb1k997cs0g";
};
# {getopt, "0.8.2"},
getopt = fetchHex {
pkg = "getopt";
version = "0.8.2";
sha256 = "1xw30h59zbw957cyjd8n50hf9y09jnv9dyry6x3avfwzcyrnsvkk";
};
# {bbmustache, "1.0.4"},
bbmustache = fetchHex {
pkg = "bbmustache";
version = "1.0.4";
sha256 = "04lvwm7f78x8bys0js33higswjkyimbygp4n72cxz1kfnryx9c03";
};
# {relx, "3.8.0"},
relx = fetchHex {
pkg = "relx";
version = "3.8.0";
sha256 = "0y89iirjz3kc1rzkdvc6p3ssmwcm2hqgkklhgm4pkbc14fcz57hq";
version = "3.17.0";
sha256 = "1xjybi93m8gj9f9z3lkc7xbg3k5cw56yl78rcz5qfirr0223sby2";
};
# {cf, "0.2.1"},
cf = fetchHex {
pkg = "cf";
version = "0.2.1";
sha256 = "19d0yvg8wwa57cqhn3vqfvw978nafw8j2rvb92s3ryidxjkrmvms";
};
# {cth_readable, "1.1.0"},
cth_readable = fetchHex {
pkg = "cth_readable";
version = "1.0.1";
sha256 = "1cnc4fbypckqllfi5h73rdb24dz576k3177gzvp1kbymwkp1xcz1";
version = "1.2.2";
sha256 = "0kb9v4998liwyidpjkhcg1nin6djjzxcx6b313pbjicbp4r58n3p";
};
# {eunit_formatters, "0.2.0"}
eunit_formatters = fetchHex {
pkg = "eunit_formatters";
version = "0.2.0";
sha256 = "03kiszlbgzscfd2ns7na6bzbfzmcqdb5cx3p6qy3657jk2fai332";
version = "0.3.1";
sha256 = "0cg9dasv60v09q3q4wja76pld0546mhmlpb0khagyylv890hg934";
};
# {eunit_formatters, "0.2.0"}
rebar3_hex = fetchHex {
pkg = "rebar3_hex";
version = "1.12.0";
@ -81,19 +70,21 @@ stdenv.mkDerivation {
src = fetchurl {
url = "https://github.com/rebar/rebar3/archive/${version}.tar.gz";
sha256 = "0px66scjdia9aaa5z36qzxb848r56m0k98g0bxw065a2narsh4xy";
sha256 = "0r4wpnpi81ha4iirv9hcif3vrgc82qd51kah7rnhvpym55wcy9ml";
};
inherit bootstrapper;
patches = if hermeticRebar3 == true
then [ ./hermetic-bootstrap.patch ./hermetic-rebar3.patch ]
else [];
buildInputs = [ erlang tree ];
propagatedBuildInputs = [ registrySnapshot rebar3-nix-bootstrap ];
propagatedBuildInputs = [ hexRegistrySnapshot ];
postPatch = ''
echo postPatch
rebar3-nix-bootstrap registry-only
${erlang}/bin/escript ${bootstrapper} registry-only
echo "$ERL_LIBS"
mkdir -p _build/default/lib/
mkdir -p _build/default/plugins

147
pkgs/development/tools/build-managers/rebar3/hermetic-bootstrap.patch
View file

@ -1,39 +1,61 @@
diff --git a/bootstrap b/bootstrap
index 25bd658..b2a986b 100755
index 35759b0..939c838 100755
--- a/bootstrap
+++ b/bootstrap
@@ -8,9 +8,6 @@ main(_Args) ->
@@ -7,9 +7,11 @@ main(_) ->
application:start(asn1),
application:start(public_key),
application:start(ssl),
- inets:start(),
- inets:start(httpc, [{profile, rebar}]),
- set_httpc_options(),
+ %% Removed for hermeticity on Nix
+ %%
+ %% inets:start(),
+ %% inets:start(httpc, [{profile, rebar}]),
+ %% set_httpc_options(),
%% Fetch and build deps required to build rebar3
BaseDeps = [{providers, []}
@@ -33,7 +30,6 @@ main(_Args) ->
setup_env(),
os:putenv("REBAR_PROFILE", "bootstrap"),
- rebar3:run(["update"]),
{ok, State} = rebar3:run(["compile"]),
reset_env(),
os:putenv("REBAR_PROFILE", ""),
@@ -71,33 +67,7 @@ fetch_and_compile({Name, ErlFirstFiles}, Deps) ->
@@ -74,12 +76,12 @@ default_registry_file() ->
filename:join([CacheDir, "hex", "default", "registry"]).
fetch_and_compile({Name, ErlFirstFiles}, Deps) ->
- case lists:keyfind(Name, 1, Deps) of
- {Name, Vsn} ->
- ok = fetch({pkg, atom_to_binary(Name, utf8), list_to_binary(Vsn)}, Name);
- {Name, _, Source} ->
- ok = fetch(Source, Name)
- end,
+ %% case lists:keyfind(Name, 1, Deps) of
+ %% {Name, Vsn} ->
+ %% ok = fetch({pkg, atom_to_binary(Name, utf8), list_to_binary(Vsn)}, Name);
+ %% {Name, _, Source} ->
+ %% ok = fetch(Source, Name)
+ %% end,
%% Hack: erlware_commons depends on a .script file to check if it is being built with
%% rebar2 or rebar3. But since rebar3 isn't built yet it can't get the vsn with get_key.
@@ -88,63 +90,63 @@ fetch_and_compile({Name, ErlFirstFiles}, Deps) ->
compile(Name, ErlFirstFiles).
fetch({pkg, Name, Vsn}, App) ->
-fetch({pkg, Name, Vsn}, App) ->
- Dir = filename:join([filename:absname("_build/default/lib/"), App]),
- CDN = "https://s3.amazonaws.com/s3.hex.pm/tarballs",
- Package = binary_to_list(<<Name/binary, "-", Vsn/binary, ".tar">>),
- Url = string:join([CDN, Package], "/"),
- case request(Url) of
- {ok, Binary} ->
- {ok, Contents} = extract(Binary),
- ok = erl_tar:extract({binary, Contents}, [{cwd, Dir}, compressed]);
- _ ->
- io:format("Error: Unable to fetch package ~p ~p~n", [Name, Vsn])
- case filelib:is_dir(Dir) of
- false ->
- CDN = "https://s3.amazonaws.com/s3.hex.pm/tarballs",
- Package = binary_to_list(<<Name/binary, "-", Vsn/binary, ".tar">>),
- Url = string:join([CDN, Package], "/"),
- case request(Url) of
- {ok, Binary} ->
- {ok, Contents} = extract(Binary),
- ok = erl_tar:extract({binary, Contents}, [{cwd, Dir}, compressed]);
- _ ->
- io:format("Error: Unable to fetch package ~p ~p~n", [Name, Vsn])
- end;
- true ->
- io:format("Dependency ~s already exists~n", [Name])
- end.
-
-extract(Binary) ->
@ -51,14 +73,17 @@ index 25bd658..b2a986b 100755
- Error ->
- Error
- end.
+ ok.
get_rebar_config() ->
{ok, [[Home]]} = init:get_argument(home),
@@ -109,20 +79,6 @@ get_rebar_config() ->
[]
end.
-
-get_rebar_config() ->
- {ok, [[Home]]} = init:get_argument(home),
- ConfDir = filename:join(Home, ".config/rebar3"),
- case file:consult(filename:join(ConfDir, "rebar.config")) of
- {ok, Config} ->
- Config;
- _ ->
- []
- end.
-
-get_http_vars(Scheme) ->
- proplists:get_value(Scheme, get_rebar_config(), []).
-
@ -72,7 +97,63 @@ index 25bd658..b2a986b 100755
-set_httpc_options(Scheme, Proxy) ->
- {ok, {_, _, Host, Port, _, _}} = http_uri:parse(Proxy),
- httpc:set_options([{Scheme, {{Host, Port}, []}}], rebar).
-
+%% fetch({pkg, Name, Vsn}, App) ->
+%% Dir = filename:join([filename:absname("_build/default/lib/"), App]),
+%% case filelib:is_dir(Dir) of
+%% false ->
+%% CDN = "https://s3.amazonaws.com/s3.hex.pm/tarballs",
+%% Package = binary_to_list(<<Name/binary, "-", Vsn/binary, ".tar">>),
+%% Url = string:join([CDN, Package], "/"),
+%% case request(Url) of
+%% {ok, Binary} ->
+%% {ok, Contents} = extract(Binary),
+%% ok = erl_tar:extract({binary, Contents}, [{cwd, Dir}, compressed]);
+%% _ ->
+%% io:format("Error: Unable to fetch package ~p ~p~n", [Name, Vsn])
+%% end;
+%% true ->
+%% io:format("Dependency ~s already exists~n", [Name])
+%% end.
+
+%% extract(Binary) ->
+%% {ok, Files} = erl_tar:extract({binary, Binary}, [memory]),
+%% {"contents.tar.gz", Contents} = lists:keyfind("contents.tar.gz", 1, Files),
+%% {ok, Contents}.
+
+%% request(Url) ->
+%% case httpc:request(get, {Url, []},
+%% [{relaxed, true}],
+%% [{body_format, binary}],
+%% rebar) of
+%% {ok, {{_Version, 200, _Reason}, _Headers, Body}} ->
+%% {ok, Body};
+%% Error ->
+%% Error
+%% end.
+
+%% get_rebar_config() ->
+%% {ok, [[Home]]} = init:get_argument(home),
+%% ConfDir = filename:join(Home, ".config/rebar3"),
+%% case file:consult(filename:join(ConfDir, "rebar.config")) of
+%% {ok, Config} ->
+%% Config;
+%% _ ->
+%% []
+%% end.
+
+%% get_http_vars(Scheme) ->
+%% proplists:get_value(Scheme, get_rebar_config(), []).
+
+%% set_httpc_options() ->
+%% set_httpc_options(https_proxy, get_http_vars(https_proxy)),
+%% set_httpc_options(proxy, get_http_vars(http_proxy)).
+
+%% set_httpc_options(_, []) ->
+%% ok;
+
+%% set_httpc_options(Scheme, Proxy) ->
+%% {ok, {_, _, Host, Port, _, _}} = http_uri:parse(Proxy),
+%% httpc:set_options([{Scheme, {{Host, Port}, []}}], rebar).
compile(App, FirstFiles) ->
Dir = filename:join(filename:absname("_build/default/lib/"), App),
filelib:ensure_dir(filename:join([Dir, "ebin", "dummy.beam"])),

67
pkgs/development/tools/build-managers/rebar3/hermetic-rebar3.patch
View file

@ -1,8 +1,8 @@
diff --git a/src/rebar3.erl b/src/rebar3.erl
index 2b73844..af1d871 100644
diff a/src/rebar3.erl b/src/rebar3.erl
index c1a1ae4..1bf1ea0 100644
--- a/src/rebar3.erl
+++ b/src/rebar3.erl
@@ -282,9 +282,11 @@ start_and_load_apps(Caller) ->
@@ -294,9 +294,11 @@ start_and_load_apps(Caller) ->
ensure_running(crypto, Caller),
ensure_running(asn1, Caller),
ensure_running(public_key, Caller),
@ -10,21 +10,14 @@ index 2b73844..af1d871 100644
- inets:start(),
- inets:start(httpc, [{profile, rebar}]).
+ ensure_running(ssl, Caller).
+%% Removed due to the hermicity requirements of Nix
+%% Removed due to the hermicity requirements of Nix
+%%
+%% inets:start(),
+%% inets:start(httpc, [{profile, rebar}]).
ensure_running(App, Caller) ->
case application:start(App) of
@@ -339,4 +341,4 @@ safe_define_test_macro(Opts) ->
test_defined([{d, 'TEST'}|_]) -> true;
test_defined([{d, 'TEST', true}|_]) -> true;
test_defined([_|Rest]) -> test_defined(Rest);
-test_defined([]) -> false.
\ No newline at end of file
+test_defined([]) -> false.
diff --git a/src/rebar_hermicity.erl b/src/rebar_hermicity.erl
diff a/src/rebar_hermicity.erl b/src/rebar_hermicity.erl
new file mode 100644
index 0000000..d814e2a
--- /dev/null
@ -72,37 +65,29 @@ index 0000000..d814e2a
+ "are as follows:", []),
+ ?ERROR("Requesnt: ~p ~s", [Method, Url]),
+ erlang:halt(1).
diff --git a/src/rebar_pkg_resource.erl b/src/rebar_pkg_resource.erl
index 4f55ad1..f76fd5d 100644
diff a/src/rebar_pkg_resource.erl b/src/rebar_pkg_resource.erl
index ec7e09d..03be343 100644
--- a/src/rebar_pkg_resource.erl
+++ b/src/rebar_pkg_resource.erl
@@ -100,10 +100,10 @@ make_vsn(_) ->
@@ -104,7 +104,7 @@ make_vsn(_) ->
{error, "Replacing version of type pkg not supported."}.
request(Url, ETag) ->
- case httpc:request(get, {Url, [{"if-none-match", ETag} || ETag =/= false]},
- [{ssl, ssl_opts(Url)}, {relaxed, true}],
- [{body_format, binary}],
- rebar) of
+ case rebar_hermicity:request(get, {Url, [{"if-none-match", ETag} || ETag =/= false]},
+ [{ssl, ssl_opts(Url)}, {relaxed, true}],
+ [{body_format, binary}],
+ rebar) of
{ok, {{_Version, 200, _Reason}, Headers, Body}} ->
?DEBUG("Successfully downloaded ~s", [Url]),
{"etag", ETag1} = lists:keyfind("etag", 1, Headers),
diff --git a/src/rebar_prv_update.erl b/src/rebar_prv_update.erl
index 6637ebe..d82c1d8 100644
- case httpc:request(get, {Url, [{"if-none-match", ETag} || ETag =/= false]++[{"User-Agent", rebar_utils:user_agent()}]},
+ case rebar_hermicity:request(get, {Url, [{"if-none-match", ETag} || ETag =/= false]++[{"User-Agent", rebar_utils:user_agent()}]},
[{ssl, ssl_opts(Url)}, {relaxed, true}],
[{body_format, binary}],
rebar) of
diff a/src/rebar_prv_update.erl b/src/rebar_prv_update.erl
index 5e1e253..ea25b9e 100644
--- a/src/rebar_prv_update.erl
+++ b/src/rebar_prv_update.erl
@@ -44,8 +44,8 @@ do(State) ->
TmpFile = filename:join(TmpDir, "packages.gz"),
Url = rebar_state:get(State, rebar_packages_cdn, ?DEFAULT_HEX_REGISTRY),
- case httpc:request(get, {Url, []},
- [], [{stream, TmpFile}, {sync, true}],
+ case rebar_hermicity:request(get, {Url, []},
+ [], [{stream, TmpFile}, {sync, true}],
rebar) of
{ok, saved_to_file} ->
{ok, Data} = file:read_file(TmpFile),
@@ -52,7 +52,7 @@ do(State) ->
case rebar_utils:url_append_path(CDN, ?REMOTE_REGISTRY_FILE) of
{ok, Url} ->
?DEBUG("Fetching registry from ~p", [Url]),
- case httpc:request(get, {Url, [{"User-Agent", rebar_utils:user_agent()}]},
+ case rebar_hermicity:request(get, {Url, [{"User-Agent", rebar_utils:user_agent()}]},
[], [{stream, TmpFile}, {sync, true}],
rebar) of
{ok, saved_to_file} ->

255
pkgs/development/tools/build-managers/rebar3/rebar3-nix-bootstrap Executable file
View file

@ -0,0 +1,255 @@
#!/usr/bin/env escript
%% -*- erlang-indent-level: 4;indent-tabs-mode: nil -*-
%%! -smp enable
%%% ---------------------------------------------------------------------------
%%% @doc
%%% The purpose of this command is to prepare a rebar3 project so that
%%% rebar3 understands that the dependencies are all already
%%% installed. If you want a hygienic build on nix then you must run
%%% this command before running rebar3. I suggest that you add a
%%% `Makefile` to your project and have the bootstrap command be a
%%% dependency of the build commands. See the nix documentation for
%%% more information.
%%%
%%% This command designed to have as few dependencies as possible so
%%% that it can be a dependency of root level packages like rebar3. To
%%% that end it does many things in a fairly simplistic way. That is
%%% by design.
%%%
%%% ### Assumptions
%%%
%%% This command makes the following assumptions:
%%%
%%% * It is run in a nix-shell or nix-build environment
%%% * that all dependencies have been added to the ERL_LIBS
%%% Environment Variable
-record(data, {version
, registry_only = false
, compile_ports
, erl_libs
, plugins
, root
, name
, registry_snapshot}).
-define(HEX_REGISTRY_PATH, ".cache/rebar3/hex/default/registry").
main(Args) ->
{ok, ArgData} = parse_args(Args),
{ok, RequiredData} = gather_required_data_from_the_environment(ArgData),
do_the_bootstrap(RequiredData).
%% @doc There are two modes 'registery_only' where the register is
%% created from hex and everything else.
-spec do_the_bootstrap(#data{}) -> ok.
do_the_bootstrap(RequiredData = #data{registry_only = true}) ->
ok = bootstrap_registry(RequiredData);
do_the_bootstrap(RequiredData) ->
ok = bootstrap_registry(RequiredData),
ok = bootstrap_configs(RequiredData),
ok = bootstrap_plugins(RequiredData),
ok = bootstrap_libs(RequiredData).
%% @doc
%% Argument parsing is super simple only because we want to keep the
%% dependencies minimal. For now there can be two entries on the
%% command line, "registery-only"
-spec parse_args([string()]) -> #data{}.
parse_args(["registry-only"]) ->
{ok, #data{registry_only = true}};
parse_args([]) ->
{ok, #data{registry_only = false}};
parse_args(Args) ->
io:format("Unexpected command line arguments passed in: ~p~n", [Args]),
erlang:halt(120).
-spec bootstrap_configs(#data{}) -> ok.
bootstrap_configs(RequiredData)->
io:format("Boostrapping app and rebar configurations~n"),
ok = if_single_app_project_update_app_src_version(RequiredData),
ok = if_compile_ports_add_pc_plugin(RequiredData).
-spec bootstrap_plugins(#data{}) -> ok.
bootstrap_plugins(#data{plugins = Plugins}) ->
io:format("Bootstrapping rebar3 plugins~n"),
Target = "_build/default/plugins/",
Paths = string:tokens(Plugins, " "),
CopiableFiles =
lists:foldl(fun(Path, Acc) ->
gather_dependency(Path) ++ Acc
end, [], Paths),
lists:foreach(fun (Path) ->
ok = link_app(Path, Target)
end, CopiableFiles).
-spec bootstrap_libs(#data{}) -> ok.
bootstrap_libs(#data{erl_libs = ErlLibs}) ->
io:format("Bootstrapping dependent librariesXXXX~n"),
Target = "_build/default/lib/",
Paths = string:tokens(ErlLibs, ":"),
CopiableFiles =
lists:foldl(fun(Path, Acc) ->
gather_directory_contents(Path) ++ Acc
end, [], Paths),
lists:foreach(fun (Path) ->
ok = link_app(Path, Target)
end, CopiableFiles).
-spec gather_dependency(string()) -> [{string(), string()}].
gather_dependency(Path) ->
FullLibrary = filename:join(Path, "lib/erlang/lib/"),
case filelib:is_dir(FullLibrary) of
true ->
gather_directory_contents(FullLibrary);
false ->
[raw_hex(Path)]
end.
-spec raw_hex(string()) -> {string(), string()}.
raw_hex(Path) ->
[_, Name] = re:split(Path, "-hex-source-"),
{Path, erlang:binary_to_list(Name)}.
-spec gather_directory_contents(string()) -> [{string(), string()}].
gather_directory_contents(Path) ->
{ok, Names} = file:list_dir(Path),
lists:map(fun(AppName) ->
{filename:join(Path, AppName), fixup_app_name(AppName)}
end, Names).
%% @doc
%% Makes a symlink from the directory pointed at by Path to a
%% directory of the same name in Target. So if we had a Path of
%% {`foo/bar/baz/bash`, `baz`} and a Target of `faz/foo/foos`, the symlink
%% would be `faz/foo/foos/baz`.
-spec link_app({string(), string()}, string()) -> ok.
link_app({Path, TargetFile}, TargetDir) ->
Target = filename:join(TargetDir, TargetFile),
make_symlink(Path, Target).
-spec make_symlink(string(), string()) -> ok.
make_symlink(Path, TargetFile) ->
file:delete(TargetFile),
ok = filelib:ensure_dir(TargetFile),
io:format("Making symlink from ~s to ~s~n", [Path, TargetFile]),
ok = file:make_symlink(Path, TargetFile).
%% @doc
%% This takes an app name in the standard OTP <name>-<version> format
%% and returns just the app name. Why? because rebar is doesn't
%% respect OTP conventions in some cases.
-spec fixup_app_name(string()) -> string().
fixup_app_name(FileName) ->
case string:tokens(FileName, "-") of
[Name] -> Name;
[Name, _Version] -> Name
end.
-spec bootstrap_registry(#data{}) -> ok.
bootstrap_registry(#data{registry_snapshot = RegistrySnapshot}) ->
io:format("Bootstrapping Hex Registry for Rebar~n"),
make_sure_registry_snapshot_exists(RegistrySnapshot),
filelib:ensure_dir(?HEX_REGISTRY_PATH),
ok = case filelib:is_file(?HEX_REGISTRY_PATH) of
true ->
file:delete(?HEX_REGISTRY_PATH);
false ->
ok
end,
ok = file:make_symlink(RegistrySnapshot,
?HEX_REGISTRY_PATH).
-spec make_sure_registry_snapshot_exists(string()) -> ok.
make_sure_registry_snapshot_exists(RegistrySnapshot) ->
case filelib:is_file(RegistrySnapshot) of
true ->
ok;
false ->
stderr("Registry snapshot (~s) does not exist!", [RegistrySnapshot]),
erlang:halt(1)
end.
-spec gather_required_data_from_the_environment(#data{}) -> {ok, map()}.
gather_required_data_from_the_environment(ArgData) ->
{ok, ArgData#data{ version = guard_env("version")
, erl_libs = os:getenv("ERL_LIBS", [])
, plugins = os:getenv("buildPlugins", [])
, root = code:root_dir()
, name = guard_env("name")
, compile_ports = nix2bool(os:getenv("compilePorts", ""))
, registry_snapshot = guard_env("HEX_REGISTRY_SNAPSHOT")}}.
-spec nix2bool(any()) -> boolean().
nix2bool("1") ->
true;
nix2bool("") ->
false.
-spec guard_env(string()) -> string().
guard_env(Name) ->
case os:getenv(Name) of
false ->
stderr("Expected Environment variable ~s! Are you sure you are "
"running in a Nix environment? Either a nix-build, "
"nix-shell, etc?~n", [Name]),
erlang:halt(1);
Variable ->
Variable
end.
%% @doc
%% If the compile ports flag is set, rewrite the rebar config to
%% include the 'pc' plugin.
-spec if_compile_ports_add_pc_plugin(#data{}) -> ok.
if_compile_ports_add_pc_plugin(#data{compile_ports = true}) ->
ConfigTerms = add_pc_to_plugins(read_rebar_config()),
Text = lists:map(fun(Term) -> io_lib:format("~tp.~n", [Term]) end,
ConfigTerms),
file:write_file("rebar.config", Text);
if_compile_ports_add_pc_plugin(_) ->
ok.
-spec add_pc_to_plugins([term()]) -> [term()].
add_pc_to_plugins(Config) ->
PluginList = case lists:keysearch(plugins, 1, Config) of
{ok, {plugins, ExistingPluginList}} -> ExistingPluginList;
_ -> []
end,
lists:keystore(plugins, 1, Config, {plugins, [pc | PluginList]}).
-spec read_rebar_config() -> [term()].
read_rebar_config() ->
case file:consult("rebar.config") of
{ok, Terms} ->
Terms;
_ ->
stderr("Unable to read rebar config!", []),
erlang:halt(1)
end.
-spec if_single_app_project_update_app_src_version(#data{}) -> ok.
if_single_app_project_update_app_src_version(#data{name = Name,
version = Version}) ->
SrcFile = filename:join("src",
lists:concat([Name, ".app.src"])),
case filelib:is_file(SrcFile) of
true ->
update_app_src_with_version(SrcFile, Version);
false ->
ok
end.
-spec update_app_src_with_version(string(), string()) -> ok.
update_app_src_with_version(SrcFile, Version) ->
{ok, [{application, Name, Details}]} = file:consult(SrcFile),
NewDetails = lists:keyreplace(vsn, 1, Details, {vsn, Version}),
ok = file:write_file(SrcFile, io_lib:fwrite("~p.\n", [{application, Name, NewDetails}])).
%% @doc
%% Write the result of the format string out to stderr.
-spec stderr(string(), [term()]) -> ok.
stderr(FormatStr, Args) ->
io:put_chars(standard_error, io_lib:format(FormatStr, Args)).

6
pkgs/development/tools/erlang/cuter/default.nix
View file

@ -1,5 +1,5 @@
{stdenv, autoconf, which, writeText, makeWrapper, fetchFromGitHub, erlang,
erlangPackages, z3, python27 }:
beamPackages, z3, python27 }:
stdenv.mkDerivation rec {
name = "cuter";
@ -13,9 +13,9 @@ stdenv.mkDerivation rec {
};
setupHook = writeText "setupHook.sh" ''
addToSearchPath ERL_LIBS "$1/lib/erlang/lib/"
addToSearchPath ERL_LIBS "$1/lib/erlang/lib/"
'';
buildInputs = with erlangPackages; [ autoconf erlang z3 python27 makeWrapper which ];
buildInputs = with beamPackages; [ autoconf erlang z3 python27 makeWrapper which ];
buildFlags = "PWD=$(out)/lib/erlang/lib/cuter-${version} cuter_target";
configurePhase = ''

14
pkgs/development/tools/erlang/hex2nix/default.nix
View file

@ -2,16 +2,16 @@
buildRebar3 rec {
name = "hex2nix";
version = "0.0.3";
version = "0.0.5";
src = fetchFromGitHub {
owner = "erlang-nix";
repo = "hex2nix";
rev = "${version}";
sha256 = "1snlcb60al7fz3z4c4rqrb9gqdyihyhsrr90n40v9rdm98csry3k";
};
owner = "erlang-nix";
repo = "hex2nix";
rev = "${version}";
sha256 = "07bk18nib4xms8q1i4sv53drvlyllm47map4c95669lsh0j08sax";
};
erlangDeps = [ ibrowse jsx erlware_commons getopt ];
beamDeps = [ ibrowse jsx erlware_commons getopt ];
DEBUG=1;

24
pkgs/development/tools/erlang/rebar3-nix-bootstrap/default.nix
View file

@ -1,24 +0,0 @@
{stdenv, fetchFromGitHub, erlang }:
stdenv.mkDerivation rec {
name = "rebar3-nix-bootstrap";
version = "0.0.3";
src = fetchFromGitHub {
owner = "erlang-nix";
repo = "rebar3-nix-bootstrap";
rev = "${version}";
sha256 = "01yyaz104jj3mxx8k10q3rwpn2rh13q1ja5r0iq37qyjmg8xflhq";
};
buildInputs = [ erlang ];
installFlags = "PREFIX=$(out)";
meta = {
description = "Shim command to help bootstrap a rebar3 project on Nix";
license = stdenv.lib.licenses.asl20;
homepage = "https://github.com/erlang-nix/rebar3-nix-bootstrap";
maintainers = with stdenv.lib.maintainers; [ ericbmerritt ];
};
}

38
pkgs/development/tools/erlang/relx-exe/default.nix Normal file
View file

@ -0,0 +1,38 @@
{ stdenv, beamPackages, makeWrapper, fetchHex, erlang }:
beamPackages.buildRebar3 {
name = "relx-exe";
version = "3.18.0";
src = fetchHex {
pkg = "relx";
version = "3.18.0";
sha256 =
"e76e0446b8d1b113f2b7dcc713f032ccdf1dbda33d76edfeb19c2b6b686dcad7";
};
buildInputs = [ makeWrapper erlang ];
beamDeps = with beamPackages; [
providers_1_6_0
getopt_0_8_2
erlware_commons_0_19_0
cf_0_2_1
bbmustache_1_0_4
];
postBuild = ''
HOME=. rebar3 escriptize
'';
postInstall = ''
mkdir -p "$out/bin"
cp -r "_build/default/bin/relx" "$out/bin/relx"
'';
meta = {
description = "Executable command for Relx";
license = stdenv.lib.licenses.asl20;
homepage = "https://github.com/erlware/relx";
maintainers = with stdenv.lib.maintainers; [ ericbmerritt ];
};
}

14
pkgs/top-level/all-packages.nix
View file

@ -5321,14 +5321,16 @@ in
rebar = callPackage ../development/tools/build-managers/rebar { };
rebar3-open = callPackage ../development/tools/build-managers/rebar3 { hermeticRebar3 = false; };
rebar3 = callPackage ../development/tools/build-managers/rebar3 { hermeticRebar3 = true; };
rebar3-nix-bootstrap = callPackage ../development/tools/erlang/rebar3-nix-bootstrap { };
fetchHex = callPackage ../development/tools/build-managers/rebar3/fetch-hex.nix { };
hexRegistrySnapshot = callPackage ../development/beam-modules/hex-registry-snapshot.nix { };
fetchHex = callPackage ../development/beam-modules/fetch-hex.nix { };
erlangPackages = callPackage ../development/erlang-modules { };
cuter = erlangPackages.callPackage ../development/tools/erlang/cuter { };
hex2nix = erlangPackages.callPackage ../development/tools/erlang/hex2nix { };
beamPackages = callPackage ../development/beam-modules { };
hex2nix = beamPackages.callPackage ../development/tools/erlang/hex2nix { };
cuter = callPackage ../development/tools/erlang/cuter { };
elixir = callPackage ../development/interpreters/elixir { };
relxExe = callPackage ../development/tools/erlang/relx-exe {};
elixir = callPackage ../development/interpreters/elixir { debugInfo = true; };
groovy = callPackage ../development/interpreters/groovy { };