stable

Clone or download

Read-only

request #18503: Setup reproducible build and dev environments

The build and dev can now be constructed with the package manager Nix [0]. This contribution introduces changes in the build process and propose a way for developers to get a working development environment. Until now, the sources are built using a Docker image maintained outside of this source repo. This make hard to know what tools are needed to build the Tuleap sources at a specific version. The tools in this image are also only partially pinned so even if the build succeed some changes might be introduced which is something that should be avoided [1]. This image is replaced by a Nix derivation describing the desired environment. This derivation is then loaded using the Docker image nixos/nix. The Docker build cache is leveraged to avoid building multiple time the same derivation from scratch on a CI agent. For developers, a shell.nix file is added in the repo. It can be used with nix-shell [2] and gives you shell with all the tools needed to work on Tuleap (so all the build tools plus a few ones such as docker-compose). The scripts tools/utils/nix/dev-tools/php.sh and tools/utils/nix/dev-tools/node.sh are also provided to ease the integration with IDEs (this is an easy/quick&dirty way to achieve that, you might want to use your own way depending on your setup). To start working on Tuleap a developer needs 3 things: * a working Docker installation * git for the first clone of the source * Nix To test, you can: * install Nix [3][4] on your system and run nix-shell at the root of sources * use the Docker image nixos/nix and run it with something like `docker run --rm -ti -v "$(pwd)":/tuleap -w /tuleap nixos/nix nix-shell` (note that if you run this command as is and you generate files, they will be created as root) All the Tuleap dev tools accessible via the Makefile at the root of the repo are expected to work without errors. You might note that a bit of PHP code as been modified. Both changes are here to allow the PHPUnit unit test suite to work in a nixos/nix container with a pure nix-shell. The change in the User_SSHKeyValidator class makes the command compatible with a wider range of shell. The change skipping a test if /usr/bin/perl is not present is here because it seemed overkill to do changes in fileforge.pl for this specific use case. To work day to day with nix-shell you might want to take a look at direnv [5] potentially with nix-direnv [6] to get a faster and more persistent shell environnement. direnv also allows you to use a shell different than bash and keep all of your configurations. [0] https://nixos.org/ [1] https://reproducible-builds.org/ [2] https://nixos.org/manual/nix/unstable/command-ref/nix-shell.html [3] https://nixos.org/download.html#nix-quick-install [4] https://nixos.org/manual/nix/stable/#chap-quick-start [5] https://direnv.net/ [6] https://github.com/nix-community/nix-direnv Change-Id: Ic016d33f342252a88cf4493e3640b4004ed54dfd

Modified Files

Name
A shell.nix +10 −0 Go to diff View file
M src/common/User/User_SSHKeyValidator.class.php +1 −1 Go to diff View file
M tests/Jenkinsfile +20 −3 Go to diff View file
M tests/Jenkinsfile-nightly +19 −3 Go to diff View file
M tests/Jenkinsfile-security-taint-analysis +19 −3 Go to diff View file
M tests/actions.groovy +0 −13 Go to diff View file
M tests/e2e/distlp/wrap.sh +1 −1 Go to diff View file
M tests/e2e/full/wrap.sh +1 −1 Go to diff View file
M tests/e2e/full/wrap_for_dev_context.sh +1 −1 Go to diff View file
M tests/integration/bin/run-compose.sh +1 −1 Go to diff View file
M tests/psalm/psalm-config-plugins-git-ignore.php +1 −1 Go to diff View file
M tests/rest/bin/run-compose.sh +1 −1 Go to diff View file
M tests/soap/bin/run-compose.sh +1 −1 Go to diff View file
M tests/unit/common/FRS/FRSFileFactoryTest.php +3 −0 Go to diff View file
M tools/rpm/Makefile +1 −1 Go to diff View file
M tools/rpm/build_and_run_packages.sh +3 −6 Go to diff View file
M tools/rpm/tuleap.rhel7.spec +1 −0 Go to diff View file
M tools/utils/generate-mo.sh +2 −2 Go to diff View file
A tools/utils/nix/build-tools.dockerfile +11 −0 Go to diff View file
A tools/utils/nix/build-tools/build-tools-general.nix +15 −0 Go to diff View file
A tools/utils/nix/build-tools/build-tools-js.nix +5 −0 Go to diff View file
A tools/utils/nix/build-tools/build-tools-php.nix +46 −0 Go to diff View file
A tools/utils/nix/build-tools/default.nix +8 −0 Go to diff View file
A tools/utils/nix/dev-tools/default.nix +11 −0 Go to diff View file
A tools/utils/nix/dev-tools/dev-tools-docker.nix +5 −0 Go to diff View file
A tools/utils/nix/dev-tools/dev-tools-php.nix +7 −0 Go to diff View file
A tools/utils/nix/dev-tools/dev-tools-tests.nix +6 −0 Go to diff View file
A tools/utils/nix/dev-tools/node.sh +6 −0 Go to diff View file
A tools/utils/nix/dev-tools/php.sh +6 −0 Go to diff View file
A tools/utils/nix/php-base.nix +5 −0 Go to diff View file
A tools/utils/nix/pinned-nixpkgs.nix +5 −0 Go to diff View file
A tools/utils/scripts/generated-files-builder.sh +33 −0 Go to diff View file