Currently Tuleap CI pipelines do not take advantages of the various cache possibilities we have. Using them would help us shave us some time from the test pipeline and be a little less dependent on external infrastructure.

Possible candidates that could benefit from a shared cache between CI agents:

• pnpm
• Composer
• Nix store (at least for the "build tools" env)
• Turborepo (we should probably bump to the latest version first but it looks like we have some troubleshooting to do first) for the build and JS unit tests phases

In our context (full control on the infrastructure, possibly large content to cache, "split monorepo") the usual object storage based cache is probably not the right approach:

• determining the cache keys will not be easy/not possible
• pushing and fetching the content to the object storage to save and restore it will be slow

Our best bet is likely to mount a block storage to our CI agents and to use that.

Only the pipeline tuleap-gerrit-tests (and equivalent for plugins maintained out of tree) should use the cache. Anything happening post merge should not rely on it so we can catch possible cache issues early and to avoid the possibility for a non merged contribution to manipulate the cache (especially anything building the final release artifacts must not use it).

Implementation proposal:

• Mount a /build_cache volume on CI agent machines
• This volume is an Amazon EFS resource so we can mount it on multiple agents simultaneously
• The "cache volume" must only be accessible from the VPC used by the CI agents
• We will need a "stable" name to configure the mount point in the image we build with Packer, the easiest to achieve that is probably to use a private Route53 zone
• We have a pipeline running at the end of each day that removes everything in our "cache volume":
  • It makes it easy to keep the "cache volume" to a reasonable size
  • It provides us a "break glass" solution if we end up in a situation where what we have in caches breaks our test pipelines