Given I have pushed commits with "closing references" such as Closes art #000 on a pull request branch, when I click on "merge" on the web interface then the artifacts I referenced in the commit messages are not closed (or sometimes they are but in specific conditions).

The root cause is that the code that closes these artifacts finds the username doing the push and assumes it is a valid Tuleap user. In the case of a pull request merge, it turns out that the push is made as a technical user (gitolite) that has no matching user account on Tuleap-side. Tuleap therefore builds an "anonymous" user account for it. This user is then the basis for all other checks, such as verifying access to the Tuleap project, verifying access to the referenced artifact, etc.

Thus, on platforms allowing anonymous users, on public projects, when the artifact is accessible to anonymous, it works and the resulting comment references a username @ with nothing else after. In all other cases, it will do nothing (the artifacts won't be closed).