Login New User
    •  
      request #14409 Memory limit exhausted while uploading git-lfs
    Infos
    #14409
    Guilhem Bonnefille (CS) (gbonnefille)
    2020-02-21 14:06
    2020-01-15 17:51
    15078
    Details
    Memory limit exhausted while uploading git-lfs
    While uploading files via git-lfs, the git command failed:
    - with generic error: "Failed to push some refs
    - without error in gitolite logs
    - with the following error in nginx

    2020/01/13 17:17:45 [error] 3677#0: *399559 FastCGI sent in stderr: "PHP message: PHP Fatal error: Allowed memory size of 205520896 bytes exhausted (tried to allocate 199237632 bytes) in /usr/share/tuleap/plugins/gitlfs/vendor/league/flysystem/src/Adapter/Local.php on line 159" while reading response header from upstream, client: 172.27.40.104, server: odin.si.c-s.fr, request: "PUT /uploads/git-lfs/objects/021be08a0874d4accf1a2e5b0be36f10ff10654549674b74e0228a5a36f0475e HTTP/1.1", upstream: "fastcgi://127.0.0.1:9002", host: "tuleap.example.fr"
    2020/01/13 17:17:45 [error] 3677#0: *399559 FastCGI sent in stderr: "PHP message: PHP Warning: Error while sending STMT_CLOSE packet. PID=3634 in Unknown on line 0" while reading upstream, client: 172.27.40.104, server: odin.si.c-s.fr, request: "PUT /uploads/git-lfs/objects/021be08a0874d4accf1a2e5b0be36f10ff10654549674b74e0228a5a36f0475e HTTP/1.1", upstream: "fastcgi://127.0.0.1:9002", host: "tuleap.example.fr"

    The concerned file is 211 MB.

    The concerned project is configured without any limit on file syze, in the Git.

    The Git-LFS limit is set to 512 MB on the platform.

    The PHP memory-limit is set to 256 MB.

    (Effective platform's version is 11.7)
    Empty
    SCM/Git
    All
    Empty
    Empty
    Stage
    Thomas Gerbet (tgerbet)
    Closed
    2020-02-21
    Attachments
    Empty
    References

    List of items referenced by or referencing this item.

    Artifact Tracker v5

    Follow-ups

    • User avatar
      Integrated into Tuleap 11.11.99.129

      • Status changed from Under review to Closed
      • Connected artifacts
      • Close date set to 2020-02-21
    • User avatar
      The issue is fixed when using the latest PHP packages from the remi repository.

      The minimal version of PHP required by Tuleap will be raised to 7.3.15: gerrit #17867.

      • Status changed from Verified to Under review
    • User avatar
      PHP 7.3.15 has been released with the fix. I will confirm tomorrow but with my last build from a few days ago of the 7.3 branch, the issue was resolved.
    • User avatar
      The issue is caused by a bug in PHP present since PHP 7.3.10/7.4.0 (or to be more precise since https://github.com/php/php-src/commit/05560b67bc87a2bcbfd5b48a48443a62f3311e7d). The bug has been fixed (https://bugs.php.net/bug.php?id=78902) but is not yet present in one of the released version.

      ATM I have no idea on how work around it while keeping the feature usable. While I do not recommend it, it is possible to increase the memory limit for those requests (see /etc/opt/remi/php73/php-fpm.d/tuleap-long-running-request.conf) so the file you are trying to upload can fit in memory.


      I will keep the request open until the fix is available in one of the PHP version supported by Tuleap.

      • Reported in version changed from 11.8 to All
    • User avatar
      @gbonnefille Would you be able to provide us the version of PHP you are currently running (can be obtained with /opt/remi/php73/root/usr/bin/php -v)?
    • User avatar
      Hum in fact this one is a bit strange. I have been able to reproduce on two distinct production instances but it seems to work just fine on a my dev instance...
    • User avatar
      I have been able to reproduce. There is indeed an issue here.

      • Status changed from Acknowledged to Verified
      • Platform cleared values: CentOS 6
    • User avatar
      Hi,

      Thanks for the report, I will try to reproduce.

      • Status changed from New to Acknowledged
      • Assigned to changed from None to Thomas Gerbet (tgerbet)