•  
     
    story #7801 use git mirror while master is down
Summary
sysadmin
use git mirror while master is down
I can stop master server for maintainance without disturbing git users (read access)
Side effect: reduce master load related to gitolite membership computation
  • Add a REST route to fetch membership for a given list of users (potentially big: 1000 users for instance)
  • Add a caching mechanism for tuleap-gitolite-membership (redis)
    • for a user (john_doe), user groups are cached (eg: site_active, gpig_projectmembers, ug_103)
    • by default therer is a TTL for cache entry (configurable, default should be 10 mn)
  • Keep track of active users per mirror in redis too:
    • username
    • last access date
    • number of access
  • Add an "update-cache" option to tuleap-gitolite-membership
    • The will fetch the membership of all users identified as active users (previous step)
    • The entries are cached with configured TTL
  • The "update-cache" can be combined with a "without-expire" option
    • When used, entries are fetch as usual (for all active users)
    • But there is no expiration set
    • This is meant to be used when the master is planned to be down
  • Add a "disable-ttl" option to tuleap-gitolite-membership
    • This is the same purpose than "without-expire" but for emergencies (if master is unexpectedly down)
    • This invalidate the TTL and slave can continue to work until master is up again

Here is a schema of the proposed architecture (the elements in blue are the new ones)

Nouha Terzi (terzino), Denis PILAT (denis_pilat), francois jean-marie (francoisjean-marie), Loïc Lefort (lefort)
Status
Empty
Canceled
Development
  • [ ] Does it involves User Interface? 
  • [ ] Are there any mockups?
  • [ ] Are permissions checked?
  • [ ] Does it need Javascript development?
  • [ ] Does it need a forge upgrade bucket?
  • [ ] Does it need to execute things in system events?
  • [ ] Does it impact project creation (templates)?
  • [ ] Is it exploratory?
Empty
Details
#7801
Manuel Vacelet (vaceletm)
2018-07-17 11:38
2015-01-28 10:45
7803

References
Referencing story #7801
Referenced by story #7801

Follow-ups

User avatar
Loïc Lefort (lefort)2015-08-14 10:06
Hello

With cache user membership, we have a cache solution that is working but it does not contain all features described here.
Especially, cache is not updated automatically when mirror checks groups from master for a user, and there is no TTL for data in cache.

For information, without cache, time to retrive groups for a user, using tuleap-gitolite-membership, is around 0.5 second. This time
is not significant when getting a large git from mirror, but becomes dominant when getting a small git from mirror.
User avatar
Was implemented with cache user membership

  • Status changed from Ready (stalled) to Done
  • Permissions set to
User avatar
Looping our architect FJM. I think an internal discussion is necessary.
But what you wrote is a good summary of what have been discussed to me.
Nouha, Loic, any comment ? Some use cases are missing ?

  • CC list francois jean-marie (francoisjean-marie) added
User avatar
Denis, Nouha, Loïc,

Here is the write-up of the discussion we had by email, please review and comment if it's OK for you.

  • CC list set to Denis PILAT (denis_pilat), Nouha Terzi (terzino), Loïc Lefort (lefort)
User avatar
  • Acceptance criteria
    Something went wrong, the follow up content couldn't be loaded
    Only formatting have been changed, you should switch to markup to see the changes