•  
      request #9747 Using REST API to create Project
    Infos
    #9747
    Alaney Dória (alaney)
    2017-08-01 17:03
    2016-12-18 02:11
    9736
    Details
    Using REST API to create Project
    One interesting thing that can be very useful is the integration between the management team and development team. In normal organization, even thought we can have a development team, there is the commercial team that sales, the consultant or analyst. Most of this area uses ERPs to perform all their managements with CRM and sales.

    For my specific case we intent to integrated Odoo to Tuleap. To do that It would be necessary to allow the creation of new projects from Odoo. This is mainly to allow the creation of projects in Odoo that can have many different tasks and one of them can be development. Then from Odoo this project would be created in Tuleap with development task associated to the development team.


    Project admin
    All
    Empty
    • [x] enhancement
    • [x] internal improvement
    Empty
    Stage
    Empty
    Closed
    2017-08-01
    Attachments
    Empty
    References

    Follow-ups

    • User avatar
      • Status changed from Verified to Closed
      • Close date set to 2017-08-01
    • User avatar
    • User avatar
      gerrit #8960 integrated into Tuleap 9.9.99.135
    • User avatar
      Ok Nice,

      Regarding the services creation. shouldn't this be done in projectCreator based on template Id supplied?

      Regarding project custom fields what is the name of the field to map the custom fields I couldn't find it? I will add the custom field to the function signature.
    • User avatar
      Thomas Gerbet (tgerbet)2017-07-11 10:07
      A contribution to respect sys_use_project_registration parameter is under review: gerrit #8960.
    • User avatar
      Thank you for all the support and review.

      For the new things point me where should I look and I start to work on it.

      Thank you once again
    • User avatar
      gerrit #8058 integrated into Tuleap 9.9.99.121

      Thank you!

      Now that this first step is implemented there are missing points:

      * The route should respect sys_use_project_registration in local.inc (if disabled then the route should return 403).
      * The project created should use the same services as the template (currently no service are inherited).
      * We should be able to specify categorisation of the project when this is mandatory
      * We should be able to specify custom project fields when this is mandatory
    • User avatar

      I'm noticing that something is not right. in field mapping. based on project creator class we have the create which we have the shortName and publicName but shortName is actually short Description, while in documentation says that is the unix name and the publicName is actually mapping to descriptive group name.

      So my message was:

      {"short_name":"testes9747",
      "public_name":"Testes 9747",
      "is_public": true,
      "templateId":100
      }

      The reply is:

      ......

        "additional_informations": [],
        "id": 115,
        "uri": "projects/115",
        "label": "Testes 9747",
        "shortname": "testes9747"
      }

      but in user interface I have

      Is this correct?

      I'm going to put in method more user friendly names for the methods instead of short_name e bla bla.

    • User avatar
      You get a 403 Forbidden because… you explicitly asked to :)

      When you encounter an exception in your post() method, you are throwing a 403 RestException, which means "Forbidden". When the data in the request is not valid, you should send instead a 400, which means "Bad request".

      In your test, your data is not valid, because you are asking to create a project with a shortname containing an underscore, which is not allowed.

      If you fix those two issues, then your test should pass.

      I've made other comments in the review.
    • User avatar
      Done,

      I publish it. Can someone look in to ProjectTest?
    • User avatar
      You should push your code in the corresponding gerrit review so that we can figure out what is the issue.
    • User avatar
      Well for now it's solved.

      Now during testing I'm having a problem :

      1) ProjectTest::testPOST
      Guzzle\Http\Exception\ClientErrorResponseException: Client error response
      [status code] 403
      [reason phrase] Forbidden

      This is my message
      {"shortName":"test_9747","publicName":"Project Test Request 9747","visibility":"public","templateId":100}

      Maybe it's related to authentication?

      Whe I call the getResponse from what I notice it call the services with token authentication.
    • User avatar
      I fully agree with you @nterray about the contribution that must be integrated faster. But this is a process issue and it's something to work with the other maintainers. I suggested the Builder object as a "short term" solution so that the developer will be no moar blocked for its next step.
    • User avatar
      Ok Thanks,

      Going to work on thisl.
    • User avatar
    • User avatar
      I agree Nicolas,

      What path should I follow? Maybe I have to create the DashboardDuplicator? It's complicated :).

      Can I have a example of the dashboard duplicator instantiation?

      I have to do this method because after that I will create a connector from Odoo Project Management to Tuleap.
    • User avatar
      The issue here is not related to the relative complexity of ProjectCreator instantiation. The issue is that we failed to integrate the contribution as soon as possible: the sooner a contribution is integrated, the less the contributor has to worry about internal API change. Introducing a builder will only hide the root cause. What if we change a public method signature? What if we change the returning type of a method? What if do a big refactoring? As Tuleap is evolving rapidly, this situation occurs quite often ¯\_(ツ)_/¯.

      As soon as a contribution is integrated, the introduction of a parameter in the internal API does not concern anymore the original contributor, but the maintainers of the application.

      Therefore in order to decrease the frustration encountered by @alaney, we should speed up the integration of the contributions.
    • User avatar
      You have a good example of such an object with Tracker_Artifact_XMLImportBuilder
    • User avatar
      Huumm,

      Well I think your idea is good. Because it would simplify the creation for the Rest API. Since my experience still not so big and I'm only messing with Rest Api so I don't have a broad knowledge of all structure.

      I'm trying to have this finish so it can be release in one of the versions.
    • User avatar
      It will depend on the other development done by the other Tuleap contributors.

      We worked last month on new dashboards and widgets, and with the dependency injection (SOLID patterns), you have to instantiate a new object in your current work.

      To ease the creation of this object, you (or we I don't know at this point) can create a ProjectCreatorBuilder object that encapsulate the creation creation of a ProjectCreator object and provide to the developer a unique way to get a ProjectCreator instance.
    • User avatar
      Hi,

      I noticed that now my method for creating the a project from REST API it's not working. So what I found out is that now, ProjectCreator needs as 6th parameter the ProjectDashboardDuplicator. Now the problem is to have this object I have to create a bunch of other object just to have a ProjectDashboardDuplicator. This is a problem.

      If we are designing a REST API we have to think that CRUD operation has to be available. Unless we define an easy way to instantiate the creation of project, in this way it will become technically complicated and impossible to do so. We have to assume that the project will be created from other application that doesn't have anything to do with Tuleap. From someone that it's outside and want to contribute what I have done now is completely discarded because of this dependency.

      My question now is, will this happen very often?
    • User avatar

      Switched the discussion to gerrit #8058 because this it's easier to point the lines

    • User avatar
      Where is the @access in my method? I don't have any.

      If you are testing from the explorer you will see that all the methods from REST API are basically accessible without authentication. But during the test process it gives forbidden.

      I think I'm going to look into AuthenticatedResource and BasicAuthentication. If anyone have any info on this I would be glad.
    • User avatar

      Well I only tested from API explorer. The REST web service allow anonymous access

      Some endpoints are available w/o authentication when the corresponding resource is public.

      But for private resources, authentication & authorization applies.

      I don't know where I can find an example to enforce authentication

      You should remove @access hybrid and make the method protected

    • User avatar
      Well I only tested from API explorer. The REST web service allow anonymous access. You can check my code I call $this->checkAccess(); but I don't know where I can find an example to enforce authentication.

      Maybe that is the reason that is given forbidden in ProjectTest. I need help there. I already pushed the code with expection development
    • User avatar
      Thomas Gerbet (tgerbet)2017-06-12 12:20
      Does the code available for review on Gerrit give you such error?

      For the authentication, I think it was a comment of @vaceletm on the review. I did not check your code but it should not be possible to create a project without being authenticated, it is not the normal of Tuleap. Which documents are you referring to? I did check the documentation and I did not found mention of project creation without being logged in.
    • User avatar
      I already did the exception.

      The ProjectTest I have

      1) ProjectTest::testPOST
      Guzzle\Http\Exception\ClientErrorResponseException: Client error response
      [status code] 403
      [reason phrase] Forbidden
      [url] http://localhost/api/v1/projects

      Any hint about this.

      And regarding authentication, someone said that it was allowing access without authentication but if you read the documents Its normal behaviour from Tuleap.
    • User avatar
      Thomas Gerbet (tgerbet)2017-06-05 14:39

      Hi,

      I took a quick look at the ProjectCreator code and to achieve what you want to do you can probably catch the exceptions Project_InvalidShortName_Exception, Project_InvalidShortName_Exception and Project_Creation_Exception.

      That would give you something like:

      try { $project = $this->project_creator->create($short_name, $public_name,$data); } catch (Project_InvalidShortName_Exception $ex) { thrown new RestException(XXX, $ex->getMessage()); } ...
    • User avatar
      Ok Thanks.

      I want to return a message is there any example I can follow or any advice into how should it be done?

      The Idea is to return to the caller a message so it can show to the user.
    • User avatar
      Thomas Gerbet (tgerbet)2017-06-01 08:42
      Hello,

      Catching the exceptions Project_InvalidShortName_Exception and Project_InvalidFullName_Exception when you call ProjectCreator::create should do the trick, no?
    • User avatar
      Would be there a way of report better error from project creation like,shortName already exist, or should I do it manually?
    • User avatar
      Thomas Gerbet (tgerbet)2017-05-23 08:58
      Hi,

      I took a quick look at your contribution on gerrit #8058.

      The Jenkins failure on the gerrit_distlp-integration-tests should be gone with a rebase. The two others jobs failures are due to the REST tests. The easiest way to launch them locally is probably to use the Docker images, take a look at the documentation: https://tuleap-documentation.readthedocs.io/en/latest/developer-guide/tests.html#rest-tests
    • User avatar
      Hi,

      what is the best way to run the tuleap tests. right now I'm not able to integrated because jenkins fails on test, so I would like to be able to test the ProjectTest.php

      Best regards.
    • User avatar
      Trying to push my commit to draft gives me this

      Counting objects: 11, done.
      Delta compression using up to 4 threads.
      Compressing objects: 100% (11/11), done.
      Writing objects: 100% (11/11), 2.06 KiB | 0 bytes/s, done.
      Total 11 (delta 9), reused 0 (delta 0)
      remote: Resolving deltas: 100% (9/9)
      remote: Counting objects: 44823, done
      remote: Processing changes: refs: 1, done
      remote:
      remote: ERROR: In commit 482b1d558c3e876438385a78f9d4b9472ef7debe
      remote: ERROR: committer email address root@localhost.localdomain
      remote: ERROR: does not match your user account.
      remote: ERROR:
      remote: ERROR: The following addresses are currently registered:
      remote: ERROR: alaney.doria@alien-group.com
      remote: ERROR:
      remote: ERROR: To register an email address, please visit:
      remote: ERROR: https://gerrit.tuleap.net/#/settings/contact
      remote:
      remote:
      To https://gerrit.tuleap.net/a/tuleap
    • User avatar
      Hi,

      I'm able to test and create project using the rest api. but all the project created in the edit project show me the following error. Also when we call the creation service it doesn't return. Do you have any Idea of what it can be?

      ( ! ) Notice: Undefined index: group_name in /usr/share/tuleap/src/common/project/Group.class.php on line 169
      Call Stack
      # Time Memory Function Location
      1 0.0000 647056 {main}( ) ../groupedit.php:0
      2 0.0729 6427080 Tuleap\Project\Admin\ProjectDetailsPresenter->__construct( ) ../groupedit.php:110
      3 0.0744 6438440 Group->getPublicName( )
    • User avatar

      Great !

      To see the method in API explorer, you probably need to clear some caches, on the development container:

      • In /etc/tuleap/conf/local.inc, add "$DEBUG_MODE = 1;"
      • Then run /usr/bin/tuleap -c

      And you should see the new route

      PS: you can also join us on chat.tuleap.org, there is a chan dedicated to development questions (#dev)

    • User avatar
      last edited by: Alaney Dória (alaney) 2017-03-28 10:58

      I'm back :).

      Hi, I already reorganized my code. Can you tell me what I have to todo to see my net method in api explorer?

      Best regards.

    • User avatar
      Ok thanks.

      I'm reorganizing my source so I can make push to draft. I'm a bit busy with some project but I believe till the end of this week I can make my commit.

      Regards.
    • User avatar

      The username & passwords are the same when you are using the web interface.

      However if you want to push using http then you will need to generate a dedicated password for that in gerrit ("Your name" > Settings > HTTP Password)

    • User avatar
      I still with problem to commit to gerrit, the password is the same as tuleap.net?
    • User avatar
      When I open the project on Tuleap it shows that message on top. Also if you are using the api explorer it doesn't return the Id.
    • User avatar

      Where/when are you having this issue ?

    • User avatar
      Ok sorry.

      We are able to create project but we have the following issue
      ( ! ) Notice: Undefined index: group_name in /usr/share/tuleap/src/common/project/Group.class.php on line 169
      Call Stack
      # Time Memory Function Location
      1 0.0001 632040 {main}( ) ../groupedit.php:0
      2 0.1278 6320552 Tuleap\Project\Admin\ProjectDetailsPresenter->__construct( ) ../groupedit.php:109
      3 0.1290 6331584 Group->getPublicName( )
    • User avatar

      Please avoid sharing code throug tracker, it's unreadable. If you want to exchange on code either push on gerrit or on a personal fork or anywhere but not in trackers.

    • User avatar
      I've this, but the request is giving me 500 internal server error. when I look to the log nothing is there. I'm going to continue digging

      public function __construct() {
      $this->user_manager = UserManager::instance();
      $this->project_manager = ProjectManager::instance();
      $this->reference_manager = ReferenceManager::instance();
      $this->ugroup_manager = new UGroupManager();
      $this->json_decoder = new JsonDecoder();
      $ugroup_user_dao = new UGroupUserDao();

      $this->ugroup_duplicator = new UgroupDuplicator(new UGroupDao(),
      $this->ugroup_manager,
      new UGroupBinding($ugroup_user_dao, $this->ugroup_manager),
      $ugroup_user_dao, EventManager::instance());
      $send_notifications = false;
      $force_activation = true;

      $this->project_creator = new ProjectCreator(
      $this->project_manager,
      $this->reference_manager,
      $this->ugroup_duplicator,
      $send_notifications,
      new FRSPermissionCreator(new FRSPermissionDao(), new UGroupDao()),
      $force_activation);
      parent::__construct();
      }

      /**
      * Creates a new Project
      *
      * Creates a new project
      *
      * @access hybrid
      * @url POST
      * @status 201
      *
      * @param string $unixName {@from body}
      * @param string $shortName Description
      * @param string $publicName
      * @param boolean $visibility
      * @param int $templateId Template for this project.
      * @return int
      */
      public function create($unixName, $shortName, $publicName,$visibility, $templateId)
      {
      $this->checkAccess();
      /*
      * Site admin password (admin): 3mkpn3HGvSVRBQ0
      * root: nOaJHulPFR7WyAP
      * {
      "unixName": "gpig",
      "shortName": "Guinea Pig",
      "publicName": "A test project",
      "visibility": "public",
      "templateId": "100"
      }
      */
      $data = array(
      "project" => array("form_unix_name" => $unixName),
      "project" => array("form_full_name" => $publicName),
      "project" => array("form_short_description" => $shortName),
      "project" => array("built_from_template" => $templateId),
      "project" => array("is_public" => $visibility));

      $project = $this->project_creator->create($shortName, $publicName,$data);

      return $project->getID();
      }
    • User avatar

      All the logs should be in /var/log/httpd/error_log

    • User avatar
      How can I debug?. I'm using Netbeans and the docker tuleap image. I'm having now 500 internal server error.
    • User avatar

      You can have a look at src/utils/import_project_xml.php to check how the 2 classes are instanciated

    • User avatar
      Ok, later on after work I will push. by the way, regarding

      UgroupDuplicator;
      FRSPermissionCreator;

      any advice about where should I follow to create the proper instances.

      Regards.
    • User avatar

      You are missing a "use ReferenceManager" in the use list.

      Could you try again to push as draft, I added you as contributor to gerrit

      Remember:

      • You should only have 1 commit to push (rebase + squash if needed) 
      • The push command is: git push gerrit HEAD:refs/drafts/master
    • User avatar

      My push is being rejected. Probably I'm doing or did something wrong. I'm putting here part of what I have done.


      namespace Tuleap\Project\REST\v1;

      use Tuleap\Project\PaginatedProjects;
      use Tuleap\Project\REST\ProjectRepresentation;
      use Tuleap\Project\REST\UserGroupRepresentation;
      use Tuleap\REST\v1\GitRepositoryRepresentationBase;
      use Tuleap\REST\v1\PhpWikiPageRepresentation;
      use Tuleap\REST\v1\OrderRepresentationBase;
      use Tuleap\REST\v1\MilestoneRepresentationBase;
      use Tuleap\REST\ProjectAuthorization;
      use Tuleap\REST\Header;
      use Tuleap\REST\JsonDecoder;
      use Tuleap\REST\ResourcesInjector;
      use Tuleap\REST\AuthenticatedResource;
      use Tuleap\Project\UgroupDuplicator;
      use Tuleap\FRS\FRSPermissionCreator;
      use ProjectManager;
      use ProjectCreator;
      use UserManager;
      use PFUser;
      use Project;
      use EventManager;
      use Event;
      use ProjectUGroup;
      use UGroupManager;
      use URLVerification;
      use Luracast\Restler\RestException;
      use PaginatedWikiPagesFactory;
      use WikiDao;
      use Wiki;


      /**
       * Wrapper for project related REST methods
       */

      class ProjectResource extends AuthenticatedResource {

          const MAX_LIMIT = 50;

          /** @var UserManager */
          private $user_manager;

          /** @var ProjectManager */
          private $project_manager;

          /** @var UGroupManager */
          private $ugroup_manager;
          
          /** @var ProjectCreator*/
          private $project_creator;
          
          /**
           * @var UgroupDuplicator
           */
          private $ugroup_duplicator;

          public function __construct() {
              $this->user_manager    = UserManager::instance();
              $this->project_manager = ProjectManager::instance();
              $this->reference_manager = ReferenceManager::instance();
              $this->ugroup_manager  = new UGroupManager();
              $this->json_decoder    = new JsonDecoder();
              
              $send_notifications = false;
              $force_activation   = true;
              
              $this->project_creator = new ProjectCreator(
                      $this->project_manager,
                      $this->reference_manager,
                      null,
                      $send_notifications,
                      null,
                      $force_activation);
              parent::__construct();
          }

          /**
           * Creates a new Project
           *
           * Creates a new project
           *
           * @access hybrid
           * @url POST
           * @status 201
           *
           * @param string $unixName {@from body}
           * @param string $shortName Description
           * @param string $publicName
           * @param boolean $visibility
           * @param int $templateId Template for this project.
           * @return @return Tuleap\Project\REST\ProjectRepresentation
           */
          public function create($unixName, $shortName, $publicName,$visibility, $templateId)
          {
              $this->checkAccess();
              /*
               * Site admin password (admin): 3mkpn3HGvSVRBQ0
               * root: nOaJHulPFR7WyAP
               * {
          "unixName": "gpig",
          "shortName": "Guinea Pig",
          "publicName": "A test project",
          "visibility": "public",
          "templateId": "100"
      }
               */
              $data = array(
                  "project" => array("form_unix_name" => $unixName),
                  "project" => array("form_full_name" => $publicName),
                  "project" => array("form_short_description" => $shortName),
                  "project" => array("built_from_template" => $templateId),
                  "project" => array("is_public" => $visibility));

              $project = $this->project_creator->create($shortName, $publicName,$data);
              
              return $project;
          }
         

    • User avatar

      Fatal error: Class 'Tuleap\Reference\ReferenceManager' not found in /usr/share/tuleap/src/common/project/REST/v1/ProjectResource.class.php on line 85

      Could you share your code (for instance as a draft on gerrit) because on Tuleap master, on line 85 of ProjectResource it's just comments.

      Based on your direction I'm using the ProjectXMLImport to make the project creator, I'm able to call the service but I'm having the above error. I have added require_once from common/reference/ReferenceManager.class.php to the project but it keeps telling that it cant find the class. I'm already able to call the service using the api explorer.

      I don't think using ProjectXMLImport is needed, you should have a look at src/common/project/ProjectCreator.class.php instead.

      Also could you give me some explanation if the UgroupDuplicator and FRSPermissionCreator are required for the ProjectCreator construction, can just pass null there?

      No, you must pass concrete instances of those 2 classes.

       

    • User avatar
      last edited by: Alaney Dória (alaney) 2017-01-30 06:25
      Hi,

      I took this weekend and started to do this methdo but I'm having a problem to get the ReferenceManager

      Fatal error: Class 'Tuleap\Reference\ReferenceManager' not found in /usr/share/tuleap/src/common/project/REST/v1/ProjectResource.class.php on line 85

      Based on your direction I'm using the ProjectXMLImport to make the project creator, I'm able to call the service but I'm having the above error. I have added require_once from common/reference/ReferenceManager.class.php to the project but it keeps telling that it cant find the class. I'm already able to call the service using the api explorer.

      Also could you give me some explanation if the UgroupDuplicator and FRSPermissionCreator are required for the ProjectCreator construction, can just pass null there?
    • User avatar

      The description of the enhancement was moved to a user story: story #9890

    • User avatar

      You can reach them with sales@enalean.com

    • User avatar

      Ok. How I submit my quest them?
       

    • User avatar

      Hi,

      It's probably a matter of days to have it done properly. 

      It's not on our high priority list yet but if you are willing to invest, you can finance it through OpenRoadmap

    • User avatar
      Hi,

      I was analysing the code, Wouldn't be faster if some of you do this actually? Because I still have to do the Odoo connector and update this https://github.com/djurodrljaca/tuleap-rest-api-client. how long would it take to have this method?

      Best regards.
    • User avatar
      Ok. got it. Going to stud it and develop it. If I have any question I return to you. You can assign this to me.
    • User avatar

      Maybe you didn't clone the right repository, for instance src/common/project/REST/v1/ProjectResource.class.php is there:

      https://tuleap.net/plugins/git/tuleap/tuleap/stable?p=tuleap%2Fstable.git&a=tree&hb=e2f0bae4e6bcea09f05f117d686e1ab8c9f24a45&h=de3bf85c2e09f70019cecc2008190c1d4838c59c&f=src/common/project/REST/v1

    • User avatar
      Hi Manuel,

      Those folder and files that you mentioned I couldn't find in source repository. I did the clone from git. It's anything i'm missing?
    • User avatar

      You should have a look at developer quick start

    • User avatar
      Thanks, from which repository I should clone to start?
    • User avatar

      So let's dig a little bit more in what should be done. We need to created a new route with a payload

      POST /projects

      We need to have there:

      • project unix name
      • project real name
      • short desctiption
      • visibility (public, private, public inc. restricted)
      • project custom fields
      • project categories (trove cat) // can be ignored in a first implementation but mandatory
      • the template we inherit from

      So the payload should looks like

      {
          "shortname": string,
          "label": string,
          "description": string,
          "visibility": string ('public', 'private', 'unrestricted'),
          "fields": [
              {
                  "name": "value"
              }, ...
          ],
          "template": int, (the project id to inherit from)
      }

      Example:

      {
          "shortname": "gpig",
          "label": "Guinea Pig",
          "description": "A test project",
          "visibility": "public",
          "fields": [
              {
                  "Full Project Description": "A longer project description"
              }
          ],
          "template": 100,
      }

      If you want to have a look, you can see what we have done for other routes.

      Then you need to create a new method "post" in src/common/project/REST/v1/ProjectResource.class.php that will basically call ProjectCreator (src/common/project/ProjectCreator.class.php). You can have a look at src/common/project/ProjectXMLImporter.class.php for an example on how to use it.

      Of course, you will need to add some tests in  tests/rest/ProjectTest.php as well.

    • User avatar
      Were to follow, maybe I can do it, and them move to create my integration?
    • User avatar

      That shouldn't be really complex to do as the SOAP API already offer this feature.


      • Status changed from New to Verified