•  
      story #8206 Work with "values by field" for GET and POST on /artifacts/:id
    Summary
    Developer
    Work with "values by field" for GET and POST on /artifacts/:id

    GET /artifacts/:id

    The artifact representation sent in GET request will have an additional field in order to easily retrieve values:

        GET /artifacts/:id?…
        200 Ok
        <response>
            <id>42</id>
            ...
            <values>
                <item>
                    <field_id>12</field_id>
                    <type>float</type>
                    <name>story_points</name>
                    <label>Story Points</label>
                    <value>1.6</value>
                </item>
            </values>
            <values_by_field>
                <story_point>1.6</story_point>
            </values_by_field>
        </response>

    In order to not clutter to much the response, one can ask for a specific value format:

    GET /artifcats/:id?values_format=flat_array    # only flat array of values is returned (as of today)
    GET /artifcats/:id?values_format=by_field      # only values indexed by field are returned
    GET /artifcats/:id?values_format=all           # like in the example above

    If values_format is ommitted then we consider that values_format = flat_array

    POST /artifacts/:id

    The route will accept two formats of values to create an artifact.

        POST /artifacts/:id
        <artifact>
            <values>
                <item>
                    <field_id>12</field_id>
                    <value>3.14</value>
                </item>
            </values>
        </artifact>
        200 Ok
    

    Or

        POST /artifacts/:id
        <artifact>
            <values_by_field>
                <story_points>3.14</story_points>
            </values_by_field>
        </artifact>
        200 Ok

    If both format are submitted we will raise an Exception as we cannot handle both format at a time (400 Bad Request). If the submitted value does not correspond to the expected type then 400 Bad Request will be raised as well.

    The POST request will handle only alphanumeric fields (no need to work with complex fields like list field as of today).

    POST response (201 created) should return the parametrized url.

    1. GET can return both formats (based on url parameter)
    2. POST works only with one format
    3. Integration tests in XML (real usecase)
    Empty
    matej.m.lajcak@tieto.com, tomas.zapletal@tieto.com, Patricia Carrasco (pcar), Emilio Palmiero (empa), Stephan Bill (stephanbill)
    Status
    Empty
    Done
    Development
    • [ ] Does it involves User Interface? 
    • [ ] Are there any mockups?
    • [x] Are permissions checked?
    • [ ] Does it need Javascript development?
    • [ ] Does it need a forge upgrade bucket?
    • [ ] Does it need to execute things in system events?
    • [ ] Does it impact project creation (templates)?
    • [ ] Is it exploratory?
    Empty
    Details
    #8206
    Yannis ROSSETTO (rossettoy)
    2018-07-17 11:59
    2015-07-07 09:45
    8165

    References
    Referencing story #8206

    Git commit

    tuleap/tuleap/stable

    Merge commit 'refs/changes/94/4194/6' of ssh://gerrit.tuleap.net:29418/tuleap into stable 885f830f99
    story #8206 Work with "values by field" for GET on /artifacts/:id d4646b9313
    Merge commit 'refs/changes/15/4215/5' of ssh://gerrit.tuleap.net:29418/tuleap into stable 442a51bb95
    story #8206: "values by field" for GET on /artifacts/:id returns full REST value c04f49ae8c
    Merge commit 'refs/changes/04/4204/9' of ssh://gerrit.tuleap.net:29418/tuleap into HEAD fd5e4befc8
    story #8206 Work with "values by field" for POST on /artifacts/:id a5b80e7a3a
    Merge commit 'refs/changes/59/4259/2' of ssh://gerrit.tuleap.net:29418/tuleap into HEAD aa940635a7
    story #8206 Add parameter in URL 74d5fa6b4a
    Merge commit 'refs/changes/61/4261/3' of ssh://gerrit.tuleap.net:29418/tuleap into HEAD 8bea932e49
    story #8206 Add Location header 0a6ebda949
    Merge commit 'refs/changes/24/4324/2' of ssh://gerrit.tuleap.net:29418/tuleap into HEAD 0410a58f00
    story #8206: send consistent WWW-AUTHENTICATE header for 401 responses 71b78f7051

    Follow-ups

    User avatar
    Hi Tomas, would this change have any impact on Focal Point integration?

    • CC list Stephan Bill (stephanbill) added
    User avatar

    Tomas, Patricia,

    For your information, we need to make some evolutions there to be aligned with RFC (and reverse proxies):

    • POST /artifcat/:id will now return a 201 (instead of 200) as of today
    • The "Last-Modified" header will be sent following RFC1123 date (insted of ISO8601) as of today.

    Those changes should be transparent to you but feel free to comment to report any problem this might raise

    User avatar
    dylan bowden (dylan)2015-08-06 15:13
    last edited by: dylan bowden (dylan) 2015-08-06 16:08
    Hi, just so I make the right change,

    For all requests that return 401 (unauthenticated), you would like the " WWW-Authenticate" header to be

    Www-Authenticate: Basic realm="Restricted Tuleap API" Token Realm="Restricted Tuleap API""

    thanks
    User avatar
    Hello Tomas,

    Thanks for your detailed feedback. We will take a look on this with the team in order to fit your needs and the RFCs.
    We will come back to you when we will have more information to share with your or a new patch to test in your side.
    User avatar
    last edited by: Tomas Zapletal (xtomaza) 2015-07-30 16:24

    Hi,

    today I tested POST request from FocalPoint to Tuleap and I experienced problems with authentication.
    For every POST request I send from Focal Point to Tuleap I receive 401 Unauthorized response.
    After some investigation I found out that Tuleap doesn't fully follow Basic access authentication protocol in this case.

    The problem is that Focal Point firstly sends POST request without any credentials (according to specification and security reasons) to find out what
    authentication methods Tuleap supports.
    As response Tuleap should send http response with error code 401 Unauthorize (which it does correctly) and header field named "WWW-Authenticate" containing supported 
    authentication methods (Basic, OAuth, ...).

    Here is example received from Hansoft, please notice WWW-Authenticate field:

        Status Code: 401 Unauthorized
        Content-Language: en
        Content-Length: 951
        Content-Type: text/html;charset=utf-8
        Date: Thu, 30 Jul 2015 07:28:58 GMT
        Server: Apache-Coyote/1.1
        Set-Cookie: JSESSIONID=24931C6E3380B256E6CA33FEE7A57BB6; Path=/test/; HttpOnly
        WWW-Authenticate: OAuth realm="Hansoft"
                          Basic realm="Hansoft"
                          
    But we receive this response from Tuleap:

        Status Code: 401 Unauthorized
        Access-Control-Allow-Credentials: true
        Access-Control-Allow-Headers: Accept, Accept-Charset, Authorization, Content-Type, Origin, X-Auth-UserId, X-Auth-Token
        Access-Control-Allow-Origin: *
        Access-Control-Expose-Headers: X-PAGINATION-SIZE, X-PAGINATION-LIMIT-MAX, X-PAGINATION-LIMIT
        Cache-Control: no-cache, must-revalidate
        Connection: close
        Content-Language: en
        Content-Length: 551
        Content-Type: application/xml; charset=utf-8
        Date: Thu, 30 Jul 2015 07:29:45 GMT
        Expires: 0
        Server: Apache/2.2.15 (Red Hat)
        Vary: Accept
        WWW-Authenticate: Query name="api_key"
        X-Powered-By: Luracast Restler v3.0.0rc6
        
    We also noticed that if Tuleap receives POST request with bad credentials then it works fine and Tuleap returns following response:

        Status Code: 401 Unauthorized
        Access-Control-Allow-Credentials: true
        Access-Control-Allow-Headers: Accept, Accept-Charset, Authorization, Content-Type, Origin, X-Auth-UserId, X-Auth-Token
        Access-Control-Allow-Origin: *
        Access-Control-Expose-Headers: X-PAGINATION-SIZE, X-PAGINATION-LIMIT-MAX, X-PAGINATION-LIMIT
        Cache-Control: no-cache, must-revalidate
        Connection: close
        Content-Language: en
        Content-Length: 599
        Content-Type: application/xml; charset=utf-8
        Date: Thu, 30 Jul 2015 13:53:10 GMT
        Expires: 0
        Server: Apache/2.2.15 (Red Hat)
        Vary: Accept
        WWW-Authenticate: Basic realm="Restricted Tuleap API"
                          Query name="api_key"
        X-Powered-By: Luracast Restler v3.0.0rc6

    And here is body of response:

    <?xml version="1.0" encoding="UTF-8"?>

    <response>
        <error>
            <code>401</code>
            <message>Unauthorized</message>
        </error>
        <debug>
            <source>Restler.php:940 at authenticate stage</source>
            <stages>
                <success>
                    <text>get</text>
                    <text>route</text>
                    <text>negotiate</text>
                </success>
                <failure>
                    <text>authenticate</text>
                    <text>message</text>
                </failure>
            </stages>
        </debug>
    </response>

     

    You can check more information here:    

    10.4.2 401 Unauthorized
    The request requires user authentication. The response MUST include a WWW-Authenticate header field (section 14.47) containing a challenge applicable to the requested resource. The client MAY repeat the request with a suitable Authorization header field (section 14.8). If the request already included Authorization credentials, then the 401 response indicates that authorization has been refused for those credentials. If the 401 response contains the same challenge as the prior response, and the user agent has already attempted authentication at least once, then the user SHOULD be presented the entity that was given in the response, since that entity might include relevant diagnostic information. HTTP access authentication is explained in "HTTP Authentication: Basic and Digest Access Authentication" [43].

    Source: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.2


    Protocol
    Server side
    When the server wants the user agent to authenticate itself towards the server, it must respond appropriately to unauthenticated requests.
    Unauthenticated requests should return an HTTP 401 Not Authorized response code containing a WWW-Authenticate HTTP header.[5]
    The WWW-Authenticate header for basic authentication (used most often) is constructed as following:

    WWW-Authenticate: Basic realm="nmrs_m7VKmomQ2YM3:"

    Source: https://en.wikipedia.org/wiki/Basic_access_authentication

    User avatar
    Are you testing on Tuleap 8.4.99.22 ? Because on my side, when the POST returns 200, I've got:

    "Date": "Mon, 27 Jul 2015 10:08:40 GMT",
    "Server": "Apache/2.2.3 (CentOS)",
    "X-Powered-By": "Luracast Restler v3.0.0rc6",
    "Vary": "Accept,User-Agent",
    "Allow": "OPTIONS, POST",
    "Access-Control-Allow-Methods": "OPTIONS, POST",
    "Last-Modified": "2015-07-27T12:08:40+02:00",
    "Etag": "1437991720",
    "Location": "https://sonde.cro.enalean.com/api/v1/artifacts/8491?values_format=by_field",
    "Cache-Control": "no-cache, must-revalidate",
    "Expires": "0",
    "Content-Language": "en-US",
    "Access-Control-Allow-Origin": "*",
    "Access-Control-Allow-Credentials": "true",
    "Access-Control-Allow-Headers": "Accept, Accept-Charset, Authorization, Content-Type, Origin, X-Auth-UserId, X-Auth-Token",
    "Access-Control-Expose-Headers": "X-PAGINATION-SIZE, X-PAGINATION-LIMIT-MAX, X-PAGINATION-LIMIT",
    "Content-Length": "255",
    "Keep-Alive": "timeout=15, max=100",
    "Connection": "Keep-Alive",
    "Content-Type": "application/xml; charset=utf-8"
    }
    User avatar

    Hi,

    I tested latest build today, please look at http response of POST request, there is no "Location" field in header.

        Status Code: 200 OK
        Access-Control-Allow-Credentials: true
        Access-Control-Allow-Headers: Accept, Accept-Charset, Authorization, Content-Type, Origin, X-Auth-UserId, X-Auth-Token
        Access-Control-Allow-Methods: OPTIONS, POST
        Access-Control-Allow-Origin: *
        Access-Control-Expose-Headers: X-PAGINATION-SIZE, X-PAGINATION-LIMIT-MAX, X-PAGINATION-LIMIT
        Allow: OPTIONS, POST
        Cache-Control: no-cache, must-revalidate
        Connection: close
        Content-Language: en
        Content-Length: 249
        Content-Type: application/xml; charset=utf-8
        Date: Mon, 27 Jul 2015 09:57:57 GMT
        Etag: 1437991077
        Expires: 0
        Last-Modified: 2015-07-27T11:57:57+02:00
        Server: Apache/2.2.15 (Red Hat)
        Set-Cookie: TULEAP_session_hash=1fe7f0caa159dea7b0fe88c5179e3b8c; path=/; domain=.openalm-dev.lmera.ericsson.se; httponly
        TULEAP_user_token=5081faba121322def20c2b349b56b3e3; expires=Tue, 28-Jul-2015 09:57:57 GMT; path=/; domain=.openalm-dev.lmera.ericsson.se
        TULEAP_user_id=13156; path=/; domain=.openalm-dev.lmera.ericsson.se
        Vary: Accept
        X-Powered-By: Luracast Restler v3.0.0rc6
    User avatar
    Thanks for sharing this Thomas. We will add this location header for GET and POST artifacts/:id
    User avatar
    last edited by: Tomas Zapletal (xtomaza) 2015-07-24 09:52
    Here is example of http response header returned to Focal Point from another tool called Hansoft, please see field called Location.



    Status Code: 201 Created
    Content-Type: application/rdf+xml
    Date: Fri, 24 Jul 2015 07:47:04 GMT
    Etag: 1437724024198
    Location: http://ESEKIWDP2526.RND.ERICSSON.SE:8443/test/services/tasks/3395
    Server: Apache-Coyote/1.1
    Transfer-Encoding: chunked

    URL "http://ESEKIWDP2526.RND.ERICSSON.SE:8443/test/services/tasks/3395" can be used for updates (PUT) or fetch (GET) of task with id 3395.
    User avatar
    Maybe I was wrong here, I ment complete URL to artifact. URL which can be used for updates of artifact.
    Usually it is returned in responses (with an HTTP status code of 201 or 202) from an HTTP server in the HTTP Location header field to provide information about the location of a newly created resource.
    User avatar
    Hi Yannis,

    please add there complete URI of artifact, because Focal Point uses the URI provided in POST response for updates (PUT) of artifact.
    User avatar
    Hello Patricia,

    The patch was just merged. It will be available in Tuleap 8.4.99.20 in the next minutes (when the packages will be built).
    User avatar
    Hi Yannis, please let us know when a build is available so that we can try it out. Many thanks
    User avatar
    last edited by: Yannis ROSSETTO (rossettoy) 2015-07-23 16:52

    You are right, it was missing.

    So when you create an artifact by field with post, it will add ?values_format=by_field in the URI.

    Examples

    1. With values_by_field -> artifacts/8490?values_format=by_field
    2. With values -> artifacts/8490

     

    A patch is under review to add this parameter in the returned URI: gerrit #4259

    User avatar
    last edited by: Tomas Zapletal (xtomaza) 2015-07-23 13:09
    Hi,

    I tested new POST method today. It looks good. I have just comment, seems that POST response (201 created) doesn't return the parametrized url in response header.

    Could you please check it ?
    User avatar

    Hi,

    I did some testing today. Parametrized GET method looks very good, but I experience some problems with POST method.

    If I send minimal xml version of artifact (only mandatory field "Slogan") in new "by_field" format :

    <artifact>
        <tracker>
            <id>9725</id>
            <uri>trackers/9725</uri>
        </tracker>
        <values_by_field>
            <slogan>
                <value>new slogan</value>
            </slogan>
        </values_by_field>
    </artifact>

    to creation factory URL - https://openalm-dev.lmera.ericsson.se/api/v1/artifacts/ I got following error :

    <?xml version="1.0" encoding="UTF-8"?>
    <response>
        <error>
            <code>400</code>
            <message>Bad Request: `values` is required.</message>
        </error>
        <debug>
            <source>Validator.php:391 at validate stage</source>
            <stages>
                <success>
                    <text>get</text>
                    <text>route</text>
                    <text>negotiate</text>
                    <text>authenticate</text>
                </success>
                <failure>
                    <text>validate</text>
                    <text>message</text>
                </failure>
            </stages>
        </debug>
    </response>

    I also noticed that if I do the same in old "flat_array" format then it works fine:

    <artifact>
        <tracker>
            <id>9725</id>
            <uri>trackers/9725</uri>
        </tracker>
        <values>
            <item>
                <field_id>318278</field_id>
                <type>string</type>
                <label>Slogan</label>
                <value>new slogan/value>
            </item>
        </values>
    </artifact>

    • Status changed from Done to On going
    User avatar
    Joris MASSON (jmasson)2015-07-13 16:27
    last edited by: Joris MASSON (jmasson) 2015-07-13 16:29

    We made some changes to the GET route in order to be more consistent with the existing value format.

    Now with GET /artifacts/:id, you will have something like this:

    GET /artifacts/:id?…
        200 Ok
        <response>
            <id>42</id>
            ...
            <values>
                <item>
                    <field_id>12</field_id>
                    <type>float</type>
                    <name>story_points</name>
                    <label>Story Points</label>
                    <value>1.6</value>
                </item>
            </values>
            <values_by_field>
                <story_point>
                    <field_id>12</field_id>
                    <type>float</type>
                    <name>story_point</name>
                    <label>Story Point</label>
                    <value>1.6</value>
                </story_point>
            </values_by_field>
        </response>

    The GET route now returns only alphanumeric fields (no more list and date fields) to be consistent with what the POST route can handle.

    For the POST route, you will have to provide something like this:

    POST /artifacts/:id
        <artifact>
            <values_by_field>
                <story_points>
                    <value>3.14</value>
                </story_points>
            </values_by_field>
        </artifact>
        200 Ok

    if you submit to the POST route the additional information you retrieved with GET, like "field_id", "label", "name" or "type", the POST route will ignore them and only take into account "value".

    These two patches are currently under review, we will notify you when they are merged so you can test them.

    User avatar

    For your information a first step is available in Tuleap 8.3.99.28. You can now parameterize the GET route in order to retrieve artifact values indexed by field. Example:

    GET /artifacts/4096?values_format=by_field
    {
      "id": 4096,
      "uri": "artifacts/4096",
      ...,
      "values": null,
      "values_by_field": {
        "story_points_11": 3.14,
        "i_want_to": "see the Start and End Date of a Release in all views",
        ...
      }
    }
    

    Please note that only a subset of field types are supported:

    • Field int/float
    • Field string/text
    • Field lists (selectbox, multi selectbox, radio button, checkbox)
    • Field open lists
    • Field date (with and without time)

    Could you please check on your side that this is the intended behavior?

    We are currently working on the POST operation.

    User avatar
    • CC list set to matej.m.lajcak@tieto.com, tomas.zapletal@tieto.com, Emilio Palmiero (empa), Patricia Carrasco (pcar)
    User avatar
    • So that
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes