Overview

Provide a way to aggregate data across several trackers and several projects.

It will be done via a Widget to be installed on a Dashboard (personal or project). Unlike current "Tracker report" widget, it will be possible to directly interact with the widget to refine a query. For instance the widget displays by default all open artifacts, then directly in the widget, any user will be able to add "and category = test" to run it's own query.

This widget will rely on:

• Tracker Query Language for the definition of the search
• The selection of the columns to display will be done with a selector (as it's already done in tracker table reports)
• The order of the columns and their size will be manipulated in drag'n drop like tracker table report. However the underlying libs will be upgraded to fix current bugs. In order to be consistent, the changes will be done on tracker table report too.

On the technical side it will be a dedicated Angular app backed by REST routes. All the work on this widget report will be root for future redesign of tracker reports.

Always There Fields and semantics

Some fields are specials, we have 2 categories:

Always There Fields, they are "always there behind the scenes" fields but that might not be used in one tracker (or renamed):

• Submitted by
• Submitted on
• Last updated by
• Last updated on
• Artifact ID

Semantics:

• Title
• Status
• Assigned to
• Description

The proposal is to treat them like if they were normal fields and propose a unified syntax to query / display them. That is to say:

• Regardtheless the actual field pointed by "Status" semantic (progress, stage, etc), in cross tracker search, people will always query on "status". Eg: status = OPEN() and start_date >= NOW() - 1w
• Regardtheless the name or the label associated to "Submitted by" field (Creator, Starter, etc), in cross tracker search, people will always query on "submitted_on". Eg. submitted_on = '2017-04-24';

Corresponding identifiers:

• Submitted by: submitted_by
• Submitted on: submitted_on
• Last updated by: last_update_by
• Last updated on: last_update_on
• Artifact ID: id
• Title: title
• Status: status
• Assigned to: assigned_to
• Description: description

Limits and tracker consistency

With the given definition, we can end with configuration nightmare like having a field called (name) summary with "title" semantic and, in the same tracker a 'title' field that refers to something completly different. In this scenario, we cannot know if a search on title is for the semantic or the lone title field.

So, in order to make life easier, the Runtime Check will ensure, before running the query that:

• All the fields used in query term or in columns are present and accessible for read by the user
  • If I want to search on "submitted_on", the field must be active in all trackers I selected and I should have the right to read it
• There are no overlaping field names
  • If I want to search on "submitted_on", the only "submitted_on" field name accepted in involved trackers is the one the correspond to "Submitted on" field. If someone created a lone date field with name "submitted_on" in one of the tracker, the query will not be executed and the user will get an error to align its trackers structures.

Definition of boundaries

On the bright side, you can do anything with the trackers. The problem is that people actually do anything with them. And most of the time they aren't even aware about the configuration details and subtle differences (like different names or permissions). We need to define clear boundaries on what we can search on. Either people align their trackers or they won't be able to search across them.

As it's a tough problem due to the highly configurable nature of Tuleap trackers and the fine grain permissions, some preconditions are needed:

• Limit the number of trackers on which the query would be applied (for instance 10)
  • limit the amount of data we need to search on
  • allow to resonate on a finite state (for instance in term of number of fields)
  • allow to run consistency check ahead of the query execution
• Duck typing: if it has the same name and the same type, it's the same field
  • Given a set of tracker, if 2 different trackers have a "name" field but one is a String and the other one is a Select box, it will not be possible to search on this field or to display it as a column
• Permissions handling
  • Given an already defined query, when a random user want to look at the result, there is a check in first hand to ensure that the user has the right to search on all the fields of the query in all invoved tracker.
  • Same goes for the columns, if at least one field of one tracker cannot be displayed to the current user, nothing is displayed at all.
• Sorting
  • It will not be possible to sort on multiple columns in the same time (no multisort)
• Allowed fields
  • It's possible to search or display only on fields that are common in all trackers according to the Duck Typing definition
  • There are "fake fields" available as columns: "project" and "tracker".

Ways of working

A possible scenario of usage:

Configuration:

1. Select the trackers that will be searched on (regardtheless of the project)
2. Define the query (TQL)
3. Select the columns, their positions, their size and the sort (UI)

Runtime:

1. Ensure field consistency and permissions across trackers
   1. The current user is allowed to access all fields used in the search query and displayed in the report. Those fields belongs to a tracker the user can access. Those tracker belongs to a project the user can see.
   2. The used fields (search and display) are consistent across tracker (same names/types).
2. Perform the query and render the result

Refinement:

1. Based on this first execution, the user can modify the query for doing it's own searches

Spikes & mockups

Spikes

• We need to spike the most efficient way to build and execute the query. Either "Monstro query" approach one huge query that embed all the search terms or a union of smaller queries. Investigation on real data set is needed.
• Which libs shall replace the bogus one used to build the tracker report columns (drag'n drop and resize)

Mock-ups

• How do we select the trackers on which search will apply (across projects)
• How do we select columns, place them and rezise (+ alignment of Tracker report view)
• How do we interact with widget during the "Refinement of query" phase

Steps

Given the large amount of work that is needed, the following steps are identified to progressively come close the a full featured widget:

1. Create a new widget (for project and user dashboards) where it's possible to select the trackers on which we want to run queries. At this step, the only selection possible is the trackers, everything else is hardcoded:
   • the query will be 'status = "Open"'
   • the sort will be on artifact id
   • the displayed columns will be hardoded (like artifact links v2 reports)
   • only the first N elements will be displayed
2. Add runtime check for permissions and consistency
3. Allow to define the query with sematic and "always there" fields (artifact id, submitted on, etc)
4. Add pagination to "load more" artifacts
5. Allow to select columns and resize with semantic and "always there" fields
6. Allow to select on which column the sort is done
7. Introduce duck typing for columns
8. Introduce duck typing for query