epic #18001 Markdown

story #18336 add support of markdown

Artifact

Summary

As aEmpty

I want to *

add support of markdown

So thatEmpty

Acceptance criteria

CommonMark spec of Markdown is supported. "Markdown" is used as label (instead of CommonMark). The documentation will refer to CommonMark for those who want the nifty details.

User interface modifications

At the end of this story, Markdown is rendered in HTML in the following views:

Artifact Tooltips, when a Text field is chosen in the Tooltip Semantic
Cardwall and Cardwall renderer when a Text field is chosen in the Card fields Semantic
Modal v2 in Cardwall when I display comments (follow-ups)
Planning view and Kanban when a Text field is chosen in the Card fields Semantic. HTML tags are removed as a post-processing step, leaving only the text content for those card fields. The same treatment will apply to Markdown rendered in HTML.
Artifact Modal (in Kanban, etc.) to display comments (follow-ups)
In the test steps of a Test Execution in TestManagement
In the comment of a Test Execution in TestManagement. This is only in case someone modifies the comment "by hand" in Trackers. The comment is also truncated, the same truncation logic should be applied (rendered HTML is truncated)
In the notification e-mails (in text and HTML format) sent by Tracker plugin

I can choose the "Markdown" format and write Markdown formatted text in a plain Textarea field (no CKEditor) in the following views:

Creation of comments (follow-ups) on the Artifact view
Edition of comments on the Artifact view
Text fields on the Artifact view (creation & edition of artifacts)
Step definition fields on the Artifact view
Creation of comments in the Mass Change view
Artifact Modal (in Kanban, etc.) in Text fields
Creation of comments (follow-ups) in the Artifact Modal
Modal v2 in Cardwall for Text fields

There is a syntax help button "? help" in the following views:

in the Artifact view next to Markdown-enabled fields (follow-ups, Text fields, Step definitions).
In the Artifact Modal

The syntax help details mostly used formatting preferences.

No feature flag seems required as markdown is an improvement of text in 95% of the cases.

There is a "Design Check" to ensure that the possibilities unlocked by markdown renders well
Existing comments & Text fields that are in Text or HTML formats cannot be switch to markdown (yet)
Existing comments that are in markdown cannot be converted to HTML or Text formats (yet)

Definition of "broken"

The interface is considered "broken" when Markdown is displayed as-is (raw, source Markdown) in read-only views. It should always be converted to HTML when displayed. Raw Markdown is only shown in edition views, such as edition or creation of Text fields, creation or edition of Follow-up comments, etc.

REST API modifications

Why is the REST API is modified this way ?

Given the rule above, Tuleap's Javascript apps such as Planning View, Kanban and TestManagement (and likely other unknown consumers of the REST API) will be "broken" if we simply push a new "commonmark" format for Text fields and follow-ups. Their display logic relies on the (no longer true) fact that there were only two possible formats, so when the format is not "html", they would simply assume that it was "text". If we were to push a new "commonmark" format, those apps would naively treat it as "text" and would display the raw Markdown source. Given the rule above, their interface would be broken.

Also, we are not able to extract references from text on the client-side. Extraction of references (such as art #123) must be done on the server-side.

Finally, rendering Markdown on the server-side should guarantee that rendering is uniform (it avoids depending on N different npm libraries in various parts of Tuleap).

How it the API modified ?

REST API should prevent a breakage about unsupported format (first of all in Tuleap JS app) so:

Format is advertised as "html" and the output associated is Markdown rendered as HTML. This is for backward-compatibility reasons only.
a new property named "post_processed_body" is added. It contains the Markdown rendered as HTML and the references extracted as HTML links (this is the actual "post-processing" step). It is meant for display in HTML user interfaces. It is named this way to keep consistency with request #8527 where the same property was added to follow-up comments in order to process references.
a new property named "commonmark" is added and its value is the unprocessed Markdown source text. It allows clients to render a field or comment in "edition mode" to let people edit the Markdown source.

{
  "format": "html",
  "value" / "body": "<p>Text in <strong>markdown</strong> with ref art #123</p>",
  "post_processed_body": "<p>Text in <strong>markdown</strong> with ref <a href=\"https://tuleap/goto?key=art&amp;val=123&amp;group_id=101\" title=\"Tracker Artifact\" class=\"cross-reference\">art #123</a></p>",
  "commonmark": "Text in **markdown** with ref art #123"
}