•  
      epic #40128 Introduce Structure into Artidoc
    Summary
    Introduce Structure into Artidoc
    Artidoc

    Artidoc Structure covers the following elements:

    • having titles to organize sections (1. Introduction > 1.1 Context, etc)

    • having zones of "free" text. Free text is a text that is not part of the tracker associated to an Artidoc. For instance, given an Artidoc associated to a Requirement tracker, a new free text section will not generate a new Requirement artifact.

    Persistence of Structure

    There was a lot of thoughts on how to store Structure elements. There was two options: either store them as special artifacts or dedicated objects. We decided for "Dedicated objects", see bellow for the arguments. The arguments of Artifacts can be found in followup comments of this epic.

    Dedicated objects

    Artidoc Structure sections would be backed as dedicated objects, at artidoc level

    • Good, because it's simple, we don't need to worry about Artifact side effects

    • Good, because as it's simple, the implementation should be faster for initial use case

    • Good, because it doesn't add more pressure to Artifact* tables (in term of volume)

    • Bad, because there is a need for a specific management of images (including TUS upload)

    • Bad, because we will have to implement specific features for

      • Full Text Search

      • Import / Export

      • Cross tracker search / tracker integration (search on presence in a section, display sections of documents)

    • Bad, because we will have to implement history from day 1 (including images)

    • Bad, because it cannot be extended to manage "fields" (but maybe we can convert them to artifacts, or have explicit Requirement Artifacts for that)

    • Bad, because we might need to manage comments with two different approaches if we decide to store artidoc comments as Artifact Follow-up comments (but that might not be a good idea after all).

    Free text

    Free text is a section (Title + Description) that is not backed by an artifact. It's possible to have Free Text without description (pure title), Free Text with both Title and Description but it's not possible to have Free Text without title.

    Free text section offers the same formatting possibilities than artifact sections, including images and Subtitles

    When an Artidoc is duplicated, the free text sections are duplicated too.

    It's possible to create Free Text sections in an empty document.

    When author remove a Free Text from a document, they should confirm removal (it's not possible to re-add afterward unlike artifacts).

    Introduction of free text have an impact on the management of document versions (see dedicated section after).

    Management of titles

    The management of titles is a challenge in itself. With the original (16.0) approach we have two levels of titles to manage:

    • One that is given by the artifact (and with the introduction of Structure, of artifacts)

    • One that is inside the paragraph of the section (description)

    With the introduction of Structure, we will have two series of titles to manage. The titles given by the Structure (1. Introduction, 1.1 Context) and the titles inside paragraphs.

    This might not be too much of an issue for Requirement Sections because it's a special type of Sections. But for Structure, it will be unbearable for end users. They will be proposed to create a new Structure. This Structure could be just a Title, just a Text or both. If they choose to text, they can select a Title inside the paragraph but this title won't be taken into account in the outline 🤯.

    Structure have 3 level of titles

    Structure provides 3 levels of title (Heading 1, heading 2 and heading 3). While technically we can go up to 6 levels of titles, it's quite hard to make visual differences between levels after level 3. Plus it seems to be a good practice to limit the number of levels to reduce the complexity of documents. Finally, if a compelling need to have more than 3 levels of titles arise, we can add a new one whereas removing a level if it generates too much of complexity is way more complex.

    Structure titles are the only one that appears in the Table of Contents.

    Descriptions have one level of title

    Descriptions (of artifacts or free text) only propose one level of title that is called Subtitle. Subtitles doesn't appears in the table of contents and cannot be linked.

    The goal is to let users organize their descriptions with a meaningful tag (subtitle vs using bold to fake a title) while limiting the risk of misunderstanding between Headings and and Subtitles.

    Note: existing artifacts that use more than one level of title in their description are rendered with an appropriate font size. It's however not possible to use the other description titles size (h2 and h3) in Artidoc.

    Scenario of title usage

    Limitation of headings

    Example:

    1. Requirements
    2. <h1>Radiation [-> cannot move]
    2.1 <h2>Advanced spacecraft [-> can move to h1]
    2.1.1 <h3>Life support [-> can move to h1, h2]
    • Radiation title cannot be changed because there is already an h2 and h3 below

    • Advanced spacecraft can move to h1 but not h2 because there is a h3 below

    • Life support can move to h1 or h2

    Reorder section doesn't change the depth of titles/sections

    1. Requirements
    2. Radiation
    2.1 Advanced spacecraft
    2.1.1 Life support [move bottom]
    3. Random stuff

    Becomes

    1. Requirements
    2. Radiation
    2.1 Advanced spacecraft
    3. Random stuff
    3.1.1 Life support [move top]

    Becomes

    1.1.1 Life support
    2. Requirements
    3. Radiation [move top]
    3.1 Advanced spacecraft
    4. Random stuff

    Becomes

    1. Radiation
    1.1.1 Life support [move down]
    2. Requirements
    2.1 Advanced spacecraft
    3. Random stuff

    Becomes

    1. Radiation
    2. Requirements
    2.1.1 Life support
    2.2 Advanced spacecraft
    4. Random stuff

    User can create incoherent levels of titles

    1. Requirements
    2. Radiation
    3. Advanced spacecraft
    4. Random stuff -> move h3

    Becomes

    1. Requirements
    2. Radiation
    3. Advanced spacecraft
    3.1.1. Random stuff
    Progress
    2024-12-05
    2025-02-26 (59 working days)
    On going
    Details
    #40128
    Manuel Vacelet (vaceletm)
    2025-02-04 15:51
    2024-11-04 16:24
    Attachments
    Empty
    References
    Referencing epic #40128

    Follow-ups

    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    • links
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar

    The following text is a copy of the original thoughts regarding management titles, they are kept as followup to keep the original description focused on the expected behaviour

    Management of titles

    Several options are possible:

    1. Forbid usage of titles inside paragraph

    2. Unify title usage between structure and paragraph

    3. Don't address the issue

    4. Forbid usage of titles inside paragraph only for Structure artifacts

    Forbid usage of titles inside paragraph

    • Good, because it solves the problem

    • Bad, because it puts constraints on underlying artifacts that must be enforced everywhere an artifact can appear (artifact view, artifact modal, API, etc).

    • Bad, because existing documents might be broken (need to manage the existing docs)

    • Bad, because users will have to rely on "Fake titles" to organize their Requirements (font size + bold).

    Unify title usage between structure and paragraph

    For instance: given I'm editing 2.1.1 Lens coating (that is a h3), I can only select title H5 - H6

    • Good, because it solves the problem

    • Bad, because it puts constraints on underlying artifacts (Same as prop n⁰1)

    • Bad, because the artifact cannot appear in two Artidocs at two different places (otherwise the consistency between the titles will be broken in one of the documents)

    Don't address the issue

    • Good, because it means less development right now

    • Bad, because it leave users with questionable UX (two hierarchy of titles)

    Forbid usage of titles inside paragraph only for Structure artifacts

    • Good, because it solves the problem

    • Good, because as it focuses only on Structure artifacts, there is no existing documents to manage

    • Bad, because it puts constraints on underlying artifacts (Same as prop n⁰1). However, as those artifacts are supposed to be hidden, this might not be needed in practice as the important thing is to have something consistent at Artidoc level. If someone tries to fiddle with description at API level, it will be their problem (if any).

    • Bad, because it creates a difference in behavior between Structure paragraphs and Requirement paragraphs


    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar

    Keep reflexion around the hidden tracker as a followup comment so we can refer to hit later on but keep the epic concise.

    Hidden tracker

    Technically speaking, Artidoc Structure is backed by Artifacts from a specific Tracker "Text (text)" (or Artidoc Structure (artidoc_structure) ?) with a limited structure (Title, Description, Links). This tracker is not meant for general usage so it's hidden.

    Rational behind Structure as Artifacts (ADR format):

    • Good, because we are already able to manage sections backed as artifacts. So we don't need to introduce a new mechanism.

    • Good, because being able to manage a document with parent/child relationship is also a requirement (think Epics/User Stories) even if it's not the primary target of Requirement Management.

    • Good, because the import/export mechanism will work the same for Structure Sections and regular Sections.

    • Good, because the history of changes comes for free.

    • Good, because the Structure Artifacts will be indexed in Full Text Search.

    • Good, because we can leverage search on cross tracker capabilities (like IS LINKED FROM/TO when will need to use Structure in Tracker context)

    • Good, because management of comments will be unified between the two types of elements.

    • Bad, because Artifacts are more complex to manage than... anything else.

    • Bad, because it introduce a new concept of "Hidden tracker". Due to the extensive presence of Artifacts everywhere in Tuleap, we will need to ensure they don't popup in unexpected places (esp. via Artifact Links).

    • Bad, because it might introduce some weirdness at Artifact level when the text are modified (1.2 Context becomes 1.2 Terms and definitions and later something else. At artifact level, the history might be clumsy depending of the way the document is re-organized).`

    How to Hide a Tracker

    The tracker that will hold the Titles and Free texts will be named "Artidoc Structure". This tracker:

    • is hidden from the list of trackers of a project,

    • cannot be modified by a project or site administrator (administration blocked).

    The goal is to completely hide the tracker existence for users (end users, project admins and even site admin).

    Some lesson learned on TTM (test_exec) and Program Management (is_mirror_of).

    • test_exec is a typical example of something backed in an artifact but that could have been done with a dedicated object. It works mostly fine and almost transparently for end users. However, some users have introduced changes on the structure itself (to capture more information on test exec) and are now requesting to display and manage those fields in TTM. On the bright side, they managed to work around some limitations of the tool, on the other side, the resulting UX isn't great.

    • is_mirror_of is a bit different. The main lesson learned here is that, as soon as we display the information about the artifact link and type, people expect to be able to manipulate it directly at artifact level (and it's not possible for is_mirror_of, the logic is in Program Management).

    Conclusion, it Artidoc Structure is an Artifact, it should be completely hidden. That means block usage in

    • Artifact view

    • Tracker reports

    • Artifact Links reverse links

    • Direct API usage

      • Should not be listed in tracker API

    • Hierarchy / Trigger

    • Backlog (planning configuration)

    • Program management

    • TTM

    • Verify also in semantics

    At some point, from architectural stand point, we should refrain ourselves from relying on artifacts directly in the code (hexagonal architecture) and always use proxy.

    Permissions

    Permissions of Structure artifacts should be tight to document permissions (one should be able to write a document of type "Minutes" while not being able to write a document of type "Specifications"). We cannot rely on current tracker permissions mechanism (too static), tracker should delegate permissions management to document.

    Cross references

    When someone type a reference inside a Structure artifact, the referred artifact will automatically have a cross-reference back to the section (that should be hidden). The expected behavior would be that the cross-reference point to the Artidoc itself, not the section (that should be hidden).

    Tracker creation

    There is one Artidoc Structure tracker per project (to ensure separation of data and ease import/export). The tracker is created on the fly when not existing. When the tracker exists in the template, it is duplicated a project creation.

    Import/export

    Not needed in the first implementation but import/export should be under the responsibility of Document instead of directly trackers.

    Relation between Structure Artifacts

    Given the following organization:

    1. Introduction (Structure)   
      1.1 Context (Structure)
      1.2 Terms and Definitions (Structure)
    2. Requirements (Structure)
      2.1 Optical (Structure)
        2.1.1 Lens coating (Requirement)
    • The relation between Introduction and Context is a parent/child

    • The order between Context and Terms and Definitions is defined by rank

    • The relation between Optical and Lens coating is an artifact link of type "is_parent_section" (Optical -is_parent_section-> Lens Coating))

      • The new link type is used in order to be able to associate special behavior (as we did before with artifact folders or requirement management)

      • Re-using parent/child between Structure and Requirement could lead to conflicts if Requirement is part of a hierarchy.


    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    • Status changed from Planned to On going
    • Start date set to 2024-12-05
    • End date set to 2025-02-26
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes