•  
      epic #40128 Structure with Freetext
    Summary
    Structure with Freetext
    Artidoc

    Artidoc Structure covers the following elements:

    • having titles to organize sections (1. Introduction > 1.1 Context, etc)

    • having zones of "free" text. Free text is a text that is not part of the tracker associated to an Artidoc. For instance, given an Artidoc associated to a Requirement tracker, a new free text section will not generate a new Requirement artifact.

    Persistence of Structure

    There was a lot of thoughts on how to store Structure elements. There was two options: either store them as special artifacts or dedicated objects. We decided for "Dedicated objects", see bellow for the arguments. The arguments of Artifacts can be found in followup comments of this epic.

    Dedicated objects

    Artidoc Structure sections would be backed as dedicated objects, at artidoc level

    • Good, because it's simple, we don't need to worry about Artifact side effects

    • Good, because as it's simple, the implementation should be faster for initial use case

    • Good, because it doesn't add more pressure to Artifact* tables (in term of volume)

    • Bad, because there is a need for a specific management of images (including TUS upload)

    • Bad, because we will have to implement specific features for

      • Full Text Search

      • Import / Export

      • Cross tracker search / tracker integration (search on presence in a section, display sections of documents)

    • Bad, because we will have to implement history from day 1 (including images)

    • Bad, because it cannot be extended to manage "fields" (but maybe we can convert them to artifacts, or have explicit Requirement Artifacts for that)

    • Bad, because we might need to manage comments with two different approaches if we decide to store artidoc comments as Artifact Follow-up comments (but that might not be a good idea after all).

    Free text

    Free text is a section (Title + Description) that is not backed by an artifact. It's possible to have Free Text without description (pure title), Free Text with both Title and Description but it's not possible to have Free Text without title.

    Free text section offers the same formatting possibilities than artifact sections, including images and Subtitles

    When an Artidoc is duplicated, the free text sections are duplicated too.

    It's possible to create Free Text sections in an empty document.

    When author remove a Free Text from a document, they should confirm removal (it's not possible to re-add afterward unlike artifacts).

    Introduction of free text have an impact on the management of document versions (see dedicated section after).

    Progress
    2024-12-05
    2025-02-05 (44 working days)
    Closed
    Details
    #40128
    Manuel Vacelet (vaceletm)
    2025-03-11 12:08
    2024-11-04 16:24
    Attachments
    Empty
    References
    Referencing epic #40128

    Follow-ups

    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    • End date changed from 2025-02-26 to 2025-03-26
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    • links
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar

    The following text is a copy of the original thoughts regarding management titles, they are kept as followup to keep the original description focused on the expected behaviour

    Management of titles

    Several options are possible:

    1. Forbid usage of titles inside paragraph

    2. Unify title usage between structure and paragraph

    3. Don't address the issue

    4. Forbid usage of titles inside paragraph only for Structure artifacts

    Forbid usage of titles inside paragraph

    • Good, because it solves the problem

    • Bad, because it puts constraints on underlying artifacts that must be enforced everywhere an artifact can appear (artifact view, artifact modal, API, etc).

    • Bad, because existing documents might be broken (need to manage the existing docs)

    • Bad, because users will have to rely on "Fake titles" to organize their Requirements (font size + bold).

    Unify title usage between structure and paragraph

    For instance: given I'm editing 2.1.1 Lens coating (that is a h3), I can only select title H5 - H6

    • Good, because it solves the problem

    • Bad, because it puts constraints on underlying artifacts (Same as prop n⁰1)

    • Bad, because the artifact cannot appear in two Artidocs at two different places (otherwise the consistency between the titles will be broken in one of the documents)

    Don't address the issue

    • Good, because it means less development right now

    • Bad, because it leave users with questionable UX (two hierarchy of titles)

    Forbid usage of titles inside paragraph only for Structure artifacts

    • Good, because it solves the problem

    • Good, because as it focuses only on Structure artifacts, there is no existing documents to manage

    • Bad, because it puts constraints on underlying artifacts (Same as prop n⁰1). However, as those artifacts are supposed to be hidden, this might not be needed in practice as the important thing is to have something consistent at Artidoc level. If someone tries to fiddle with description at API level, it will be their problem (if any).

    • Bad, because it creates a difference in behavior between Structure paragraphs and Requirement paragraphs


    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar

    Keep reflexion around the hidden tracker as a followup comment so we can refer to hit later on but keep the epic concise.

    Hidden tracker

    Technically speaking, Artidoc Structure is backed by Artifacts from a specific Tracker "Text (text)" (or Artidoc Structure (artidoc_structure) ?) with a limited structure (Title, Description, Links). This tracker is not meant for general usage so it's hidden.

    Rational behind Structure as Artifacts (ADR format):

    • Good, because we are already able to manage sections backed as artifacts. So we don't need to introduce a new mechanism.

    • Good, because being able to manage a document with parent/child relationship is also a requirement (think Epics/User Stories) even if it's not the primary target of Requirement Management.

    • Good, because the import/export mechanism will work the same for Structure Sections and regular Sections.

    • Good, because the history of changes comes for free.

    • Good, because the Structure Artifacts will be indexed in Full Text Search.

    • Good, because we can leverage search on cross tracker capabilities (like IS LINKED FROM/TO when will need to use Structure in Tracker context)

    • Good, because management of comments will be unified between the two types of elements.

    • Bad, because Artifacts are more complex to manage than... anything else.

    • Bad, because it introduce a new concept of "Hidden tracker". Due to the extensive presence of Artifacts everywhere in Tuleap, we will need to ensure they don't popup in unexpected places (esp. via Artifact Links).

    • Bad, because it might introduce some weirdness at Artifact level when the text are modified (1.2 Context becomes 1.2 Terms and definitions and later something else. At artifact level, the history might be clumsy depending of the way the document is re-organized).`

    How to Hide a Tracker

    The tracker that will hold the Titles and Free texts will be named "Artidoc Structure". This tracker:

    • is hidden from the list of trackers of a project,

    • cannot be modified by a project or site administrator (administration blocked).

    The goal is to completely hide the tracker existence for users (end users, project admins and even site admin).

    Some lesson learned on TTM (test_exec) and Program Management (is_mirror_of).

    • test_exec is a typical example of something backed in an artifact but that could have been done with a dedicated object. It works mostly fine and almost transparently for end users. However, some users have introduced changes on the structure itself (to capture more information on test exec) and are now requesting to display and manage those fields in TTM. On the bright side, they managed to work around some limitations of the tool, on the other side, the resulting UX isn't great.

    • is_mirror_of is a bit different. The main lesson learned here is that, as soon as we display the information about the artifact link and type, people expect to be able to manipulate it directly at artifact level (and it's not possible for is_mirror_of, the logic is in Program Management).

    Conclusion, it Artidoc Structure is an Artifact, it should be completely hidden. That means block usage in

    • Artifact view

    • Tracker reports

    • Artifact Links reverse links

    • Direct API usage

      • Should not be listed in tracker API

    • Hierarchy / Trigger

    • Backlog (planning configuration)

    • Program management

    • TTM

    • Verify also in semantics

    At some point, from architectural stand point, we should refrain ourselves from relying on artifacts directly in the code (hexagonal architecture) and always use proxy.

    Permissions

    Permissions of Structure artifacts should be tight to document permissions (one should be able to write a document of type "Minutes" while not being able to write a document of type "Specifications"). We cannot rely on current tracker permissions mechanism (too static), tracker should delegate permissions management to document.

    Cross references

    When someone type a reference inside a Structure artifact, the referred artifact will automatically have a cross-reference back to the section (that should be hidden). The expected behavior would be that the cross-reference point to the Artidoc itself, not the section (that should be hidden).

    Tracker creation

    There is one Artidoc Structure tracker per project (to ensure separation of data and ease import/export). The tracker is created on the fly when not existing. When the tracker exists in the template, it is duplicated a project creation.

    Import/export

    Not needed in the first implementation but import/export should be under the responsibility of Document instead of directly trackers.

    Relation between Structure Artifacts

    Given the following organization:

    1. Introduction (Structure)   
      1.1 Context (Structure)
      1.2 Terms and Definitions (Structure)
    2. Requirements (Structure)
      2.1 Optical (Structure)
        2.1.1 Lens coating (Requirement)
    • The relation between Introduction and Context is a parent/child

    • The order between Context and Terms and Definitions is defined by rank

    • The relation between Optical and Lens coating is an artifact link of type "is_parent_section" (Optical -is_parent_section-> Lens Coating))

      • The new link type is used in order to be able to associate special behavior (as we did before with artifact folders or requirement management)

      • Re-using parent/child between Structure and Requirement could lead to conflicts if Requirement is part of a hierarchy.


    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    • Status changed from Planned to On going
    • Start date set to 2024-12-05
    • End date set to 2025-02-26
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes
    User avatar
    • Description
      Something went wrong, the follow up content couldn't be loaded
      Only formatting have been changed, you should switch to markup to see the changes