Thanks for your feedback Yannis.
I am going to test it in Tuleap 8.4 for lists and open lists.

This feature is already merged in Tuleap 8.4 (for all lists and open lists).

In Tuleap 8.5, we already worked on permissions_on_artifact field, the cross references in the followup comments were modified to not be taken into account and the field artifact_link is skipped as discussed in our last meeting.

Tha last thing we are working on for Tuleap 8.5 is the export/import of attachments.

Ok, thanks.
It works well now.

Could you tell me when it will be possible to export/import select box and multi select box too?

Before doing these two commands, please do

$> cd /usr/share/codendi

I get this: -bash: src/utils/php-launcher.sh: No such file or directory

The login of one of your site_admin

In the first command:

src/utils/php-launcher.sh src/utils/export_project_xml.php -p PROJECT_ID_TO_EXPORT -u admin -t TRACKER_V5_ID -f > /tmp/data.xml

Do you mean by "admin" the project admin or the site admin?

Ok so to test on Tuleap 8.3 you must have a tracker with only the following fields:

1. string / text
2. int / float
3. date / datetime

To test it, you have to run these 2 commands:

1. src/utils/php-launcher.sh src/utils/export_project_xml.php -p PROJECT_ID_TO_EXPORT -u admin -t TRACKER_V5_ID -f > /tmp/data.xml
2. src/utils/php-launcher.sh src/utils/import_project_xml.php NEW_PROJECT_ID USERNAME_IMPORT /tmp/data.xml

On Tuleap 8.3.

Hello Salma,

On which version will you test this feature ?

Hello,

I need to test what is done in this story.
Could you explain me how to proceed, please?

Best regards,
Salma

Other things that are missing in the current state:

• dry-run mode
• file of users mapping

See denis' comment for details.

Hello Nicolas,

Sorry for the late answer.
The most big tracker v3 we have to migrate contains actually 3203 artifacts (and it will increase).

Best regards,
Salma

Current status

We can export a tracker (v5) from a platform to another with its artifacts. However, we faced difficulties to handle all the fields. The remaining work as been detailed in another story (see story #8162).

For now, we are able to:

1. Export and import project's static ugroups + members (based on LDAP ID or username). If a ugroup with a same name exists, the ugroup from XML (with its members) is skipped.
2. Export and import tracker structure (with all fields)
3. Export and import artifacts with alphanumeric and date (with or without time) fields + comments (if comment was edited, we export and import only the last version)

 

Remaining work

The remaining work (with some other points) is detailed in the story #8162.

Due to unexpected difficulties encountered during the development, we must shift the initial estimation (5 points for the complete story becomes 8 points for this first step and 8 points for the remaining work).

Hi,

Could you please tell us how many artifacts in a tracker you want to migrate ?

In order to not consume all performances of the server during export, we would like to put a threshold. Your numbers will probably give us a clue to define such threshold.

Regards,
Nicolas Terray

@denis_pilat there are no reason that the migration TV3 -> TV5 (with the content) done by the Web UI does not work on PHP 5.1 instances.

Edit: I just tested this on our PHP5.1 RHEL5 machine and the migration works with the content

That could be a possibility yes.
We need to check if the tracker (TV3) we want to migrate are "exportable".
THey are running on CodexSTN (32 bits, RHEL5) ...

To fit your requirement:

* Export your TV3 in TV5 in Tuleap by the UI. (on server1)
* Export the newly created TV5 with the CLI tool (on server1)
* Import this exported TV5 in the new server with the CLI tool (on server2)

Unless I haven't well understood, I think it will not fit our initial requirement which is to move a tv3 from one server to tv5 on another one .
Tv3@server_1 ===> import with cli@server_1 ===> export with cli@server_2 ===> Tv5@server_2
This requirement is part of the acceptance criteria.
Could you please elaborate more?
thank you in advance.

Hello Nouha,

You will generate the output by doing in console:

$> php export_project_xml 110 --tracker 15 > project.xml

You will have an XML (containing project's ugroups and the selected TV5) like:

<project>
  <ugroups>
    <ugroup name="ug01">
       <members>
         <member>user01</member>
       </members>
    </ugroup>
      ...
  </ugroups>
  <trackers>
    <tracker>
      ...
    </tracker>
  </trackers>
</project>

Once generated, in the console, you will launch the following command to import this XML:

$> php import_project_xml 114 PATH_TO_project.xml

 

Do you have any other question ?

Hello,

Could you please detail more?
what would be the input (tv3 or tv5) and what would be the output?

regards,
Nouha

I suggest that a follow-up comment is clearly added

This would have an impact on the result: all impacted artifacts would have their "Last modified date" changed to the date of import. Would it be a problem ?

Note: I don't know the technical impact of adding this (hence if it can be done with the first estimation).

we might consider to create the accounts on the destination, and to set them as “deleted”

This is possible but cannot be covered by this story with the current estimation.

I suggest we go without it as a first step and add it later on when it's mandatory

 

“In both cases, we will have to deal with inconsistencies.”: YES of course! And you’ll be paid for that !

1. In case the use does not exists on the destination, whaterver the choice, and since it’s import of legacy data, I suggest that a follow-up comment is clearly added into the destination to say that “George Clooney as an assignee has been replaced by “none” during the migration step.
2. As a result of the migration steps, the list of artifact where inconsistencies are discovered (permission issue as you suggest, but any others) must be the output of the migration. Site_admin will then inform the tracker admin that he has admin action to do.

 

“I'm tempted to say that the easiest solution would be to re-assign at import:”

Good idea, after the dry-run, site_admin might have the possibility to replace any inconsistencies with a single user. But the above 1, 2 suggestions remains valid, adding a follow-up comments + having the list of artifacts.

Migration between 2 platforms with huge number of differences:

Yes a mapping file with a syntax like you propose can do the job. Potentially it could be the unique option of the script where we might have a syntax like

terzino:nouha_terzi
dpilat:denis_pilat
* :none  // for unwanted mapping.

 

The –dry-run option can generate such map file as an output.

Then in some case Nouha, we might consider to create the accounts on the destination, and to set them as “deleted”. That also would be a good option for persons we don’t want to map to “none”

In the maping file, we can consider

terzino:none
dpilat:#CREATION_OF_USER#
* : #CREATION_OF_USER#

 

Waiting for your feedback ?

 

Denis

 

It is Ok for me.
@denis_pilat, what do you think?

What about a migration between 2 platforms without the same authentication model (ldap vs db) where the convention of naming is different (shor login vs surname_lastname), the site admin, will be prompted for all users ( ~ 1000 users).

IMHO, such a case is really different.

In such scenario, I expect that the 1000 users are automatically created on the platform.

Replacing all different users by only one would be too aggressive to be default. For support purpose for instance, it's easier to be able to replicate precisely the assignee/submitter as it has a great impact on the behaviour of the tools.

That said we could

• Ask for a mapping instead of prompting (the import analysis would dump a list of username and you would be able to re-launch the import with the mapping file like "terzino:nouha_terzi\ndpilat:denis_pilat" and so on.
• Add an option at import like "replace all non existant users by XYZ" (XYZ provided by the admin).

It seems to cover both cases

 

Oh if Georges Clooney, it will be always OK for me:).
However, I think the scenario seemed to be too tailored to the migration between 2 servers that are too similar point of vue users (aka codexstn & codex)
What about a migration between 2 platforms without the same authentication model (ldap vs db) where the convention of naming is different (shor login vs surname_lastname), the site admin, will be prompted for all users ( ~ 1000 users).

I think it would be better to have George Clooney in replacement for all non existing user since the begin of the migration (passed as arg), and not to treat it one by one.

What do you think?

There are 2 possiblities:
- None
- A new fake/admin user "Unkown user" (like forge__workflow_manager or im-admin)

In both cases, we will have to deal with inconsistencies. To name a few:
- "assigned_to" mandatory field with "None" as value: the artifact is not valid :/
- artifact submitted_by "None" (esp. a problem with permissions tight to ugroup, as "I can see all artifacts submitted by group")

I'm tempted to say that the easiest solution would be to re-assign at import:
- Example, sanity check at import tells that "User john_doe is not valid on this platform"
- Admin decide to replace "John_Doe" by "George_Clooney"

What do you think ?

"When it's a member of a user group, no problem, she's just "skipped" but when the user is creator or assigned to an artifact, who should be submitter/assignee then ?".
CAN we have "none" or "nobody" or a kind of "unknown user following import" ?

I just created the corresponding epic and the spec proposal here:
https://tuleap.net/plugins/tracker/?aid=8107&group_id=101

> If you wish we can share it and see what can be co-developed.

Of course, let's create an epic and discussing the point there: https://tuleap.net/plugins/tracker/?tracker=149&func=new-artifact

Ok then, let's start with this feature .
FYI, some of the listed items are already ongoing at ST side as we planed to develop the possibility to import/export project between forges:
> add docman export to the intermediate format (nota: export should be ok but will need to rewrite the import as it relies on SOAP as of today)
We have a sort of specification, we detailed what will be covered/imported.
If you wish we can share it and see what can be co-developed.

Yes it can be made more generic but we need to re-evaluate the estimation as, as of today, there is no way to export a tv5 data to the XML format.

IMHO, this should be done after in a dedicated story as there is nothing in this story that would be done differently by adding this new requirement.
There are actually other possible stories to make it more generic:
- export ALL tv3 of one project into the intermediate format (the current one only export 1 tracker)
- export ONE tv5 into the intermediate format
- export ALL tv5 into the intermediate format
- add docman export to the intermediate format (nota: export should be ok but will need to rewrite the import as it relies on SOAP as of today)
- import/export list of used services

With all those we have a solid base for a "full" project export/import, some things would not be exported (forums, survey, etc) but this might not be relevant enough to block a migration.

I would say, the person doing the migration (aka the siteadmin) in case of the "creator" field and "nobody" for the assigned to one.

I think it is better to make the feature more generic, thus allowing the import from tracker v3 or V5 not only from V3.
What do you think?

Question to Denis/Nouha/Salma: what should be the behaviour when the user no longer exist (for instance left the company) ?

When it's a member of a user group, no problem, she's just "skipped" but when the user is creator or assigned to an artifact, who should be submitter/assignee then ?