Allow to re-import/continue with a closed job

Here the problem is that if a translation is done and accepted it is the end of the workflow. So if further changes are required a new job needs to be created. The problem in such case is that the old xliff file cannot be imported in the new job.

I have tried a few scenarios how this could be possible, but it is either breaking the workflow or hacking the file translator to skip validation.

What would probably solve this problem is a in-site preview. There is already an issue for this: #1998060: Provide an in-site preview for translator

We need to investigate the state diagram first.

The preview feature is way more than our current problem and requires lots of new features added.

If you think about local translator and the need for comments and other interaction in review process, it's 100% clear that a submission of the translation is no end of the process, but treated much more like a suggestion. The status of the job should represent this.

The technical problem is that the translation requires "accept" state on server side to make it pass back to the client. This is just wrong and derived from the reuse of client code.

A final closed state of a job should only happen with positive confirmation from the client side (or possibly a timeout on the server side because of lack of feedback from the client.)

For this, the XLIFF side doesn't need to have any kind of Preview of what they have uploaded.

First this issue is about local XLIFF processing without any client + server combination.

Let's make sure the fundamental process works cleanly.
- importing an XLIFF needs to work as long as there was no accept yet.
- Partial overwrites should work on items that are not accepted yet, as long as others are accepted.
-- In case of clashes, messages should be verbose about the item

Then, once a job is accepted, sure XLIFF import should not work and even disappear.

This all needs tests.

For further changes to content in Drupal, we would expect that we need to create another job and create an export again. This re-export should containt the current translation
#2006786: XLIFF export should be with empty targets
This is a hard problem. We need to think about all the workflow of copying the previous job, detecting if the source changed.
#1686544: Check for source change on review and show diff

The functionality described is kind of there...

When re-importing into job items that are to be reviewed, translations get updated.

When re-importing while some job items are accepted following message is displayed:

Translation for customized body][0][value received. Revert your changes if you wish to use it.
Updated translation for key node_title, size difference: 2 characters.
Sucessfully imported file.

And status is that all translations get imported and are set to in review state. However for the item that has been previously set as accepted the translation is imported as a revision and not set as the actual version.

So what I suggest:
- If something got accepted we should not switch it back to review
- We have that revisions feature that does not behave just as we would expect - we take action if translation is accepted and is the same as the one being imported. Question is what to do in case we have accepted item and the imported translation is different? Should we ignore it? There are two scenarios: I fixed a small typo and accepted the item; I accepted the item but the translator has fixed some wording. I guess if there is something accepted we should not do any updates to the translation, otherwise it gets complicated and user can always just unaccept the item and run the import again.
- Adding test coverage for all this workflow is not tested at all.

Here is the workflow:

- reimporting of xliff will update all changed items A) if in translated state it will update the actual translation and the previous will become an older revision B) if accepted the imported translation will become a revision on the top, while it is possible to revert to make it as actual translation
- if nothing changed the import for given data item will skip

Here are some findings:
- the review actions reviewed/unreviewed do not have test coverage, will open a followup for this
- clicking one of the review buttons will persist also changes of other items - not sure if we want this
- initially the review actions behaved strangely, that is due to fact that all items get updated and the comparison if translation has been updated sometimes failed due to different whitespaces. Now we remove whitespaces and newlines, is it just solution?

rerolled

Here is to summarise how we handle the translation workflow:

We update only items that are changed. In case there is local modification the import loads as a revision and adds a message that you can use it by reverting to it.

We do merge at the data items level, so updating only those that are changed, and as written above, we add a message if there is a conflict. 1 Here is one problem tough. In case we have a local modification for a data item and the status is not "accepted" and we import translation, a revision is added even though the imported XLIFF contains old translation version for that data item.

In the #2032869: Remove update is ignored if text did not change and data item is in pending state we addressed the problem of data item status not being updated in case it was rejected and same translation came back from translator. 2 This I think will cause an issue. We can have multiple data items, and two are being reviewed/rejected. We import XLIFF that has updated translation only for one of the data items. Result will be that BOTH will be switched to status "translated".

So we need to somehow deal with mentioned issues 1 and 2.

10: 2053699-reimport_file-3.patch queued for re-testing.

+++ b/entity/tmgmt.entity.job_item.inc
@@ -550,9 +550,11 @@ class TMGMTJobItem extends Entity {
       $data = $this->getData(tmgmt_ensure_keys_array($key));
       if (empty($data['#status']) || $data['#status'] != TMGMT_DATA_ITEM_STATE_ACCEPTED) {
+        // To determine if text has been updated do only the character
+        // comparison by removing the white spaces and new lines.
         // If we already have a translation text and it hasn't changed, don't
         // update anything.
-        if (!empty($data['#translation']['#text']) && $data['#translation']['#text'] == $translation['#text']) {
+        if (!empty($data['#translation']['#text']) && preg_replace('/[\n\r\s\t]/', '', $data['#translation']['#text']) == preg_replace('/[\n\r\s\t]/', '', $translation['#text'])) {
           return;
         }

Haven't re-read the whole issue, wondering why we're adding this here?

This also applies to own changes, so this means it won't accept if you add a missing space somewhere?

Is this related to the testfails?

Oh, that is actually the only thing that we are changing here, so I even less understand what's exactly going on :)

Ok, I think those patches and discussions drifted quite far from what the issue is about, being able to continue with the same xliff files after an existing job after things have been accepted and is marked as finished.

That is not easy right now, but we could get quite close with a combinatation of two things:

- We introduced the resubmit feature a while ago but only for an aborted/cancelled job. It would be trivial to make that available for a finished jobs as well, to create a new job with the same items that then extract the current source, you can also throw out certain items at that point (not yet add more apart from suggestions)

- Right now we validate an imported file to match the job and job items. We could think about supporting an import from a different job and attempt to map different id's based on the item definition. This would probably require a confirmation screen that lists both jobs and the items that we can match.

While the re-submit for closed is trivial to implement, the second part is not, so changing this to a feature request.

We should also allow to clone a job that is currently in progress. This is needed if you create a job from language A to B and later (possibly while the job is still pending / in review) want to create a job into language C. Currently it is required to first reject a job.

Allowing resubmit to be allowed for accepted items seems trivial.. so I tried to test it out and for some reason during resubmit the ids are still unique so you can't use the same file.