Problem/Motivation
TMGMT can leave job items behind after their source content entity is deleted. For example, a site can request translation for a node and then delete that node while the corresponding job item still exists.
The job item still stores the deleted entity type and entity ID, but the source entity can no longer be loaded. This can cause problems for providers that expect the referenced entity to exist.
Note: there is also a related, but separate, problem in #2913008: Cannot abort or delete item if the source entity has been deleted
That issue is about manually handling job items whose source is already gone. This issue is about optionally removing matching job items at the time the source entity is deleted.
Steps to reproduce
- Install and configure TMGMT with a content source and a translator.
- Create a translatable content entity, such as a node.
- Create or request a translation job item for that entity.
- Delete the source entity before the job item is completed.
Actual result: The related TMGMT job item still exists and references a deleted source entity.
Expected result: Provide a way to automatically remove matching job items when their source entity is deleted.
Proposed resolution
- Add an opt-in purge_stale setting that allows TMGMT to remove job items when their source entity is deleted.
- Default the setting to FALSE, including for existing sites, so current behavior is preserved unless a site explicitly opts in.
- Keep continuous jobs even when they become empty, because they are persistent containers for future continuous job items.
Remaining tasks
- Review & merge
User interface changes
Adds a configuration option to enable purging stale job items when their source entity is deleted.
The option is disabled by default.
API changes
None
Data model changes
No database schema changes.
Adds a purge_stale key to tmgmt.settings, defaulting to FALSE.
| Comment | File | Size | Author |
|---|
Issue fork tmgmt-2998143
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
loparev commentedComment #3
miro_dietikerA first feedback here:
A translator plugin needs to be made robust enough to deal with missing items. We already had many small related fixes.
Comment #4
loparev commentedI agree. But is there a reason why `ghost` job items/jobs aren't being deleted when the entity is deleted?
Comment #5
miro_dietikerThe idea was that we want to remember what we submitted to the provider.
I would prefer to keep it and maybe mark it stale if we know it's outdated.
Comment #6
loparev commentedWhat if this feature will be optional? Just as a POC, this patch provides a `Purge stale` option and TMGMT will remove outdated items/jobs only if it's enabled.
Comment #7
loparev commentedComment #8
miro_dietikerThank you for the proposal, we will need to discuss this direction.
Note also that core also doesn't support such a thing:
If you delete an entity B that is referenced from entity A, the entity A will have a stale reference item.
Comment #9
antoniya commented+1
Comment #10
berdirThanks for working on this. Makes sense to clean up, I'd even suggest to have it enabled by default, but then keep it disabled for existing installations with update function.
The properties need documentation.
I think we can write the first line for those two tests so they explain what it does in a single line, something like:
Tests purging when enabled/disabled.
Pretty sure this can be an entity query instead.
Instead of loading the jobs, why not load the job items directly and delete them.
And then I would do another query to look if there are more job items with the same job id, just doing a count query.
The reason for that is that we have continuous jobs, which might have dozens or hundreds of job items, and getItems() would load them all.
Comment #11
pratik_kamble@berdir The earlier patch was not working as 'purge_stale' config was not getting save. Updated it and made changes as per suggestions mentioned by you.
Comment #12
pratik_kambleComment #13
pratik_kambleAbove patch had malformed. Attaching new one.
Comment #15
pratik_kamble@Berdir I think even on the existing site where the module is in use. Stale Jobs remove functionality should be enabled by default.
Comment #16
tarasichTested patch from #13. Works good. Only tests should be fixed
I don't think option should be enabled by default on existing projects. It may become "surprise" for someone. Would advice against using patch from #15
Comment #19
murzI've adopted UI tests to the changed behavior with deleted entities, and have committed to the MR https://git.drupalcode.org/project/tmgmt/-/merge_requests/35 together with the last patch from @pratik_kamble. Please review.
Comment #20
odai jbr commentedA re-roll for #19 which I needed after applying a patch from Issue #3097660
Comment #21
google01 commentedUnfortunately this patch is adding code without explicit accessCheck(TRUE/FALSE) to content entity queries.
For example in the function "tmgmt_entity_delete" in the code file "tmgmt.module". This case is really worrying because it causes errors in the removal of any content, leaving Drupal 10 websites completely inoperative.
Comment #23
colorfieldAdded
accessCheck()to the MR as it is a blocker when deleting any kind of entity, including configuration, so also causes deployment issues.We might still decide whether or not we want this behaviour to become the default one (see #16).
Comment #24
dewalt commentedJust adding diff from the MR to use with composer. The diff applies for 1.15 modules version, but has conflicts with the latest dev version.
Comment #25
berdirYes, this conflicts badly with another feature we added.
Comment #26
oways23 commented@berdir
what are the new features that conflicts with this patch?
Comment #27
colorfieldRebased with latest 8.x-1.x and fixed conflicts, had to rename
tmgmt_update_8011astmgmt_update_8012.If you previously applied one of the previous patch from this issue with
tmgmt_update_8011, just make sure to run the one from https://git.drupalcode.org/project/tmgmt/-/blob/8.x-1.x/tmgmt.install?re..., otherwise creating a job will fail withDrupal\Core\Database\DatabaseExceptionWrapper: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'file_count'.Comment #28
colorfieldComment #29
redneko commentedThere are now merge conflicts that need to be resolved.
Comment #31
herved commentedI'll work on this
Comment #32
herved commentedI rebased and made a few key changes:
- Don't delete continuous jobs when they become empty, they are persistent containers that keep collecting items for new content (addresses the open review note on `tmgmt.module:157`).
- For existing installs, set
purge_staletoFALSE, as per #16 (so the feature is opt-in and existing sites keep their current behavior)For new installs, set
purge_staletoTRUE, as per #10- Skip tmgmt's own entities in the delete hook, so purging job items/jobs doesn't pointlessly re-enter it.
- Updated test coverage
- General tidy-up: guard clauses, return types, and simplifying the kernel test using entity_test.
---
#2913008: Cannot abort or delete item if the source entity has been deleted is a related but separate issue and shouldn't be bundled here IMHO: it addresses the issue when a job item whose source is already gone can't be aborted or deleted from the UI, whereas this issue auto-removes job items when their source is
deleted. Different problems; both still needed, but better addressed separately.
Comment #33
herved commentedComment #34
herved commentedComment #35
claudiu.cristeaA question on the new settings default value on new installs
Comment #36
herved commentedComment #37
claudiu.cristeaThe MR doesn't change the behavior of existing sites but applies as default for new installs. This is correct, IMHO. There is test coverage and the code looks good. RTBC!
Comment #39
claudiu.cristeaThank you all for contribution. Merged