Remove file on original/duplicated paragraph removes the file on all original/duplicated paragraphs

The patch fails with recent versions of the module. After investigating to see if the behavior still existed, or if we'd need to re-roll the patch, it would appear that this bug has already been resolved. The steps to reproduce above no longer work, at step 10, only the image that was removed is gone - the rest remain.

Step to reproduce

1. Create a paragraph with an image field.
2. Add the paragraph to a node.
3. Upload an image to the paragraph's image field.
4. Before saving, duplicate the paragraph.
5. Remove the image from the duplicated paragraph's image field.
6. Save the node.
7. You will notice that the image is also removed from the original paragraph.

Noticed the same with layout_paragraphs, D11.2 added a hook to react on entity clone in #3040556: It is not possible to react to an entity being duplicated
Maybe we should leverage this? OTOH it would not solve it for sites <11.2 :\
So the changes in the MR seem to make sense to me, but I did not check in detail.

https://git.drupalcode.org/project/paragraphs/-/merge_requests/214 Looks good, my suggestion is to close https://git.drupalcode.org/project/paragraphs/-/merge_requests/203

Looks good move to RTBC.

Tests conflict.

I don't see this is our responsibility. Files are mostly hidden entities, if you delete them then they are gone, I don't think you can even do that in core in the UI. Why would this be different for paragraphs than for files on duplicates notes, blocks or any other entity?

#30 This issue happens when a user duplicates a paragraph that contains a file field and then removes a file reference in one of the paragraphs in the form (so not really the file entity itself).

This is most probably not restricted to paragraphs though. Maybe now that hook_entity_duplicate is here (https://www.drupal.org/node/3268812) the file module should implement this logic.

Something doesn't add up to me. Can you try to reproduce with a duplicated node? Does it happen if you duplicate a node with a paragraph with a file? Maybe it's just a UI/widget but when the same file is used multiple times on the same page? I'm aware of similar bugs with entity browser that should be mostly fixed at this point.

That would be a core bug in the file widget or even file element then.

Files have built-in usage management, they are explicitly designed to be reused (although these days mostly through media entities) and deleted only if all usages are removed, including old revisions. Duplicating the file leads to duplication and extra disk usage

One way to figure out if is just a widget problem would be to use the closed + autoclose mode, so that you always only see one file widget. If the problem can't be reproduced like that then I am almost certain the problem somewhere in the form handling around file element/widget.

#32-33, you're right, I'm using layout_paragraphs, I need to investigate what's going on.

I created a branch 3079729-test-only with a test proving the bug. The IS is still accurate.

The problem is \Drupal\file\Element\ManagedFile::submit that deletes the file as soon as we click on the remove button:

// If it's a temporary file we can safely remove it immediately,
// otherwise it's up to the implementing module to remove usages of
// files to have them removed.
if ($element['#files'][$fid] && $element['#files'][$fid]->isTemporary()) {
  $element['#files'][$fid]->delete();
}

Commenting that code makes the test pass.
That code does not account for paragraphs duplication.
Since the parent entity doesn't exist yet, we cannot rely on file usage.
I assume removing that deletion from core and deferring to cron is not safe either as it would allow a malicious user to flood the server with temporary files that only get cleaned up hours later.
So it's a tricky issue... Any ideas?

For the record, putting here something that we discussed in chat.
Not deleting files on removal is a no go as a security concern. Because I can remove and add new files back and forth and fill in the space in the server.

My idea would be that the paragraphs duplicate mechanism have some sort of way to duplicate the actual files in order to ensure that files will not be lost between paragraphs, but maybe have the paragraphs reference or the entity itself, track somehow the duplicated files.
Then, on save, those that refer duplicated files, will all share the initial one.

However, that is a theory, it involves storing sub-entity references and duplicating the file_usage table functionality but in memory.. I don't even know if it is worth :/
The simpler idea would be to assume that paragraphs always duplicate files when clicking duplicate.