Don't bother with file copies during build, keep file paths on the build and deploy from the original file

Trying to figure out the big picture here.

As far as I can tell its not quite as simple as moving body of the copy operation to a new queue, as the caller relies on the copied file, and recursively looks for more resources within the copied file.

Perhaps \Drupal\preview_site\Plugin\PreviewSite\Generate\TomeGenerator::generateBuildForPath should be split at the $remaining_assets = $this->static->exportPaths($invoke_paths); point.

Where \Drupal\tome_static\StaticGenerator::exportPaths becomes a variable/dynamic size batch recursively copying files.

Testing this locally it's deleting css and js files that it copies over.

We've been running this in prod for some time and it seems to be working well.

This is actually causing massive issues with #3230078: Add support for Javascript modules - if a page contains a javascript file that imports chunks this patch causes some sort of recursion on those chunks adding thousands of files to the export and taking a long time to complete (over an hour up from 5 minutes).

EDIT: The chunked JS files also don't make it to S3

--- a/modules/preview_site_s3/src/Plugin/PreviewSite/Deploy/S3.php
+++ b/modules/preview_site_s3/src/Plugin/PreviewSite/Deploy/S3.php

@@ -269,4 +269,32 @@ class S3 extends DeployPluginBase implements ContainerFactoryPluginInterface {
+  public function deployFilePath(PreviewSiteBuildInterface $build, string $path): void {

This is almost identical to deployArtifact

Can we reafctor that so it calls this with FileHelper::getFilePathWithoutSchema($file, $build) as the path?

Otherwise looking pretty good, great idea!

I mentioned this to @larowlan, but this has also solved OOM issues with the preview site build form - during validation it was validating the entity reference for all artifacts.

Hello,

I'm reopening this issue, because I'm experiencing a PHP Fatal Error.

Here's a part of the stack trace:

 Error: Call to undefined method Drupal\preview_site\Generate\TomeStaticExtension::getJavascriptModules() in /opt/drupal/web/modules/contrib/preview_site/src/Generate/TomeStaticExtension.php:185 Stack trace: 
#0 /opt/drupal/web/modules/contrib/preview_site/src/Plugin/PreviewSite/Generate/TomeGenerator.php(207): Drupal\preview_site\Generate\TomeStaticExtension->exportPaths(Array) 
#1 /opt/drupal/web/modules/contrib/preview_site/src/Plugin/PreviewSite/Generate/TomeGenerator.php(129): Drupal\preview_site\Plugin\PreviewSite\Generate\TomeGenerator->generateBuildForPath(Object(Drupal\preview_site\Entity\PreviewSiteBuild), '_entity:node:en...', 'http://default/...', Object(Drupal\Core\Queue\DatabaseQueue)) 
#2 /opt/drupal/web/modules/contrib/preview_site/src/Plugin/QueueWorker/Generate.php(115): Drupal\preview_site\Plugin\PreviewSite\Generate\TomeGenerator->generateBuildForItem(Object(Drupal\preview_site\Entity\PreviewSiteBuild), Object(Drupal\dynamic_entity_reference\Plugin\Field\FieldType\DynamicEntityReferenceItem), 'http://default/...', Object(Drupal\Core\Queue\DatabaseQueue)) 
#3 /opt/drupal/web/modules/contrib/preview_site/src/PreviewSiteBuilder.php(180): Drupal\preview_site\Plugin\QueueWorker\Generate->processItem(0) 
#4 /opt/drupal/web/modules/contrib/preview_site/src/PreviewSiteBuilder.php(120): Drupal\preview_site\PreviewSiteBuilder->processQueueItem('preview_site_ge...') 
#5 /opt/drupal/web/modules/contrib/preview_site/src/PreviewSiteBuilder.php(242): Drupal\preview_site\PreviewSiteBuilder->processSiteGeneration(Object(Drupal\preview_site\Entity\PreviewSiteBuild)) 
#6 /opt/drupal/packages/languagewire_tmgmt_connector/src/Adapter/PreviewSite/PreviewSiteBuilder.php(66): Drupal\preview_site\PreviewSiteBuilder::operationProcessGenerate(303, Array)
(...)

I've taken a look at the preview_site module source code, and I indeed can find the "getJavascriptModules()" method definition nowhere in there.

This issue was introduced in the release 1.1.5 of the module.

PS. This is my first time adding a comment in Drupal's issue tracker. Please forgive me if I should have opened a new issue instead.

You're right, that function is provided by https://www.drupal.org/project/tome/issues/3230078

I'm surprised tests are passing...

@acbramley There isn't test coverage for it. Do we remove the patch for now and roll a new release until the dependency is merged in?

I've reverted the commit and published 1.1.6

Great @acbramley!

Awesome! RTBC assuming green :)