Webform assumes the /tmp directory is always the same, but if there are multiple servers, each may have its own /tmp directory

Nothing gets logged to the watchdog. Is there a "debug" mode I can use?

I do have a bit more info about this one.

* I have 1038 total submissions for the month of May.
* I have downloaded the submissions as CSV files several times, and the resulting file is always slightly different; it never actually has 1038 submissions though, it seems to get stuck once in a while.

$ cd ~/Downloads/
$ wc -l webform_8226*
     660 webform_8226-2 copy.csv
     562 webform_8226-2.csv
     724 webform_8226-3.csv
    1946 webform_8226-4.csv
     562 webform_8226-downloaded-automatically copy.csv
     660 webform_8226.csv
    5114 total

As you can see above, the result is inconsistent. It's not clear how to actually consistently download the exact number of submissions and have the same results every time.

Some more information on this:

* Using a database dump from production but developing locally, this works fine.
* On Acquia cloud, this breaks, however no error message is logged.

Thanks,

I reduced the batch size from 500 to 100 and I'm still getting the problem on Acquia cloud production (not staging). And no PHP errors.

I then set the batch size to 1, and the problem is still occurring.

I am having the same problem with Safari and Chrome.

I have opened a ticket with Acquia (number 702279) to see if they have input. I'll leave at "Postpone" until I get an answer from Acquia.

a specific submission is causing a fatal error

I tried this with two completely different forms and the problem always occurs, so this is not lined to one submission.

I might be able to replicate the issue.

I cannot yet replicate it myself, this only happens on my production Acquia environment, no other environment is affected. If I manage to replicate it locally, I will package a sanitized version of the database.

Here is an interesting fact about this bug:

On my production environment where this bug occurs, on Safari on Mac, downloading an export several times in a row causes exports with different sizes, but with names which include an incremented number, as follows.

$ ls -lah *.csv
-rw-r--r--@ 1 albert  staff   219K 19 Jun 13:56 webform_50386-2.csv
-rw-r--r--@ 1 albert  staff    72K 19 Jun 13:56 webform_50386-3.csv
-rw-r--r--@ 1 albert  staff     0B 19 Jun 13:56 webform_50386-4.csv
-rw-r--r--@ 1 albert  staff    65K 19 Jun 13:57 webform_50386-5.csv
-rw-r--r--@ 1 albert  staff   333K 19 Jun 13:57 webform_50386-6.csv
-rw-r--r--@ 1 albert  staff   141K 19 Jun 13:58 webform_50386-7.csv
-rw-r--r--@ 1 albert  staff    56K 19 Jun 13:55 webform_50386.csv

On my stage environment where this bug does not occur, on Safari on Mac, downloading an export several times in a row results in there being exactly one download (which has the right number of rows): it seems that the system does not attempt to save a second version of the export if an existing version has been saved. So, no matter how many times I run the export, I get:

$ ls -lah *.csv
-rw-r--r--@ 1 albert  staff   291K 19 Jun 14:01 webform_50386.csv

Thanks @jrockowitz

I have confirmed that there is no special character in submissions or database issue, as an exact clone of the failing database works on another environment.

However, I will look into the /tmp directory issue, thanks for that idea!

Here is another interesting fact:

On an environment that works, I can download 972 submissions, and end up with a file with 973 lines (which is expected):

$ wc -l ~/Downloads/*.csv
973 /Users/albert/Downloads/webform_50386.csv

On an environment which does not work, if you download the same file often enough, you can end up with a file having number of lines which is _higher_ than actual number of submissions, in this case webform_50386-9.csv has 1032 lines but there are only a total of 972 submissions:

$ wc -l ~/Downloads/*.csv
442 /Users/albert/Downloads/webform_50386-2.csv
125 /Users/albert/Downloads/webform_50386-3.csv
809 /Users/albert/Downloads/webform_50386-4.csv
948 /Users/albert/Downloads/webform_50386-5.csv
402 /Users/albert/Downloads/webform_50386-6.csv
520 /Users/albert/Downloads/webform_50386-7.csv
361 /Users/albert/Downloads/webform_50386-8.csv
1032 /Users/albert/Downloads/webform_50386-9.csv
168 /Users/albert/Downloads/webform_50386.csv
973 /Users/albert/Downloads/uncorrupted-data-webform_50386.csv
5780 total

In this case the corrupted /Users/albert/Downloads/webform_50386-9.csv has _more_ lines than /Users/albert/Downloads/uncorrupted-data-webform_50386.csv, which is the uncorrupted export.

I checked the diff between the corrupted and uncorrupted exports, and it turns out several records are missing in the corrupted export, and other records are duplicated, which accounts for the fact that the corrupted file is larger than the uncorrupted file.

Here are the contents of my /tmp directory on Acquia:

On the broken environment:

$ ls -lah /tmp|grep webform
-rw-r--r--  1 me       me       9.2K Apr 30 18:25 webform_a3zPX8
-rw-------  1 me       me       2.3M Apr 30 20:25 webform_ecqX0Q
-rw-r--r--  1 me       me        79K May 21 16:35 webform_eRvki6
-rw-r--r--  1 me       me       8.2K May  3 19:17 webform_zzZs8R

On the working environment:

$ls -lah /tmp|grep webform

(it is empty)

I deleted all webform_* files (which were all old files from an old Drupal 7 environment) from the /tmp directory in the broken environment, and, still, getting the same behaviour.

Here is some more information on this: Acquia seems to store different production sites on the same servers, so I have another site I manage, this one a Drupal 7 site, on the same production server as the affected Drupal 8 site.

I also have Webform D7 on that site, and I am experiencing similar behavior when downloading exported Webform Submissions: downloading a submission on the Drupal 7 on that server systematically results in zero-byte downloads, but on the staging environment of that Drupal 7 site everything works as expected.

I am still waiting for a response from Acquia on this and will update this issue accordingly.

@jrockowitz thanks to your help, I have found the issue:

It sounds like Acquia is having issues with the tmp file directory.

is exactly right. Changing the /tmp directory to something else completely fixed this issue.

Here is how I found this:

drush webform-export worked perfectly. However, deleting "webform_*" files from the /tmp directory did not fix the issue.

I then changed the \Drupal\webform\Controller\WebformResultsExportController::downloadFile to this:

  public function downloadFile($file_path, $download = TRUE) {
    clearstatcache(TRUE, $file_path);
    $response = new BinaryFileResponse($file_path, 200, [], FALSE, $download ? 'attachment' : 'inline');
    // $response->deleteFileAfterSend(TRUE);
    \Drupal::logger('WFTEST')->notice('Serving file ' . $file_path);
    return $response;
  }

just to see what happens.

With that code in place, I deleted everything in my /tmp folder and my downloads folder, then exported a webform with 972 submissions, then I performed a download a first time.

This created a file with 801 lines in my /tmp folder:

$ wc -l webform_50386.csv
801 webform_50386.csv

I can then repeatedly download this file by clicking "Download the file here" (because it is not deleted after it is downloaded, as per my above changes).

The result is all over the map:

wc -l webform_*
     473 webform_50386-2.csv
     473 webform_50386-3.csv
     473 webform_50386-4.csv
    1201 webform_50386-5.csv
     473 webform_50386-6.csv
    1401 webform_50386-7.csv
     901 webform_50386.csv
    5395 total

This told me that, definitely there is a problem with /tmp folder.

Because this issue has nothing to do with Webform, I'll close it.

Thanks again for your kind assistance!

Thanks for the resource. I'm using php 7.2. And, yes, changing the /tmp directory makes everything work perfectly!

In fact #2221651: Make temporary export file path configurable is the exact issue I am having.

This got me thinking: as a lot of webservers are moving toward multi-node setups, multiple containers such as Kubernetes, and so on, might it benefit Webform to no longer assume that /tmp is always accessible and the same between requests? See also https://stackoverflow.com/questions/50955591

Hi, Acquia has gotten back to me, suggesting I use the technique at #2221651: Make temporary export file path configurable. On this page, they say:

The Webform module can have issues similar to what you may experience with Plupload. Versions 7.x-4.x or greater and 8.x-5.x or greater of the Webform module include the webform_export_path variable, which (depending on your Drupal version) you can set in your settings.php file

However, that functionality seems to have been removed in commit c938264 of Webform:

$ git diff 17a8566 c938264 | grep webform_export_path --before-context=5 --after-context=5
-
-/**
- * Returns a temporary export filename.
- */
-function _webform_export_tempname() {
-  return drupal_tempnam(variable_get('webform_export_path', 'temporary://'), 'webform_');
-}
-
-/**
- * Generate a Excel-readable CSV file containing all submissions for a Webform.
- *
$ git diff 17a8566 c938264 | grep _webform_export_tempname --before-context=5 --after-context=5
-
-  function post_process(&$results) {
-    // Our download file is currently a single XML sheet file. We need to add
-    // the peripheral XML files to make this into a XLSX directory, then zip it.
-    $file_uri = $results['file_name'];
-    $zip_uri = _webform_export_tempname();
-
-    // ZipArchive does not support stream wrappers, convert to filesystem path.
-    $zip_filepath = drupal_realpath($zip_uri);
-    $file_filepath = drupal_realpath($file_uri);
-    $zip = new ZipArchive;
--
--
-  // Determine an appropriate batch size based on the form and server specs.
-  if (!isset($options['range']['batch_size'])) {
-    $options['range']['batch_size'] = webform_export_batch_size($node);
-  }
-
-  $options['file_name'] = _webform_export_tempname();
-
-  // Set up a batch to export the results.
-  $batch = webform_results_export_batch($node, $format, $options);
-  batch_set($batch);
-}
--
--
-}
-
-/**
- * Returns a temporary export filename.
- */
-function _webform_export_tempname() {
-  return drupal_tempnam(variable_get('webform_export_path', 'temporary://'), 'webform_');
-}
-
-/**
- * Generate a Excel-readable CSV file containing all submissions for a Webform.
--
--
-  $options['range'] += $defaults['range'];
-
-  // Open a new Webform exporter object.
-  $exporter = webform_export_create_handler($format, $options);
-
-  $file_name = _webform_export_tempname();
-  $handle = fopen($file_name, 'w');
-  if (!$handle) {
-    return;
-  }
-

I informed Acquia of your suggestion to always direct same-source traffic to the same server, however my understanding is that as of right now all Acquia "enterprise" users having more than one webserver will be unable to export any submission data.

Thanks, much appreciated!

Applying this patch and running drush updb -y works fine.

Changing the settings via the GUI at /admin/structure/webform/config/exporters and exporting the settings yields:

--- a/drupal-server/config/webform.settings.yml
+++ b/drupal-server/config/webform.settings.yml
@@ -82,7 +82,7 @@ export:
   delimiter: ','
   excel: false
   excluded_exporters: {  }
-  file_name: 'submission-[webform_submission:serial]'
+  temp_directory: /this/directory/exists
 batch:
   default_batch_export_size: 100
   default_batch_update_size: 100

(I'm not sure why it's removing "file_name" here... maybe that's unrelated?)

However I ran into the following problems:

(1) if the directory does not exist we get a fatal error:

The website encountered an unexpected error. Please try again later.
Error: Call to a member function mkdir() on null in Drupal\webform\Form\AdminConfig\WebformAdminConfigExportersForm->validateForm() (line 128 of modules/webform/src/Form/AdminConfig/WebformAdminConfigExportersForm.php).
Drupal\webform\Form\AdminConfig\WebformAdminConfigExportersForm->validateForm(Array, Object)
call_user_func_array(Array, Array) (Line: 83)

(2) if the directory exists but does not have the correct permissions I get:

The website encountered an unexpected error. Please try again later.
Error: Call to a member function chmod() on null in Drupal\webform\Form\AdminConfig\WebformAdminConfigExportersForm->validateForm() (line 131 of modules/webform/src/Form/AdminConfig/WebformAdminConfigExportersForm.php).

(3) if the directory does exist and has the correct permissions (777 in my case) and I set it using the GUI, the system still uses "/tmp", not the new directory.

No longer getting fatal errors, however the system is still, in my case, using the /tmp directory.

I noticed that you are making the following change to ./src/WebformSubmissionExporter.php:

   public function getFileTempDirectory() {
-    return file_directory_temp();
+    return $this->configFactory->get('webform.settings')->get('export.temp_directory') ?: file_directory_temp();
   }

However there is another file which contains getFileTempDirectory(), which has not been changed: ./src/Plugin/WebformExporterBase.php

  /**
   * {@inheritdoc}
   */
  public function getFileTempDirectory() {
    return file_directory_temp();
  }

Perhaps making the same change to the above method will cause the configured temporary directory to be used.

Would it also be useful to add some more context to the help text, otherwise users might not understand why we're not just using the /tmp directory. Instead of

A local file system path where temporary export files will be stored. This directory should not be accessible over the web.

something like:

A local file system path where temporary export files will be stored. This directory should be persistent between requests and should not be accessible over the web.

This works perfectly in my local tests, thanks! I will deploy this to the production site tomorrow and have more input.

Thanks for taking this on @jrockowitz, I think this will help all Acquia users who are exporting submissions.

I just deployed this to my production and it works exactly as expected: if the config is not set, the default /tmp folder is used, and if the config is set, the custom directory is used.

OK, here it is, https://www.drupal.org/node/2982200

For reference, there might be a similar issue on the feeds module, see #3010908: Articles are not being imported to production Acquia site, possibly because /tmp folder is not on the same container between requests.

Related to: #3170504: On high-availabilities setups with multiple containers, the /tmp directory might differ between calls, make the error message more descriptive