Email link to file includes '_sid_' rather than actual sid value

I am not able to replicate this issue using the attached webform.

Please provide a webform that can be used to replicate this issue.

I am having the same issue. It worked well until today.

I have tested the attached webform and it also exhibits the problem on my live server. My test server is working fine, both with my forms as well as the attached form.

It sends an email with "_sid_" in the file URL instead of the actual sid. The weird thing is that if I go to the results page and resend the submission, then the URL is correct. It only happens during the initial email upon submitting the form.

I will be going over all the changes that have happened over the last few days and try to figure out what could be causing that issue on the live server but not the test server (they both have the same module versions installed).

The uploaded file's _sid_ is replaced via \Drupal\webform\Plugin\WebformElement\WebformManagedFileBase::postSave which calls \Drupal\webform\Plugin\WebformElement\WebformManagedFileBase::addFiles

The email is built and sent via \Drupal\webform\Plugin\WebformHandler\EmailWebformHandler::postSave

Finally, WebformManagedFileBase::postSave is called before EmailWebformHandler::postSave
@see \Drupal\webform\WebformSubmissionStorage::doPostSave

I am not seeing any explanation for this regression and I can't reproduce it.

What version of Drupal and Webform are you using?

The ticket's version was currently set 8.x-5.2 but I am going to assume the issue is in latest dev release.

The only change in the 5.22 release that could possibly cause this issue is #3173856: Sort handlers with the same weight by the handlers_id but this change has no impact on the triggering of the ::postSave callbacks.

I am having the issue on Drupal 8.9.6 -8.9.7, using Webform 5.22, I tried Webform 6, same thing. Since the issue does not exist on my test server, there must be a setup difference between the two that I have not documented.

Thank you for explaining the process, I will do some debugging and see if I can figure out what is going on.

I don't think this santizing the file name is related to this issue.

#4 documents that the file should uploaded, sanitized, and renamed before email is sent.

I am bumping the priority to Major.

I have been chasing my tail trying to figure out what might be going on. All I have found so far is that the files are getting moved and renamed correctly, but when the tokens are being replaced and the file entity is loaded, it contains the old info before the move.

If for example if at the very end of \Drupal\webform\Plugin\WebformElement\WebformManagedFileBase::addFiles, I reload the file (using $this->entityTypeManager->getStorage('file')->load($fids[0])) it still outputs the URI/name before the changes were made.

If I do a database query at this same spot, it pulls the correct information, so I figured maybe it was a problem with the static cache not getting cleared? But it made no difference when I forced it to clear.

If I go to the submission page and resend the email, then the link to the file in the email is working properly, so at that stage the token replace function is able to load the file entity with the correct info.

I am not sure what could be causing this at this stage, considering I had not changed or upgraded anything when the error started happening.

Sorry, didn't mean to change the priority.

@skyhawk669 Does the issue go away if you download grade from 5.22 to 5.21?

@jrockowitz I currently am on 6.0.0-alpha20. I was on 5.22 originally, but I had upgraded just in case that fixed it, which it did not. Is it possible to downgrade back down to 5.21?

I have not been able to reproduce the problem on any of my test servers, so I assume the issue must be outside of Webform, or some sort of interaction with another system. I do have memcached running on production but not on my test server, I will go ahead an try this.

OK, after installing memcache 2.2 on my test server I can reproduce the problem! @namscott are you using any sort of caching as well?

I use memcached and varnish on production as well. I enabled memcached on my test server and it now exhibits the same issue. For me, the problem started after upgrading to drupal/memcache 2.2. Downgrading to 2.1 solved the issue on my server. This bug is being caused by this change: https://www.drupal.org/project/memcache/issues/2996615

So it looks like something that needs to be fixed by the memcache module maintainers, it is not a Webform issue.

I can was able to replicate this issue when enabling and configuring Memcache 2.2.

I am not sure if the Webform module can provide a workaround.

We could warn people using Memcache 2.2 and assume the issue will solved in Memcache 2.3

Resetting the single file cache did not work for me but resetting the entire file entities cache does solve the problem but this is not a great solution.

@skyhawk669 Thanks for figuring out the source of this issue.

Attached is a slightly better patch but using \Drupal::entityTypeManager()->getStorage('file')->resetCache($fids); does not work as expected.

Since over 8000 sites have already upgrade to Memcache 2.2. The only thing that I think we can do is warn users about this major regression via Drupal's Status report (/admin/reports/status).

Adding a warning is our only option, so I committed the patch.

Otherwise this issue needs to be addressed via #3176519: Uploaded files not linking correctly in emails sent through Webform module after upgrading to memcache 2.2

FWIW, the following patch fixed the issue for a client site using CloudFront instead of Memcache.

The patch seems like the right solution. Let's see if all tests are still passing.

Attached patch changes

$this->entityTypeManager->getStorage('file')->resetCache();
    return $this->entityTypeManager->getStorage('file')->loadMultiple((array) $value);

to

    $fids = (array) $value;
    $this->entityTypeManager->getStorage('file')->resetCache($fids);
    return $this->entityTypeManager->getStorage('file')->loadMultiple($fids);

Yeah, that is a good idea. We need to revert #26 so NW for that. Do you think we can add some tests for this?

I am not sure how much test coverage we can add and maybe it is not absolutely needed.

The patch makes sense to me.

Yeah, I had the same idea. I think #26 can be removed on commit so setting it to RTBC.

Added proposed resolution to IS.

I get an error when applying #31 to version 6.0
Could not apply patch! Skipping. The error was: Cannot apply patch https://www.drupal.org/files/issues/2021-01-14/3175525-31.patch

double post error sorry

I'm still experiencing this issue, even after the patch.

Drupal: 8.9.13
Memcache module: 2.2
Webform: 8.x-5.12

I wondered if the patch + other updates in 8.x-5.x might have fixed it, but even after removing the patch and upgrading to 8.x-5.24 (which contains the patched code), I am still experiencing the _sid_ problem. Not only is this affecting links to files, but also attachments via Mail System + Swiftmailer.

FWIW, here are the bins I have configured for memcache:

$settings['cache']['bins']['bootstrap'] = 'cache.backend.memcache';
$settings['cache']['bins']['discovery'] = 'cache.backend.memcache';
$settings['cache']['bins']['config'] = 'cache.backend.memcache';
$settings['cache']['bins']['default'] = 'cache.backend.memcache';
$settings['cache']['bins']['render'] = 'cache.backend.memcache';
$settings['cache']['bins']['dynamic_page_cache'] = 'cache.backend.memcache';
$settings['cache']['default'] = 'cache.backend.memcache';

We also still have the same issue. The upload is working, but the issue remains in the mail template. Downgrading to memcache 2.1 solves the issue.

Yes, same here.
It's still happening

Drupal: 8.9.13
Memcache module: 2.2
Webform: 8.x-5.24

Downgrade to memcache 2.1 fix the issue.

Same here. I had to downgrade the memchace module. The issue should be reopened.

Yes still happening on Drupal 9.1.4, webform 6.0.1 and Memache 2.2. I'm not familiar with this functionality but in debugging it seems like WebformManagedFileBase::getFileDestinationUri replaces the _sid_ string in the destination correctly so I'm confused why its still in the email body.

The upgrade failed for us. We noticed the patch that we were using differs to what was applied to the module. This patch fixes it for us.

If $this->entityTypeManager->getStorage('file')->resetCache(); is called the file entity cache will be reset with every file upload which could impact a website's performance.

I think this issue was addressed via https://www.drupal.org/project/memcache/releases/8.x-2.3

We don't use memcache. We use redis if that helps.
We have not dived into why this is still an issue for us, but for now that patch keeps us running.

Updating memcache to 8.2.3 fixed it for us.

new memcache version fixes issue

I rerolled the patch to apply to 6.2