Problem/Motivation

\Drupal\user\Plugin\LanguageNegotiation\LanguageNegotiationUserAdmin::isAdminPath() run a full router match and calls $this->pathProcessorManager->processInbound(). If path processors are not re-entrant then this can break routing. Unfortunately \Drupal\system\PathProcessor\PathProcessorFiles::processInbound() is not re-entrant so this language negotiation can break private files.

Proposed resolution

Remaining tasks

User interface changes

None

API changes

Hopefully none

Data model changes

None

Original issue summary

I successfully added a 'document file' element to a field file type and it's upload functionality works as well: the files get uploaded in the specified 'private files' location and is available on the files system (+ readable by the http server):
/var/www/html/webapp.local//files-private/2017-01/doc.pdf

Now, while trying to access the file via Drupal with the following url, the request respond with a 404 not found:
http://webapp.local/system/files/2017-01/doc.pdf

Symfony\Component\HttpKernel\Exception\NotFoundHttpException is reached with the message 'No route found for "GET /system/files/webform/media_accreditation_form/7/doc.pdf".

Why the '/system/files/{filepath}' route and its Drupal\system\FileDownloadController::download controller isn't reach is unclear to me.

But I found that Drupal\Core\Routing\RouteProvider::getRoutesByPath($path) is called with $path = '/system/files/2017-01/junge_koch_article_-_final.pdf' but return no route at all. The reason seems that there is no entry in the router table matching for path '/system/files/2017-01/junge_koch_article_-_final.pdf'.

Now, is it a bug in Drupal\Core\Routing\RouteProvider::getRoutesByPath($path) or there is something wrong while adding the '/system/files/2017-01/junge_koch_article_-_final.pdf' path in the router table?

Comments

drikc created an issue. See original summary.

drikc’s picture

Issue summary: View changes
drikc’s picture

Following up; it appears that in Drupal\system\PathProcessor\PathProcessorFiles::processInbound() $request->query->has('file') return TRUE which prevent to reduce and produce the correct path:

class PathProcessorFiles implements InboundPathProcessorInterface {

  /**
   * {@inheritdoc}
   */
  public function processInbound($path, Request $request) {
    if (strpos($path, '/system/files/') === 0 && !$request->query->has('file')) {
      $file_path = preg_replace('|^\/system\/files\/|', '', $path);
      $request->query->set('file', $file_path);
      return '/system/files';
    }
    return $path;
  }

}
ramreddy.kancherla’s picture

Hi @drikc,
Did you solve this problem. I am also getting 404 page not found when try to access private files for all users include admin user also.
Please let me know do you have any suggestions regarding this problem.

Thank you.

drikc’s picture

Unfortunately not (couldn't find more time on that at this moment). Moreover, I couldn't reproduce this on a genuine 8.2.5 installation...

alexpott’s picture

Version: 8.2.5 » 8.4.x-dev
Component: file system » language.module
Category: Support request » Bug report
Priority: Normal » Major
Issue summary: View changes

Discussed something very similar with @esolitos in IRC we discovered a problem with \Drupal\user\Plugin\LanguageNegotiation\LanguageNegotiationUserAdmin::isAdminPath(). Breaking private files is at least a major.

alexpott’s picture

The reason why the path processor is not re-entrant is \Drupal\image\PathProcessor\PathProcessorImageStyles::processInbound()

alexpott’s picture

Status: Active » Needs review
StatusFileSize
new2.98 KB

Here's a failing test.

Status: Needs review » Needs work

The last submitted patch, 8: 2846379-8.patch, failed testing.

alexpott’s picture

Version: 8.4.x-dev » 8.3.x-dev
Status: Needs work » Needs review
StatusFileSize
new1.44 KB
new4.41 KB

Cloning the request seems the simplest solution.

This is a straight up bug and should be fixed in 8.3.x

dawehner’s picture

+++ b/core/modules/user/src/Plugin/LanguageNegotiation/LanguageNegotiationUserAdmin.php
@@ -127,10 +127,14 @@ protected function isAdminPath(Request $request) {
+          // Some inbound path processors make changes to the request. Make a
+          // copy as we're not actually routing the request so we do not want to
+          // make changes.
+          $cloned_request = clone $request;

This is something which really should be kept in mind all the time. Good to know!

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

This looks great for me!

alexpott’s picture

Status: Reviewed & tested by the community » Needs work

Thinking about other places that call ->processInbound()

\Drupal\Core\Path\PathValidator::getPathAttributes()
This one is okay because earlier it does:

    $request = Request::create('/' . $path);
    $attributes = $this->getPathAttributes($path, $request, $access_check);

The same pattern is in \Drupal\system\PathBasedBreadcrumbBuilder::getRequestForPath() too

I think we probably should update the documentation on \Drupal\Core\PathProcessor\InboundPathProcessorInterface::processInbound() to make this consideration known. So that people can discover this gotcha.

alexpott’s picture

Title: 404 not found while accessing private document file » Processing paths in LanguageNegotiationUserAdmin causes 404s while accessing private document file
Status: Needs work » Needs review
StatusFileSize
new1.12 KB
new5.53 KB

Better docs.

ramreddy.kancherla’s picture

Status: Needs review » Reviewed & tested by the community

Thanks @alexpott. #10 patch worked perfectly for me.

esolitos’s picture

Hi, I wanted to work on this yesterday, but i was sick as hell, provided patch looks and works great, tested on 8.2.6, thanks alexpott for the quick work!

drikc’s picture

Damn, I sweated on this; at least I wasn't on a complet wrong path...

Thank you, tested on 8.2.6...

esolitos’s picture

@drikc
Don't worry you were not the only one, I spent 1 day figuring out the issue and then in half day (mostly) thanks to alexpott (and the other peeps in #drupal-contribute) we were able to narrow it down to the LanguageNegotiationUserAdmin::isAdminPath() issue.

swentel’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/lib/Drupal/Core/PathProcessor/InboundPathProcessorInterface.php
@@ -12,10 +12,17 @@
+   *   The HttpRequest object representing the request to process. Note, if this
+   *   method is being called via the path_processor_manager service and is not
+   *   part routing, the request then the current request object must be cloned

The sentence starting at 'Note, if this ..' is a bit weird if you read this out load.

esolitos’s picture

Status: Needs work » Reviewed & tested by the community

As far as i know this works in english, it's quite common writing "Please note, foo bar baz".

swentel’s picture

Well, it's more the second part: "and is not part routing, the request then the current request object must be" - the 'then' doesn't seem to belong there at all ..

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 14: 2846379-14.patch, failed testing.

esolitos’s picture

Status: Needs work » Needs review
StatusFileSize
new5.5 KB

@swentel I realized that after a while.
Here's a rerolled patch on 8.3.x with the comment change and updated to the latest commit.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

casey’s picture

StatusFileSize
new5.57 KB

Reroll for 8.4

mpp’s picture

There's another side effect due to the next line in LanguageNegotiationUserAdmin:

$attributes = $this->router->match($path);

When visiting a node translation (e.g. /node/fr/123) the content appears in the site's default language.
When commenting it out and returning TRUE or FALSE instead, the proper language is shown: french when FALSE, the users admin language when TRUE.

See match() in AccessAwareRouter:

  /**
   * {@inheritdoc}
   *
   * @throws \Symfony\Component\HttpKernel\Exception\AccessDeniedHttpException
   *   Thrown when access checking failed.
   */
  public function match($pathinfo) {
    return $this->matchRequest(Request::create($pathinfo));
  }

The request object is created statically so I assume something's still going wrong within matchRequest():

  /**
   * {@inheritdoc}
   *
   * @throws \Symfony\Component\HttpKernel\Exception\AccessDeniedHttpException
   *   Thrown when access checking failed.
   */
  public function matchRequest(Request $request) {
    $parameters = $this->router->matchRequest($request);
    $request->attributes->add($parameters);
    $this->checkAccess($request);
    // We can not return $parameters because the access check can change the
    // request attributes.
    return $request->attributes->all();
  }
catch’s picture

Title: Processing paths in LanguageNegotiationUserAdmin causes 404s while accessing private document file » Processing paths in LanguageNegotiationUserAdmin causes 404s while accessing private files
Related issues: +#2802403: Combination of language negotiation and path aliasing can cause a corrupted route cache, 404s
catch’s picture

Priority: Major » Critical

Bumping this to critical alongside #2802403: Combination of language negotiation and path aliasing can cause a corrupted route cache, 404s - visitor-facing 404s make the site unusable and there's no workaround.

alexpott’s picture

StatusFileSize
new842 bytes
new5.51 KB

re #26 / @mpp that sounds like a separate issue.

Cut out an unnecessary then.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

flocondetoile’s picture

Recently, after updated Drupal Core to 8.5 all my private files became unreachable randomly. Really disturbing.
The cause seems to be that the route, at some point, /system/files/DOCUMENT was not rewrite by the PathProcessorFiles to /system/files (or may be the original path was added after the PathProcessorFiles ran on the path)

Tried patch #29 which fix this main critical issue. But this bugs occurs just after updating to 8.5 so I have some doubt if this is the same issue.

rvanderh1’s picture

I recently updated to Drupal Core 8.5.4, and now links on my site that are in the form of http://www.somesite.com/index?sometoken=XXXX are having the "?sometoken=XXXX" part of the URL stripped out. The result can be failure to be recognized as an authorized user at the somesite.com site. Is this problem related to this thread? It seems that the problem occurs for URLs that are using the Redirect module, but the problem began occurring right after I updated Core to 8.5.4.

sealionking’s picture

after applied the patch 25 then the patch 29, the private files read ok

thanks for you guys!

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

esolitos’s picture

Status: Needs review » Reviewed & tested by the community

Patch from #29 should be the way to go.

  • larowlan committed bfed684 on 8.7.x
    Issue #2846379 by alexpott, esolitos, casey: Processing paths in...

  • larowlan committed 8480003 on 8.6.x
    Issue #2846379 by alexpott, esolitos, casey: Processing paths in...
larowlan’s picture

Status: Reviewed & tested by the community » Fixed

Fixed on commit

+++ b/core/lib/Drupal/Core/PathProcessor/InboundPathProcessorInterface.php
@@ -12,10 +12,17 @@
+   * other that the current request.

%s/that/than

Committed as bfed684 and pushed to 8.7.x.

Cherry-picked as 8480003 and pushed to 8.6.x.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.