Problem/Motivation

Issue #88183: Relative URLs in feeds should be converted to absolute ones added a response subscriber (RssResponseRelativeUrlFilter) that converts relative URLs to absolute ones. It does this for any response that has the 'application/rss' Content-Type HTTP header.

However, that Content-Type HTTP header for feeds coming from views is set way down in template_preprocess_views_view_rss(), which means that information is not cached. Once such a feed (with its relative URLs) is in the render cache, that preprocess hook will not be called for subsequent requests, and so the default Content-Type header will be used, of 'text/html'. This is both wrong, and stops that RssResponseRelativeUrlFilter service from changing the URLs.

Steps to reproduce:

1) Build an RSS feed in views using the Feed display, or use an existing one (e.g. /rss.xml)
2) Ensure there is a 'real' render cache in place, not the null one often used for development.
3) View the feed
4) View the feed again to ensure you're getting the cached version this time.

The 'Content-Type' HTTP header will be 'text/html', but it should be 'application/xml'. Plus any relative URLs that would have been converted to be absolute when viewing the feed for the first time, before it was cached, will not have been converted.

Proposed resolution

The response object is deliberately passed down from the Feed display plugin through to the theme layer, to allow the HTTP header to be set. Cache it, so that the headers can then be used for subsequent requests.

Alternative possible resolutions are mentioned in comment 2.

Remaining tasks

Write tests - perhaps just extend those written for #88183: Relative URLs in feeds should be converted to absolute ones? Review the solution, and confirm it is the correct one.

User interface changes

Links will correctly be absolute, and the Content-Type HTTP header will correctly be application/rss.

API changes

None, as far as I can tell.

Data model changes

None, as far as I can tell.

Issue fork drupal-2959134

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

james.williams created an issue. See original summary.

james.williams’s picture

Issue summary: View changes
Issue tags: +Needs tests
StatusFileSize
new1019 bytes

The response object is deliberately passed down from the Feed display plugin through to the theme layer, to allow the HTTP header to be set. Cache it, so that the headers can then be used for subsequent requests.

Here's a patch to implement this. It will need tests adding, though perhaps we can just add to those written for #88183: Relative URLs in feeds should be converted to absolute ones, to use a render cache and view the feed twice?

I've added steps to reproduce, to the original issue description. I had some thoughts about possible alternative solutions too, in case anyone is interested:

As an alternative, we could remove the current code that sets the header from template_preprocess_views_view_rss(), and add it with a $build['#attached']['http_header'] in \Drupal\views\Plugin\views\style\Rss::render(), but then that still needs processing back up in \Drupal\views\Plugin\views\display\Feed::buildResponse() - possibly requiring a refactor of \Drupal\Core\Render\HtmlResponseAttachmentsProcessor::setHeaders() to avoid duplicating code. I could understand if that is preferred to caching the whole response object.

Another idea I had was to allow arrays to be set as values within the #cache_properties array, to allow caching just a part of a certain property - so then just the headers from $build['#response']->headers could be cached, rather than the whole object. But that would be a bigger API change. I believe the approach in comment 2's patch is the simplest instead, and that caching the whole response object isn't really a problem.

For reference, here's what gets serialized into a cache using a database backend for the response:

s:9:"#response";O:35:"Drupal\Core\Cache\CacheableResponse":7:{s:7:"headers";O:50:"Symfony\Component\HttpFoundation\ResponseHeaderBag":5:{s:23:"*computedCacheControl";a:2:{s:8:"no-cache";b:1;s:7:"private";b:1;}s:10:"*cookies";a:0:{}s:14:"*headerNames";a:3:{s:13:"cache-control";s:13:"Cache-Control";s:4:"date";s:4:"Date";s:12:"content-type";s:12:"Content-Type";}s:10:"*headers";a:3:{s:13:"cache-control";a:1:{i:0;s:17:"no-cache, private";}s:4:"date";a:1:{i:0;s:29:"Fri, 06 Apr 2018 11:44:29 GMT";}s:12:"content-type";a:1:{i:0;s:34:"application/rss+xml; charset=utf-8";}}s:15:"*cacheControl";a:0:{}}s:10:"*content";s:0:"";s:10:"*version";s:3:"1.0";s:13:"*statusCode";i:200;s:13:"*statusText";s:2:"OK";s:10:"*charset";N;s:23:"*cacheabilityMetadata";N;}

 

I don't believe that's a disastrous additional amount to add to the render cache for feeds, but it's enough to have made me think about alternatives.

james.williams’s picture

Status: Active » Needs review

Let's put the patch through the existing tests at least...

james.williams’s picture

Here's a really naive attempt at a test, let's see if it fails without the fix...

james.williams’s picture

Status: Needs review » Needs work

Hmm, either my test is wrong or the original 'bug' was actually due to something else in my environment...

james.williams’s picture

Assigned: james.williams » Unassigned
Status: Needs work » Needs review
Issue tags: -Needs tests
StatusFileSize
new1.54 KB
new2.54 KB

Here's a working test that should prove the bug - patches with & without the fix included.

For the previous test (comment 4), I hadn't realised that the testing profile enables the page_cache & dynamic_page_cache modules, which actually get around the bug, because the incorrect content is only cached at the render cache layer (the page cache layer is actually correct). So the bug is only present when the page cache module(s) are not installed. So I've created a new test method, rather than dumbly extending the existing one, that will uninstall the dynamic_page_cache module (and rebuild the container etc so that its event subscribers are no longer listening), then I've copied the minimum amount from the existing test that will prove the bug. I hope that's correct - I don't know whether uninstalling modules in this way within a test class is correct or not! (e.g. should the test clear up after itself by re-enabling the dynamic_page_cache module?)

borisson_’s picture

Status: Needs review » Needs work

The same method of uninstalling that module is also used in \Drupal\system\Tests\Session\SessionTest::testEmptyAnonymousSession. So that looks like it is used in other places.

I'm not sure if this is part of the API-First initiative, and if that means I should add the tag or not, but this looks very good - it has a failing test + a passing one.

I think that the test can be improved. In the initial issue you mention that the header is incorrect on the second time the page is viewed. We should also add a check to see that the response header is correct.

james.williams’s picture

Status: Needs work » Needs review
StatusFileSize
new2.74 KB
new1.75 KB
new904 bytes

Good idea. Patches updated :-)

Status: Needs review » Needs work
james.williams’s picture

Status: Needs work » Needs review

That was the expected result :-) The tests-only patch demonstrates the bug, the other patch fixes it.

borisson_’s picture

Status: Needs review » Reviewed & tested by the community

That looks great, the only remark I had was fixed.

lendude’s picture

Status: Reviewed & tested by the community » Needs work

Nice work!

Just some nits:

  1. +++ b/core/modules/node/tests/src/Functional/NodeRSSContentTest.php
    @@ -105,4 +105,39 @@ public function testUrlHandling() {
    +    $this->assertEquals($this->drupalGetHeader('Content-type'), 'application/rss+xml; charset=utf-8');
    ...
    +    $this->assertEquals($this->drupalGetHeader('Content-type'), 'application/rss+xml; charset=utf-8');
    

    the expected value comes first.

  2. +++ b/core/modules/node/tests/src/Functional/NodeRSSContentTest.php
    @@ -105,4 +105,39 @@ public function testUrlHandling() {
    +    $this->assertRaw(file_create_url('public://root-relative'), 'Root-relative URL is transformed to absolute.');
    ...
    +    $this->assertRaw(file_create_url('public://root-relative'), 'Root-relative URL is transformed to absolute.');
    

    assertRaw is deprecated and only takes one argument, the message arg isn't on the method.

james.williams’s picture

Status: Needs work » Needs review
StatusFileSize
new2.69 KB
new1.7 KB
new1.27 KB

Nitpicks addressed :-)

Status: Needs review » Needs work
james.williams’s picture

Status: Needs work » Needs review

That was the expected result :-)

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.9 was released on December 7, 2022 and is the final full bugfix release for the Drupal 9.4.x series. Drupal 9.4.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.5.x-dev branch from now on, and new development or disruptive changes should be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new184 bytes

The Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

plopesc’s picture

Version: 9.5.x-dev » 11.x-dev
Status: Needs work » Needs review
StatusFileSize
new3 KB

Created new version of the patch that applies against 10.1.x & 11.x.

Thanks!

plopesc’s picture

StatusFileSize
new2.99 KB

New patch version fixing indentation issues.

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +Needs Review Queue Initiative

Verified issue following the steps from the issue summary
Patch #26 seemed to solve it.

Ran the tests locally to make sure they failed and covered the issue

Failed asserting that two strings are equal.
Expected :'application/rss+xml; charset=utf-8'
Actual :'text/html; charset=UTF-8'

catch’s picture

Status: Reviewed & tested by the community » Needs work

I think we should try to consolidate the new test into the method directly above it (testUrlHandling) since it's nearly identical.

i.e. it looks like the node setup is the same, so could the addition look something like this?

+++ b/core/modules/node/tests/src/Functional/NodeRSSContentTest.php
@@ -122,4 +122,44 @@ public function testUrlHandling() {

+   // Tests root-relative URL for a feed in the render cache without page cache.

+    $this->container->get('module_installer')->uninstall(['dynamic_page_cache']);
+    $this->rebuildAll();

+
+    $this->drupalGet('rss.xml');
+    $this->assertEquals('application/rss+xml; charset=utf-8', $this->getSession()->getResponseHeader('Content-type'));
+    $this->assertSession()->responseContains($absolute_url);
+
+    $this->drupalGet('rss.xml');
+    $this->assertEquals('application/rss+xml; charset=utf-8', $this->getSession()->getResponseHeader('Content-type'));
+    $this->assertSession()->responseContains($absolute_url);
+  }
+
 }

Could also use an inline comment exlaining why the request and assertions are repeated.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

smustgrave’s picture

Status: Needs work » Postponed (maintainer needs more info)

This is a patch we are using on a legacy site and are planning a D11 upgrade. I ran the test locally without the fix and it appears to be passing.

Is this still an issue?