Sitemap generated using 'Simple Sitemap' module is accessible in search result. Ideal behavior is it should prevent sitemap from appearing in search results.

CommentFileSizeAuthor
#4 2878547-4.patch730 bytesziomizar
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

renukakulkarni created an issue. See original summary.

gbyte’s picture

Title: Simple sitemap module is not preventing sitemap to appear in search results. » Sitemap included in search results
Version: 8.x-2.9 » 8.x-2.x-dev
Status: Active » Postponed (maintainer needs more info)

What search results are you referring to? Are you referring to Drupal core's Search module?

renukakulkarni’s picture

Status: Postponed (maintainer needs more info) » Active

@gbyte.co,

I am referring to google search results. The sitemap page configured using this module comes up on google. For example, if you access "site:site_url + sitemap" (Enter your site URL in place of 'site_url' ).

There are other ways to prevent XML sitemap to appear in google search results. It seems this module is not doing that.

ziomizar’s picture

Status: Active » Needs review
FileSize
730 bytes

Following https://developers.google.com/webmasters/control-crawl-index/docs/robots... seems that suggested way to not index sitemap is using the X-Robots-Tag: noindex header.

Not sure if that should be configurable from backend, sitemap don't should be in the index.

An idea to solve this issue in the patch

gbyte’s picture

Status: Needs review » Fixed

Thanks guys for your input. The sitemap shouldn't be indexed by google apart from it being actually linked somewhere. I am going to commit your changes to prevent such a case.

  • gbyte.co committed 6d69f2b on 8.x-2.x
    Issue #2878547 by ziomizar: Sitemap included in search results
    
gbyte’s picture

Category: Bug report » Feature request

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.