Our website is receiving 404 errors for robots.txt files in subfolders. Not sure what could be causing them or how to stop them, so I'm here to ask if anyone else has had this issue. We recently launched out website, which was a redesign of our previous one. We checked to see if maybe this was something in the old site, but we can't find these anywhere.

For example, some of the paths for the errors:

/parks/ogt/robots.txt
/water/robots.txt
/FCO/Aquatic-Preserve/robots.txt
/osi/green-lodging/robots.txt
/lands/environmental-services/content/florida-forever/robots.txt
/fco/robots.txt
/fco/MjYXQ/robots.txt
/fco/coral/content/seafan/cOcMY/ZmPLh/robots.txt
/osi/clean-marina/robots.txt

Any help would be appreciated.

Comments

salwine created an issue. See original summary.

mattfarrow’s picture

I'm not seeing this problem in my 7.56 install.

I created a sub-folder robotest and added a robots.txt file.

This was readily accessible via the browser at /robotest/robots.txt.

Just to be sure, I also created a page with an alias of /robotest/hello, which also worked and didn't affect the robots.txt file.

Are you using the standard .htaccess file or have you modified it for your install?

Also, are you running Drupal on Apache or a different web server?

mattfarrow’s picture

Status: Active » Closed (cannot reproduce)

Since I can't reproduce this issue, I'm closing it.