I understand the instructions say required but theoretically could we not still use it if we have url like index.php?q=page-title ? I could always create the robot.txt myself but was curious if there was a way to make this mod work without clean urls

Comments

hass’s picture

hass commented
Status: Active » Closed (works as designed)

No, this module cannot work without clean urls. Search engines try to find a robots.txt in the site root and not behind urls with params.

dave reid’s picture

dave reid
he/him
Primary language English
Location Nebraska USA
commented

Actually the module can work without clean urls enabled! In the .htaccess file of your Drupal's root directory, add the following two lines to the mod_rewrite section, immediately after the line that says "RewriteEngine on":

     # Add redirection for the robots.txt path for use with the RobotsTxt module.
     RewriteRule ^(robots.txt)$ index.php?q=$1

I have tested this myself and I hope it helps.

hass’s picture

hass commented

Not sure why someone shouldn't use clean urls if mod_rewrite is available. But without mod_rewrite the module cannot work. Maybe we should write this to the readme.

dave reid’s picture

dave reid
he/him
Primary language English
Location Nebraska USA
commented
Status: Closed (works as designed) » Fixed

Yeah I guess that makes sense. But some people *might* have clean URL's turned off, but mod_rewrite enabled. BTW, it was already included in the README.txt that you committed. So I'm going to mark this as fixed.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.