Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
The urlfilter only lists items in the node table. But what about some of the other important nodes ...
e.g. user/###
e.g. blog/###
e.g. topics/x/y/z
e.g. taxonomies
And what about mapping them through a path alias? (oh, I see that's handled in the code)
Comments
Comment #1
deekayen CreditAttribution: deekayen commentedYou wouldn't happen to know any modules off hand that might have similar base functions for listing stuff like blog, user, topics, etc so I don't have to reinvent the wheel, would you?
Comment #2
deekayen CreditAttribution: deekayen commentedI added taxonomy/term/# and /blog to HEAD, DRUPAL-4-7, and DRUPAL4-6. I'm hesitant to go beyond that since spiders should be expected to do a bit of crawling beyond just the URLs in the URL list. For example, I can easily see admins adding /user to robots.txt (because I do).