Add Google Search Appliance to list of known crawlers

Not quite sure how this would work. Are you talking about for the redirection settings or for the CAS auth. Cas allready supports multiple login methods when configured properly. It would maybe help to understand the use case better? Wouldn't it be the users rather than the IP's that would determine the login method?

Suppose you were using the standard Drupal user authentication methods (instead of CAS) --- how would you configure Drupal to work with the GSA?

I see three possible solutions:

1. Complain to Google to change the behavior of their GSA.
2. All users, including users you think of as CAS users, can log in through the standard Drupal login form --- the username is likely the same as their cas user name, but their password is probably unknown (but could be changed through SQL or the admin interface).
3. Or, similar the idea above, write a module which alters the Drupal login form to not require a valid password when coming from the GSA IP address. (i.e. in MODULE_form_user_login_block_alter and MODULE_form_user_login_alter, unset($form['#validate']['user_login_authenticate_validate']) if the IP address matches that of your GSA.)

Since none require changes to the CAS module, I'm marking this as closed. Good luck!

Ummm.... Not sure we should close this. Getting search appliances to work correctly seems like an important goal. The problem comes when the redirection settings that the user wants to apply for most modules, should never happen from specific IP address.

Right now you could add the user/login page to the exclusion list for redirection, and this might solve the problem, but I'd like readnhead to comment here. You're basically won't fixing a feature request, and as the module maintainer, I'm not quite ready to do that yet.

Since we haven't heard back @realityloop in over 2 months, Im going to assume he was able to get it working. If not, feel free to reopen the issue.

He actually contacted me directly just before drupal con indicating that this is still something he wants to get fixed, but we were strategizing about how to do it. I have an idea of how to do it, but no time quite now. I'd like to just add support for a different syntax in the excluded pages that would do this. I don't quite have time right now but will try and get this soon.

We'll only do in 6.x-3.x dev so bumping version here.

Can you help me understand the root issue here? Is the problem that pages are being protected with 'cas_access' / 'cas_pages' which cannot be avoided by the Google Search Appliance?

Personally I'd rather see us get out of the content access business and leave the recommended solution as a combination of auto-assigned roles for CAS users and the Content Access module.

No the problem is that the search appliance is that we want the best of both worlds. We want to be able to use check for an existing auth for users, but then the search appliance will not be able to walk the code, cause it will get redirects. The solution is that we need to enhance the exclude cas processing for specific pages to allow source ip filtering, so that we can use the check authentication method for most people, but not have it mess with the search appliance. The mechanism helps eliminate redirects on to public rss feeeds as well.

We are not in the content access business, but we're trying to "facilitate logins" for those that need access rather than giving them an access denies page, when they haven't logged in yet. I use Content Access in conjunction with CAS all the time. We have lots of deep links from our public facing web server into our drupal portal, and this is what makes sure people get authenticated when they hit those links, if they (after having logged in) still don't have access, it's the Content Access module that makes sure that they get an access denied message.

We are not in the content access business, but we're trying to "facilitate logins" for those that need access rather than giving them an access denies page, when they haven't logged in yet.

What about an alternative approach to this, namely a custom access denied page -- one that would present an access denied message if the user is logged in, or drupal_goto('cas', array('query' => 'destination=....')) if the user isn't logged in.

This wouldn't interfere with the search appliance (since it'd be already logged in) but would facilitate logins as before.

Would this work?

That approach might work, but I'm not quite sure how to implement this without squashing the site administrators ability to customize their own access denied page, or forcing them to embed php code. Is there a hook for preprocessing access denied pages? In terms of ease of use of the cas module, this feels like a step backward to me.

Needless to say I use this feature heavily on my own sites, and would like to understand more what you have in mind before removing this code from the module. Could you walk me through how I would configure that? I think we should take this part of the discussion to a separate issue, since it really isn't about using search engines with the CAS module.

That being said, it doesn't fix all of this problem.

You still have the problem of what happens when the "Automatically check to see if a user is logged in" option is used, even when cas redirection is not use. We don't want SEO thrown out just because we want to facilitate the login from single sign-on. Thinking more about it, we might want to put in some search engine detection logic to make sure that cas processing is disabled when we see a search engine hitting our site? That might be a better approach than the IP address method described above.

To clarify, I don't plan on taking any action here without a thorough discussion. I'm just trying to understand the root cause of the issue and am throwing out some ideas.

What is tripping the CAS login redirection for @realityloop?

Recall that _cas_allow_check_for_login() (which is the code that determines if we should "Automatically check to see if a user is logged in") already has a large list of crawlers:

  // Check to see if we've got a search bot.
  $crawlers = array(
    'Google',
    'msnbot',
    'Rambler',
    'Yahoo',
    'AbachoBOT',
    'accoona',
    'AcoiRobot',
    'ASPSeek',
    'CrocCrawler',
    'Dumbot',
    'FAST-WebCrawler',
    'GeonaBot',
    'Gigabot',
    'Lycos',
    'MSRBOT',
    'Scooter',
    'AltaVista',
    'IDBot',
    'eStyle',
    'Scrubby',
    );
  // Return on the first find.
  foreach ($crawlers as $c) {
    if (stripos($_SERVER['HTTP_USER_AGENT'], $c) !== FALSE) {
      return FALSE;
    }
  }

Do we just need to add the GSE user agent to that list?

The GSA will log in as multiple accounts that have different levels of access, this can be done by configuring the GSA to submit forms at particular URL's, however currently it just get's redirected to CAS login which it can't handle.

And this is because the "See if user is already logged in" (gateway) option is enabled? If so, adding the GSA user agent to the list of robots to avoid gateway redirection would be sufficient.

I'm not sure where to go with this issue since we haven't heard back from realityloop in a few months. Marking as postponed until there is some clarification on what exactly is preventing the Google Search Appliance from using the regular Drupal login mechanism.

So we'd just need to add the GSA user-agent string to the list of known robots in CAS which would prevent the login redirection from occurring.

I've added gsa-crawler to the list of known crawlers. This will prevent the CAS gateway redirection which was likely causing problems for the google search appliance.

Committed to 6.x-3.x and 7.x-1.x.

Please reopen if you still experience an issue.