Expiration Grid - road map for this module

Have you thought about doing it with VBO? I'm always interested in stuff like that.

So for the node its
Node
Node type like 'page'
NID

View
view name
view display like 'page' or 'page_1' or 'default'

taxonomy
vocabulary
TID

Code that does this

/**
 * Gets page_callback & page_arguments from menu_router table
 *
 * Allows for any content type to have it's own cache expiration.
 * TODO Better support of panels.
 */
function _boost_get_menu_router() {
  $router_item = menu_get_item();

  // Handle nodes
  if (arg(0) == 'node' && is_numeric(arg(1))) {
    $node = node_load(arg(1));
    $router_item['page_callback'] = 'node';
    $router_item['page_type'] = $node->type;
    $router_item['page_id'] = arg(1);
    return $router_item;
  }
  // Handle taxonomy
  if (arg(0) == 'taxonomy' && is_numeric(arg(2))) {
    $term = taxonomy_get_term(arg(2));
    $vocab = taxonomy_vocabulary_load($term->vid);
    $router_item['page_callback'] = 'taxonomy';
    $router_item['page_type'] = $vocab->name;
    $router_item['page_id'] = arg(2);
    return $router_item;
  }
  // Handle users
  if (arg(0) == 'user' && is_numeric(arg(1))) {
    $router_item['page_callback'] = 'user';
    $router_item['page_type'] = implode(', ', user_load(array('uid' => arg(1)))->roles);
    $router_item['page_id'] = arg(1);
    return $router_item;
  }
  // Handle views
  if ($router_item['page_callback'] == 'views_page') {
    $router_item['page_callback'] = 'view';
    $router_item['page_type'] = array_shift($router_item['page_arguments']);
    $router_item['page_id'] = array_shift($router_item['page_arguments']);
    // See http://drupal.org/node/651798 for the reason why this if is needed
    if (is_array($router_item['page_id'])) {
      $router_item['page_id'] = array_shift($router_item['page_id']);
    }
    return $router_item;
  }

  // Try to handle everything else
  if (is_array($router_item['page_arguments'])) {
    foreach ($router_item['page_arguments'] as $string) {
      if (is_string($string)) {
        $router_item['page_type'] = $string;
        break;
      }
    }
  }
  // Set empty if page_arguments is an empty object.
  if (!isset($router_item['page_type']) && empty($router_item['page_arguments'])) {
    $router_item['page_type'] = '';
  }
  // Set to first object in array if page_arguments is still an array and cast it as an string.
  if (!isset($router_item['page_type']) && is_array($router_item['page_arguments'])) {
    if (is_object($router_item['page_arguments'][0])) {
      $router_item['page_type'] = (string)get_class($router_item['page_arguments'][0]);
    }
    else {
      $router_item['page_type'] = (string)$router_item['page_arguments'][0];
    }
  }


  // Handle panels
  if (strstr($router_item['page_callback'], 'page_execute')) {
    if (db_table_exists('delegator_pages')) {
      $pid = db_fetch_array(db_query_range("SELECT pid FROM {delegator_pages} WHERE name = '%s'", $router_item['page_type'], 0, 1));
    }
    elseif (db_table_exists('page_manager_pages')) {
      $pid = db_fetch_array(db_query_range("SELECT pid FROM {page_manager_pages} WHERE name = '%s'", $router_item['page_type'], 0, 1));
    }
    $router_item['page_id'] = $pid ? $pid['pid'] : 0;
  }

  return $router_item;
}

Like the idea of copying pathauto for node presets & not caching certain node types. There is a lot of potential here.

What needs to happen:
Scan every view looking for paths. Each view that contains a path is treated like a node type. Detect other entries in the boost_cache table and generate configuration options for them as well.

Each cache type can have options for (some will be specific to the content container; node, view, etc...)
* min & max cache lifetime
* is-cacheable setting
* pager/url-query setting <- auto detect and have smart defaults (nodes, no pager; views with exposed filters will allow all; etc...)
* promoted can flush front page; pager support.
* node reference flush: forwards, backwards, both
* taxonomy control; certain vocab's can have different flushing options
* menu tree options
* views handling; figure out the paging issue. flush first 2 pages asap (configurable), expire rest over a period of time.
* expire or flush (expire marks it for a crawler; flush kills from the cache instantly)
* support for coded advanced configuration (hooks!)
* make this "exportable"

Different actions can trigger different expiration/flush settings. Example: New comments should only flush the view page that the node lives on, not the entire view with all it's pagers. Or if the theme doesn't indicate the comment count on the view then have the option to not flush the view on comments. Or if the view is directly related to comments then the full view should be flushed in a graceful manner.

The configuration file (the "exportable") will come before the GUI, because the GUI will be quite complicated & it will take some time to make this graceful. Once the GUI is in place, make the manual configuration part as minimal as possible by being smart with detection and defaults.

Make all of this support domain access and be multisite friendly. This is the road map; in short I'll be taking out the smarts from boost and putting it in this module. Boost does most of this right now in some sort of fashion; making it happen based on set rules is key to success.

How to deal with all the writes that happen to the boost_cache table: Don't update all fields on each cache creation "action".

How to deal with all the writes that happen to the boost_cache_relationship table: Have a dirty flag so if the parent or child entity gets updated then it knows to recreate the relationship on cache creation. Will need some smart logic for views pagers. This is a major priority. Once I get this figured out I can then bring in views to the expires module.

views need to store the argument given to it as well as the page number its on. Finding new content on views with arguments will be a challenge; taxonomy I can make it work, other types of arguments will not be as easy to magically do. Current progress on views page cache logic is going on here
http://drupal.org/node/785766#comment-3341042

1.0 sounds like a plan. Go ahead and publish a release; if you don't I might get around to it by Friday.

This might be an interest to you: http://drupal.org/project/httprl I had some free time yesterday so I put together the code into a module. The possibilities that a non-blocking http request brings to the table is mind boggling. Any task that is not directly associated with generating the current page's html can in theory be spun off into a background task. Something to keep in mind as you develop the code.

Not all hosts have curl & I'm not sure if you can have curl "ping" a url (non-blocking mode in httprl). I might want to add in a curl implementation to httprl as well as one using sockets (like d6) as a fallback; because some hosts have socket_select disabled.

Found this library for curl that has something similar to the non blocking mode of httprl: https://github.com/jmathai/php-multi-curl

7.x boost is dumb currently. I was going to put the smarts in expire. Your best bet is to translate what is in 6.x and move it forward to 7.x. I still don't have any 7.x sites so any code that is of 7.x series in any of my modules is fairly basic.