I actually don't know what the problem is, exactly, but here are the known factors:

1) I have a dev and staging environment that are more or less identical, running latest stable version of Drupal 7. The Drupal codebase is the same.

2) Here's the code I'm using in MYMODULE:

MYMODULE_form_alter(&$form, &$form_state, $form_id) {
  if ($form_id == 'webform_client_form_TARGETNID') {
    $form['actions']['submit']['#ajax'] = array(
      'callback' => 'MYMODULE_ajax_callback',
      'wrapper' => str_replace('_','-',$form['#form_id']),
      'effect' => 'fade',
    );
    $form['#submit'][] = 'webform_extra_ajax_submit';
  }
}

function MYMODULE_ajax_submit(&$form, &$form_state) {
	$form_state['rebuild'] = TRUE;
}

function MYMODULE_ajax_callback(&$form, &$form_state) {
  if(form_get_errors()){
    drupal_get_messages();
      return $form;
  }else{
    return $form['#node']->webform['confirmation'];
  }
}

3) The above code works wonderfully in dev. It worked in staging, as well. Then I began installing various server-cache solutions. I'm using Ubuntu 10.04, and pretty much all the parts of the LAMP stack that I can think of have simply been installed using apt-get install, so nothing fancy, and versions are whatever you'd get from doing that. In fact, staging was started as an image of dev two months ago, with appropriate changes to hostname, /etc/hosts, and /etc/apache2/sites-available/MYSITE.

4) I installed apc using "pecl install apc":

echo "extension=apc.so
apc.shm_segments=1
apc.shm_size=128M
apc.ttl=7200
apc.user_ttl=7200
apc.enable_cli=1
apc.stat=1
apc.stat_ctime=1" > /etc/php5/conf.d/apc.ini
apache2ctl restart

5) I installed memcached following the steps from this page:
http://www.zayinkrige.com/installing-memcached-on-ubuntu-10-04/

6) I installed Varnish:

  • /etc/varnish/default.vcl: http://pastebin.com/5bFhmVZ0
  • updated /etc/apache2/ports.conf:
    #NameVirtualHost *:80
    #Listen 80
    NameVirtualHost *:MYPORT
    Listen *:MYPORT
  • updated /etc/apache2/sites-available/MYSITE:
    #<VirtualHost *:80>
    <VirtualHost *:MYPORT>
  • 7) I updated settings.php and downloaded/installed the APC, memcache and Varnish Drupal modules:

// Add Varnish as the page cache handler.
$conf['cache_backends'][] = 'sites/all/modules/contrib/varnish/varnish.cache.inc';
$conf['cache_class_cache_page'] = 'VarnishCache';
// Drupal 7 does not cache pages when we invoke hooks during bootstrap.
// This needs to be disabled.
$conf['page_cache_invoke_hooks'] = FALSE;

// this could be very good, but need to do more research, first.
// $conf['page_cache_without_database'] = TRUE;

$conf['cache_backends'][] = 'sites/all/modules/contrib/apc/drupal_apc_cache.inc';
$conf['cache_class_cache'] = 'DrupalAPCCache';
$conf['cache_class_cache_bootstrap'] = 'DrupalAPCCache';
//$conf['apc_show_debug'] = TRUE;  // Remove the slashes to use debug mode.

$conf['cache_backends'][] = 'sites/all/modules/contrib/memcache/memcache.inc';
$conf['cache_default_class'] = 'MemCacheDrupal';
$conf['cache_class_cache_form'] = 'DrupalDatabaseCache';

# Tell Drupal it's behind a proxy
$conf['reverse_proxy'] = TRUE;
# Tell Drupal what addresses the proxy server(s) use
$conf['reverse_proxy_addresses'] = array('127.0.0.1','MYSERVERIP');

8) Expected behavior (works on dev, and worked on staging before APC/memcached/Varnish):

  • When I submit the form without filling in all the validated fields, the form gets properly checked, fields highlighted, and the fields I filled out so far still have their data in them
  • When I submit the form with all validated fields filled out correctly, the Webform sends email correctly, and the $form['#node']->webform['confirmation'] is indeed returned correctly.
  • 9) Observed behavior (even after I've done my best to reverse as many of the changes for APC/memcached/Varnish that I've described, above):

  • When I submit the form without filling in all the validated fields, the form gets properly checked, fields highlighted, and the fields I filled out so far still have their data in them
  • When I submit the form with all validated fields filled out correctly, the Webform sends email correctly in the background, but the "Please wait" indicator is still spinning, and finally times out after 2 minutes.
  • Conclusion:
    I want caching, but I also want to figure out why this won't work. Am I not turning all these off correctly? My biggest suspect here is Varnish, but maybe it's something else? I'm not even sure how to troubleshoot this at this point. Any suggestions would be very much appreciated.

    Thank you!

    Comments

    john_b’s picture

    Correct me if I am wrong, but do you not risk Drupal writing its cache to memcache as cache backend, and trying to read cached data from APC? What are you aiming to achieve by using both APC and memcache together?

    EDIT: I probably am wrong. Still the logic of having two seperate backend caches defined is unclear.

    Digit Professionals specialising in Drupal, WordPress & CiviCRM support for publishers in non-profit and related sectors

    john_b’s picture

    On your problem, see this thread http://groups.drupal.org/node/45838
    If you are on a single box, I would stop memcached and remove it from settings.php. APC is all you need.

    As for Varnish, you will certainly get problems unless you get your .vcl right for Drupal. You do not say how you have configured Varnish, or where you got the .vcl file. You can always check for Varnish problems by bypassing Varnish (ie. put http://mysite.com:MYPORT into your browser).

    Digit Professionals specialising in Drupal, WordPress & CiviCRM support for publishers in non-profit and related sectors

    edoceo’s picture

    APC for PHP is an opcode not a data cache, from their page:

    is a free and open opcode cache for PHP. Its goal is to provide a free, open, and robust framework for caching and optimizing PHP intermediate code

    Memcache is:

    high-performance, distributed memory object caching system

    So, you would never read data from APC that was written to Memcache; it just wouldn't happen. One caches code, the other caches data.

    edoceo’s picture

    derp, I missed in the config where 'sites/all/modules/contrib/apc/drupal_apc_cache.inc' was loaded.

    That lib does store values in APC's user cache; which is for data.

    Sorry for confusion on that.

    JGO’s picture

    Hi,

    I'm having issues with my ajax based for as well.
    I'm not using anything else than the APC module.

    My ajax form works perfectly at first. After some time (I guess a few hours) , it doesn't work anymore.
    As soon as I clear the cache, it works again.

    I really hope to find a solution. I don't know if this is a core caching issue or an APC module related issue.

    edoceo’s picture

    Basically, after having these kinds of caching issues our fix-path followed something like this:

    1. disabling all caches and observing behaviour, confirming proper functionality
    2. Enabling one cache, for us we started with APC, for both opcode and userdata, full-test
    3. enabled the memcached caching; observe and then adjust (size, ttls, gc), full-test
    4. Then to varnish, for limited requests (GET only), full-test
    5. Took some time to tune the filtering (regsuball()) for cookies and headers
    6. Toggle varnish/no-varinish to confirm proper functionality
    7. adjust varnish more (not /admin and a few others)

    Check settings on both the services (apc, memcache, varnish) to ensure they are proper (size, ttl, gc) and then add the back-ends. Measure between each step.