Follow-up to #1960704: Allow collection of access statistics asynchronously

It would be wonderful to have cache status logged from the client-side. This was deemed too difficult originally because HTTP headers are unavailable from within JavaScript.

One possibility is to set a cookie (perhaps within a hook_boot()), but care will need to be taken to ensure that it doesn't itself affect page cachability.

Comments

mondrake’s picture

Assigned: Unassigned » mondrake

I'll give a first stab, ok?

mondrake’s picture

Status: Active » Needs review
StatusFileSize
new2.01 KB

Very first attempt. Just focusing on pure Drupal caching. Seems to work but can't get any test working ($_COOKIE seems to persist across different subsequent drupalGets). Also raises the bootstrap phase (can we avoid?), and there is a js function for getting the cookie value that I would not know where to put...

mondrake’s picture

Assigned: mondrake » Unassigned
iamEAP’s picture

Status: Needs review » Needs work

This looks promising! Thanks, mondrake.

+++ b/better_statistics.moduleundefined
@@ -495,3 +495,21 @@ function better_statistics_add_js_api() {
+
+function better_statistics_boot() {

Naturally, we'll need a doc block here; more importantly, we'll need hook_update_N() function to mark Better Statistics as a bootstrap module in the system table. Without it, only fresh installs of the module will fire the hook_boot implementation. Sorry, nevermind. Forgot that hook_exit implementors are also automatically marked as boostrap modules (though the doc block comment still applies).

+++ b/better_statistics.moduleundefined
@@ -495,3 +495,21 @@ function better_statistics_add_js_api() {
+    if (array_search('X-Drupal-Cache: HIT', $headers) !== FALSE) {
+      setcookie('Drupal.cache.status', 'HIT', time() + 60);

Regarding cookie persistence, perhaps we want to set the cookie time to time() or REQUEST_TIME without any additional time? If we can set it with minimum time, we'll still be able to access it on the client side, but it won't be sent with subsequent requests. That would be ideal.

+++ b/js/fields/custom.jsundefined
@@ -1,7 +1,22 @@
+function getCookieVal(key) {

Since we're not using this anywhere else, we may just want to throw it in the event listener callback; perhaps even as a closure.

+++ b/js/fields/custom.jsundefined
@@ -1,7 +1,22 @@
+  if (cacheStatus == 'void' || cacheStatus == null) {
+    event.data.cache = '';

I believe we still want to pass this in as the literal string "NULL," which is how it's currently handled.

mondrake’s picture

Assigned: Unassigned » mondrake

I'll take on a bit of this, but the rest I would need your help

mondrake’s picture

Assigned: mondrake » Unassigned
Status: Needs work » Needs review
StatusFileSize
new3.84 KB
new3.03 KB

Ok, here's an update

  • hook_boot documented
  • setting the cookie expiration to REQUEST_TIME seems too stringent - the client-side can not access the cookie value. However 60 secs seems a reasonable time to me to allow for actual transmission of the page and execution of the client side js. In any case, all pages served will bear 'HIT', or 'MISS', or 'void', that should cover all cases.
  • moved getCookie within the listener callback.

I believe we still want to pass this in as the literal string "NULL," which is how it's currently handled.

Not sure I understand. Pass to event.data.cache?

Cheers

iamEAP’s picture

Status: Needs review » Needs work

Thanks for the work, mondrake. Couple of smaller points in #1 and #2 below. #3 may merit a little more discussion.

+++ b/better_statistics.moduleundefined
@@ -97,6 +101,42 @@ function better_statistics_ctools_plugin_api_hook_name() {
+
+  // Is Better Statistics enabled to track cached pages.
+  $bs_cache_logging = variable_get('statistics_access_log_restrictions_cache', TRUE);

1. Instead of checking if cached page logging is enabled, we may want to just check that the "cache status" stats field is enabled in the accesslog. It's possible someone wants to track cache status of MISS vs. NULL only (no HITs; disabled on cached pages). More importantly, we do not want to set a cookie if the cache status stats field isn't even enabled and collecting data.

2. Re "NULL": Yes, we should set event.data.cache = 'NULL' rather than event.data.cache = '' to remain consistent with how it's stored currently (which is the literal string "NULL" rather than an implied "null" value in the DB).

3. Another potential problem: the Statistics AJAX callback itself (and actually, any AJAX callback, or even broken/404 image; anything that will bootstrap the same Drupal instance) will set the Drupal.cache.status cookie. In the case of an AJAX callback, it will always be MISS. In the case of a broken image, it could be HIT or MISS. If they happen to load/fire before the statistics.accesslog event is triggered, you'd get bad data.

Not sure how we can handle that. We could definitely exclude our own AJAX callback, and perhaps the System and Views AJAX callbacks. Don't think we can do much else, though.

mondrake’s picture

Status: Needs work » Needs review
StatusFileSize
new1.47 KB
new3.14 KB

Hi

#1 done, pls check

#2 done as well - however it seems to me in 'default' mode (i.e. data collection on hook_exit only) we are capturing null value, not a 'NULL' literal

#3 true - personally I am simply using the page restriction feature to exclude calls to system/* or the likes

socialnicheguru’s picture

patch no longer applies to dev version
git apply better_statistics-js_cache_status_detction-2019937-8.patch
error: patch failed: better_statistics.module:26
error: better_statistics.module: patch does not apply
error: patch failed: js/fields/custom.js:1
error: js/fields/custom.js: patch does not apply

patch -p1 < better_statistics-js_cache_status_detction-2019937-8.patch
patching file better_statistics.module
Reversed (or previously applied) patch detected! Assume -R? [n]
Apply anyway? [n] y
Hunk #1 succeeded at 47 with fuzz 2 (offset 21 lines).
Hunk #2 succeeded at 146 (offset 43 lines).
patching file js/fields/custom.js
Reversed (or previously applied) patch detected! Assume -R? [n]
Apply anyway? [n] y
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file js/fields/custom.js.rej
You have new mail in /var/mail/aegir
aegir@devmac:~/p/7/m/all/better_statistics$ more js/fields/custom.js.rej
--- js/fields/custom.js
+++ js/fields/custom.js
@@ -1,7 +1,24 @@
document.addEventListener('statistics.accesslog', function(event) {
- // Cache Status: Impossible to know from the client-side, so we pass in a
- // dummy value to ensure no misleading values are saved.
- event.data.cache = 'UNKN';
+
+ // Retrieve a cookie's value by key.
+ function getCookie(key) {
+ var tmp = document.cookie.match((new RegExp(key + '=[a-zA-Z0-9.()=|%/_]+($|;)', 'g')));
+ if (!tmp || !tmp[0]) {
+ return null;
+ }
+ else {
+ return unescape(tmp[0].substring(key.length + 1, tmp[0].length).replace(';', '')) || null;
+ }
+ };
+
+ // Cache Status.
+ var cacheStatus = getCookie('Drupal.cache.status');
+ if (cacheStatus == 'void' || cacheStatus == null) {
+ event.data.cache = 'NULL';
+ }
+ else {
+ event.data.cache = cacheStatus;
+ }

// Peak Memory: There's no way to know this, so we pass a bum value to
// ensure the misleading peak memory value of the callback is overridden.

iamEAP’s picture

iamEAP’s picture

Triggered a re-test of #8 and it appears to still apply to dev; not sure why you're unable to apply the patch, @socialnicheguru.

By the way, I'd really love to try and resolve this before I tag a 7.x-1.4 release. My concerns would be best allayed if someone did end-to-end testing on a free dev instance of vanilla D7 on Pantheon (which would give you that Varnish layer).

socialnicheguru’s picture

patch does apply fine.