Class 'DatabaseTasks_autoslave' not found

Hi Bnaya

Yes, please upgrade to 7.x-1.x-dev. A lot of bugs have been fixed (and some features added), and I believe this particular problem is solved in the dev version. I want to make a new release soon, but I'm looking for some more feedback on 7.x-1.x-dev before doing so. It's running stable on my system (much more stable than 7.x-1.5), but I just want to be sure that it's not just me it's working for, before making a release.

If you upgrade to 7.x-1.x-dev, please remember to copy the autoslave folder into includes/database again, and run update.php.

I only loaded the Autoslave driver on my slave server because I want the master to use only the master DB to read and write.

I'm not sure what you mean be this? In case you have multiple web servers, the autoslave driver should be loaded on all of these.

The slave should read from it's local db and write to the master only.

Again, still not sure what you mean. However, AutoSlave is designed to read from slave db's where applicable and read from the master if not (read: replication lag). You cannot have a webserver that ONLY reads from the slave, as this would mess up cache, etc.

Could you post the contents of your /my/drupal/root/includes/database/autoslave and your $databases['default']['default']?

Hi

Yes, that's wrong. In that way, the slave server won't know which tables are affected by the replication lag at any given time.

You need to use the autoslave driver on all your web servers, so that it may intelligently choose when to read from the either the master or the slave.

The main idea is, that most of the time the reads will be performed on the slave servers, there by being able to spread the main load from one db to several.

In the cases you do not care about replication lag (because you can afford/manage stale data), you can use Views to target queries directly to the slaves or manually in queries you can specify the option 'target' => 'slave'.

The scenario AutoSlave addresses, is about potentially using slave db's for all possible queries, not just the ones manually specified. However, due to replication lag and cache invalidation, it is sometimes necessary to read from the master database. Drupal core does this as well with the db_ignore_slave() whenever a node or a comment is saved. AutoSlave extends this to all queries and per table (instead of the entire database, as db_ignore_slave() does), and also handles the timeout of the "slave ignore" in a more optimized fashion.

/Thomas

So the master database doesn't get a lot of SELECT queries?

That's true. You could include it in the slave pool though, but that would probably not scale in the long run.

Does AutoSlave always forward all the SELECT queries to the slave DB (except when the slave DB hasn't updated yet)?

Yes, AutoSlave forward all the select queries to the slave DB unless certain conditions are met (see the 5 bullet points on the top of the project page).

I don't want the master webserver to connect to remote DB server if it can read it locally.

What is a "master web server"?

Sorry about the many posts

No problem :-)

Do you have any suggestion why am I seeing blank screen on some of my admin pages?

Not at this point. If you get a WSOD, you should be able to see something in your php error logs.

About your example config, yes, you're absolutely right. The configuration will only use the master db, unless it fails.

can you list the contents of your /path-to-drupal/includes/database/autoslave (recursively) ?

Hmm. That looks about. Except for the file size of database.inc, which differs from mine?

The class "DatabaseTasks_autoslave" is defined in autoslave/install.inc, which should be included by the system. Not yet sure why it doesn't happen on your setup. Looks like the system module assumes that the install.inc is loaded. (Don't know it manually determines the class name instead of using db_installer_object(), which would be the way t go IMO).

On my system the autoslave/install.inc is loaded. I'll look some more into this to see when this actually occurs.

What's the result of this query?

SELECT * FROM registry WHERE name = 'DatabaseTasks_autoslave';

I guess it should be loaded

Indeed, it should?

Maybe the 'module' column should be 'autoslave' so it will load it on hook_init?

I don't think that's necessary. Mine looks like this:

+-------------------------+-------+-----------------------------------------+--------+--------+
| name                    | type  | filename                                | module | weight |
+-------------------------+-------+-----------------------------------------+--------+--------+
| DatabaseTasks_autoslave | class | includes/database/autoslave/install.inc |        |      0 |
+-------------------------+-------+-----------------------------------------+--------+--------+

Maybe the registry cache is out of sync somehow?

Not sure. If you can see the status page with autoslave results, then it is called. Perhaps you had a broken cache of the registry?

Btw, if you encounter further issues, I'm very interested in knowing about them. Also, any feedback is appreciated. Especially if things are running great. Then I can make a new release :-)

I don't run the AutoSlave in production myself anymore (not that it's broken, I just don't have any master/slave setups besides my personal development site), but I know of some who do run it in production.

There has been a recent addition to AutoSlave, which makes it more robust against race-conditions. It's a bit heavier on the master db, but this can be mitigated using memcache.

If you're using memcache on your system, i suggest you try the following autoslave options.

$databases['default']['default']['transactionalize writes'] = TRUE;
$databases['default']['default']['affected tables backend'] = 'autoslave.affected_tables.memcache-db-accurate.inc';

Also, if you're not using the dblog module (watchdog to db), I suggest you add this as well:
$databases['default']['default']['watchdog on shutdown'] = TRUE;

Another recommendation is changing the database isolation level to READ-COMMITTED. This will entirely eliminate some race-conditions.

Remember to also use the AutoSlave cache wrapper (see the configuration example on the AutoSlave project page).

I'm planning on making the 'transactionalize writes' option default to TRUE, to make the default configuration the most robust possible.

The AutoSlave cache wrapper is for use with any cache backend that is NOT the database (see: https://drupal.org/node/1952996). It keeps cache updates "in sync" with database commits.

FYI, I've added a "Recommended configuration" section on the project page. (I intentionally left out the 'transactionalize writes' option, as this is now default TRUE in the latest commit to dev.

I think syslog is the most common option. There's a core module for that. For centralized logging, the syslog daemon on each webserver should pass on the logs to a remote syslog server. Not sure how to set that up though. Also, the /admin/reports/dblog is only available for the dblog module, so log viewing will have to be done in some other way.

It will take some time till we get some traffic peak and see it for real, but so far everything looks good.

Great. Looking forward to feedback about the traffic peak.

How can I see what changed in the latest dev release?

Phew, there are a lot of changes. Currently, they're only registrered in git. I'll make a changelog and post here. I need one anyways for the new release.

Here's a summarized change log:

Improvements:
Added "watchdog" to default master-only tables.
Improved effective logic of db_ignore_slave().
Improved affected tables logic.
Use core icons for status pages.
Use slave db for queries regarding schema, where applicable.

Bug fixes:
Fixed bug in force master during early bootstrap phase.
Fixed bug in supportsTransactionalDDL().
Fixed missing affected tables logic for db_select().
Fixed bug in invalidation file logic.
Fixed bug in "watchdog on shutdown"
Fixed warning in memcache-lock.inc (#1958556).

New features (see documentation for description of these):
Added new option "init_commands"
Added new option "use autoslave schema"
Added new option "affected tables backend"
Added new option "bypass page cache"
Added new option "preconnect"
Added new option "flag all tables"
Added new option "resync on cache miss"
Added new option "transactionalize writes"
Added new option "debug"
Added transactional safe cache wrapper.
Added drush commands.
Added new framework for affected tables backends.
Added patches (for core and drush).

Internal changes:
Removed static function getAvailableDrivers().
getConnectionOptions() now returns autoslave connection options.

Hi

That sounds odd. That would imply that write queries are being sent to the slave servers. I always advise (maybe I neglected to mention this, sry) to use readonly users for the slave connections, so that write errors may be trapped (throws an exception). However, I have not experienced this before (nor heard of it), so if you're experiencing that write queries are sent to the slaves from Drupal, I'm eager to know.

To get a better idea of what could be the cause, I have the following questions:

1.) Which version of mysql are you using?
2.) And are you using row-based or statement-based replication?
3.) What kind of isolation level are you using (repeatable-read, read-committed, etc.)

Hi

Ok. I would recommend switching to read-committed and row-based replication. If this is not possible, I would recommend applying the deadlock-mitigation-7.22.patch bundled with AutoSlave.

See also: http://gielfeldt.blogspot.dk/2013/05/drupals-variable-storage-and-lockin...

I don't believe so. I'm running with a "GRANT SELECT" only on my readonly user.

Did you figure out what was happening to the queue table?

No, but since I changed the user to SELECT only I did not see any error in the logs.

Hmmm .... scary. And you haven't seen Drupal throw any exceptions regarding database queries?

Any tips how to analyze the syslog files easily? :-)

Not besides I remember finding this a bit interesting at DrupalCon last year: http://munich2012.drupal.org/sites/default/files/slides/logging_drupal_f...

Did we fix this?

Reviving this thread. Was this fixed?