Drupal core has a caching system which this module does not use. Indeed, the separate cache backends should be separate projects so that they can get different releases. Even more, to ease the maintenance of those modules, the core mechanism should be backported to 6.x -- it should be extremely easy because AFAIK the basic functions did not change signatures. All one needs to do is exchange the the queries from DBTNG to their 6.x version.

Comments

slantview’s picture

Well there is still some benefit to using Cache Router in Drupal 7. Although I have not had time to convert my engines to DrupalCacheInterface, the whole point of the module was to be able to separate the bins so you can use multiple cache engines for different cache tables. I would not say that it is entirely useless. There are still some things that Cache Router provides that memcache api module does not. For example, you cannot use shared bins in memcache api without flushing the entire bin.

Why do we need contrib modules for these things, why can't we just port the engines to core and then cache router would be a very simple module to add on if you need different bins for different cache tables.

As far as APC, I will give Raymond commit access so he can begin work on it, but I don't think it should be a contrib module. I think of all the engines, Memcache, APC and File should be in core.

What do you think?

chx’s picture

Core provides a way to use different cache backends for dfiferent bins and allows one (not necessarily the Drupal provided SQL based) for default. Doing this happens with a consistent pattern to many other pluggable subsystems in D7, queue comes immediately to my mind because I wrote that too but I believe password hashing was ported to the interface / class name in variable pattern.

We can only port the engines into Drupal 8, too late for Drupal 7.

I stand by what I said: routing is now in core and the various backends should be separate projects.

chx’s picture

So where are we?

chx’s picture

Regular monthly bump.

andypost’s picture

1) Decide on ClassNames to prevent #564460: Class conflict, namespace collisions
DrupalCache{Engine}
Drupal{Engine}Cache

Core uses DrupalDatabaseCache

2) Create pattern for settings.php

$conf['cache_class_'. $bin] = 'Drupal{Engine}Cache'; // or any form 1)

other settings could be same as before

3) fix engines and locking (native D7 DB locking is bad idea for memcache)

4) document changes in readme

chx’s picture

Been a time since I bumped this but now that someone ported MongoDB I am getting really pissed off. What a waste of time and effort!

andypost’s picture

I was busy last month... There's a some progress with MongoDB #778150: MongoDB engine

slantview’s picture

Well maybe you should get pissed that you rewrote the core caching system without referencing how any other project was using it instead.

chx’s picture

I do not even remember whether I ported the core system to the new class based format but that's irrelevant. What's relevant that there is duplicate work for mongodb and memcached and so on. I really have no idea how could I get you to stop this project for D7 :( because it really has no point to exist.

chx’s picture

Title: The 7.x version does not use the caching system » Do not release Drupal 7 version of this and move the engines into separate projects

Retitling it to better reflect the state of affairs

JirkaRybka’s picture

This is a very sad situation. Cachrouter module was there earlier than core changes, and people are really using it already, because it's the only usable option on 6.x. What would you say, if someone wrote some other, independent implementation of fields or views for 7.x or 8.x and pushed it into core, without asking the CCK or views maintainers, and only THEN go to CCK/views issue queue, and call the whole CCK/views project waste of effort, and ask for it to be just discontinued. Well, I get the impression like THIS is what it means when Drupal community says "let it first mature in contrib"? I hope I'm wrong here. I really hope.

Also projects for (some) separate engines are going to be duplicate to other contrib now, which is discouraged, yet the CacheRouter versions are more debugged than the older projects (in particular, I'm speaking of File engine, where I recently put quite a bit of effort).

So sorry for writing like this, but I think the authors of the core changes should significantly help to sort this mess out, since it's in fact them who deprecated a progressing project, without taking the core changes all the way to where the earlier project already was (i.e. a working set of engines). Correct me if I'm wrong.

Sad situation, indeed. No offence meant..

chx’s picture

The problem here is that this is a very small subsystem and the whole change that deprecated the routing part module was done in less than two weeks. #482622: Make cache friendlier for alternative implementations If you look around how pluggable subsystems is done in D7 (say, DBTNG or queue) that's really the only way it could have happened.

I really do not know how could I help sort this mess out? Tell me please. If there are separate engines which are worse than can we take the cache router versions and just them in the D7 version of those projects? (I was not even aware that there was a separate file caching project).

Tell me. I do not want to hurt anyone and believe me, all I want is make Drupal better.

As for "let it mature in contrib" -- there was noone or nothing stopping the maintainers of this project from submitting it for core inclusion. The issue I linked above was kinda late in the D7 cycle so you can't even say we have forestalled such a move.

andypost’s picture

My vision of D7 is not complete for today. I think for now it's not suitable because most of engines are not supporting nor enumeration and wildcards for cleaning also only memcache is exposes multiple_get(). eAccelerator does not support caching userdata anymore :(

So I think only engines could stay for D7
- memcache (with problems with is_empty() and clear by wildcard, no locking)
- file-based backend
- APC (probable same troubles as memcache + no multiple get)
- xcache (I don't know much about it's roadmap and % of using, same troubles as APC)

Newcommers
- wincache - very promising (could be extended to usable with D7, very responsive team)
- Zend DC - same troubles as APC but supports namespaces, suppose it's API could be extended, but I dont know much about it.

slantview’s picture

Fine. No Drupal 7 version will be released. I'm tired of this crap. I'm tired of pouring my little spare development time into a project that has been a redheaded step-child since the beginning. Chx, you have never given your support to this project, and frankly I'm tired of having to deal with the backlash.

I'll work out a way to get my ideas in core later. My main concern is and always has been making it easier for users. Clearly not a personal goal of yours. I'll move this technology into the splinter modules and i'll submit patches to core to try to clean up the current mess of cache.inc.

I spent hours studying how every other OSS CMS / Framework was doing caching before writing this code. You spent two hours writing the new core caching system and didn't even bother looking up to see if somebody had already done it.

All I want is to make Drupal better as well, and since the only way to do that is to play your games, I'll do it.

slantview’s picture

Also, thank you to JirkaRybka for recognizing the situation as it is. It's even more insulting because I emailed and contacted chx several times during the 2 and a half years I've worked on this module from day one to try to get his help / approval and never once did I get anything besides talking down about the module and telling me how unnecessary it is. Even though it has saved hundreds / thousands of websites when memcache didn't have a usable version for D6 and there was no where else to turn.

Now he spends two hours and tells me how useless this is. Well, fine, guess Drupal will be just fine without it. I'll spend my time and efforts elsewhere in Drupal where it will be used to make Drupal better. I can no longer fight chx's personal agenda against this module. I have too many better things to do with my spare time.

So sorry I got divorced and worked on paying my rent instead of spending my time rewriting Drupal 7 core caching. Next time I guess.

I always intended to get this technology into D7, but now it will have to wait for D8, so, c'est la vie.

R.Muilwijk’s picture

Too bad this is point of discussion again as already was in September 2009 http://drupal.org/node/573104, where I already made clear this project should get into core or cache router should use drupal 7's new caching system.

@andypost, The APC project has already been ported to drupal 7.x caching system in january 2010, I've been keeping it up to date against all the alpha versions since and should still work. Still planning to produce some interfaces for nice statistics.

@all, We should provide a backported version of the drupal 7 cache system which can be included in d6 (the cache router project?). Maybe we should also start a Drupal caching group where can can setup a Roadmap together and see how we can get it in core as a team effort.

slantview’s picture

I agree wholeheartedly about starting a d.o cache roadmap. High Performance group is a good place for that IMO.

I am glad to see that APC has been active, I plan on helping to maintain that work as well as add new features. APC is both an opcode cache and a storage cache and I think it would be wise to allow people to use it as both and provide good information on when it is appropriate to use for either. I started the APC project and brought it into cache router as that made sense at the time. Since the only way out of this mess according to chx is to splinter these modules off, then I believe that we should do that.

I do not agree about backporting the D7 version of the cache system. I do not think the new system is better than the D6 version of cache router and no way will I pollute the one good version of this module with the junk that has been accepted into Drupal 7 core. I will push to make Drupal 8 core better.

slantview’s picture

I created a mini-roadmap for D8 here: http://groups.drupal.org/node/75823 and a discussion about it here: http://groups.drupal.org/node/75833

Let's move this discussion over there and try to come up with something to make Drupal better.

chx’s picture

I do not have an agenda against anyone, I only would like Drupal better. Calling D7 cache API "junk" is not something I agree with nor something I will debate , that does not lead to anything good.

Let's discuss what remains relevant in light of the junk we got. I hear cache router is superior but I do not see. How it's superior?

slantview’s picture

Chx, I apologize for letting my frustration with this get the best of me. I think you made great strides in the right direction with d7 cache and you did something that I didn't and that is actually do the work and make some progress. I have always liked your philosophy of making Drupal better, one patch at a time, one day at a time. I do not want to confuse my frustrations with my upmost respect for everything that you give to this project. You are truly an inspiration with the amount of work you have put into the Drupal project. I am sorry for being rude, it was a knee jerk reaction to your criticisms.

With that said, my technical disagreements I believe are still valid. Here are a couple reasons why I think the cache router implementation was better than what is currently.

1. Abstract class instead of interface. This allows us to add an option for static caching of data in te parent class and this is a configurable option. I believe that this is a benefit for frequently accessed data to limit the number of calls over the wire in setups such as memcache where the data has to cross a network.

2. Use $config instead of variable_get. variable_get uses caching and now you have to store that in the database since it cannot be stored in one of the cache technologies.

3. Dynamic loading of class files. In cache router you can use any of the engines and only the configured engines are included. In core we are now forced to make non-developers write php code to include the class needed. I believe this is a dangerous practice because one typo and you have a white screen of death with no warnings. By including only configuration data, you have less chance of user error.

4. Shared bins. The concept of shared bins that maintain their own index of the content contained in the bin allows for both wildcard wipes on technologies that don't support it, and allows for assigning multiple bins to one storage container (e.g. Apc and xcache). This can also be applied to memcache for storing all data in a single memcached daemon.

I think that the current implementation you did ignored some of these creative solutions. I think the current implementation is flawed and incomplete. But we can and will fix it.

Best,

Steve

Dig1’s picture

Slantview

I've been reading 'Drupal 7 APC Cache backend' http://drupal.org/node/573104 and this node and I reckon that you are making some really great points. I admire your logic and stamina and I reckon you are definitely doing the right thing by fighting your corner. So 'hats off' and 'power to your elbow' as they say here in the UK :).

Respect

geekgirlweb’s picture

+1 subscribing

Fidelix’s picture

This is a sad issue.

Each major version of Drupal needs to be faster (and provide all means to amplify that), not slower.
I am surely gonna miss Cache Router for d7...

greg.harvey’s picture

Hi all,

Appreciate the decision has been made to can this module, but struggling to find what I'm supposed to do instead. I'm using filecache on a small blog - it makes a big difference to performance on my weedy VM - what should I do for Drupal 7? Some docs we can link to here so people have a trail to follow?

Edit: Well, Boost seems to be marching on, which is a different but viable alternative. I'm testing the dev snapshot now, but it seems very much in its infancy and does not achieve what slantview was indicating, the ability to direct different bits of the cache to different bins.

Frankly, I need to read up on the changes to caching in Drupal 7 - however, it seems to me at first glance there's still a decent amount of work to do in contrib to create a module that provides an API or some easier way to achieve what CacheRouter used to do. I may be wrong - I need to read the API docs to understand what is going on.

Another edit: Ok, I get it now. I see the DrupalCacheInterface class is where it's at... so there should be a module for each sort of caching, setting its own variables to make Drupal use the correct interface for the correct bin(s). Looks cool. So someone needs to write a CacheRouter filecache implementation for the new cache API. I may be up for doing that, as I think I prefer it to Boost.

R.Muilwijk’s picture

There are already multiple cache backends available for drupal 7:
* http://drupal.org/project/apc
* http://drupal.org/project/memcache

I've send a message to gordon for maintainership of http://drupal.org/project/filecache but he didn't reply. Maybe smart to start of a new module for a file cache backend.

greg.harvey’s picture

Yes, I used the APC one for now. Perhaps someone should follow the steps required to take over the filecache module:
http://drupal.org/node/251466

I am interested in doing that, if no one else is. Seems a shame to make a new module because the ideal namespace is taken and the maintainer is not responding... =/

andypost’s picture

I'm still dreaming about universal cacherouter as API and different backends as separate projects. I think git migration could bring new frontiers for current namespace

greg.harvey’s picture

That dream should be reality now - that's what chx is talking about. I'm volunteering to take on the "separate project" for the file caching backend. I'm just saying I'd rather take on an existing namespace than make a new module needlessly. =)

andypost’s picture

Boost module has been branched to 7 #325813-44: Boost Port to 7.x

greg.harvey’s picture

Boost != simple 'filecache' engine, IMHO. But we should stop chattering here, probably. ;-)

andypost’s picture

ogi’s picture

No, only 1 abandoned project :-) filecache is actively developed now. See my post about cacherouter, fastpath_fscache and filecache.

soyarma’s picture

I haven't dived into the D7 caching to any great extent. The question I'd like answered (because it was the biggest boon of CacheRouter) is: Can I cache different things in different caching technologies?

Something I am writing for the D6 version of this module is the ability to store session data in a different caching technology. Cacherouter makes that super simple because I just define the bins and what technology they use and then poof! the cache_get and set function save the data there. I don't have to have lots of different modules enabled or loading when they aren't needed.

chx’s picture

Can I cache different things in different caching technologies? <= yes. . Also, session storage is a whole another can of worms nothing to do with caching...

soyarma’s picture

A lot of tests done lately have shown that on large sites with lots of registered users reads and writes to the sessions table amount to as much as 75% of the I/O on the MySQL DB server.

The D6 memcache API module handles moving sessions into memcache, and the addition I wrote to cacherouter also does that for any technology that cacherouter supports (though only tested in APC and memcache at present). While it is not 'caching' per-se, it is an ephemeral storage vs persistent storage issue, and all of the 'caching' technologies are essentially ephemeral storage.

jeffwidman’s picture

subscribe

kolier’s picture

Subscribe.

foripepe’s picture

subscribe

shamio’s picture

In a caching modules review on one of the groups of Drupal. i saw that Cache Router improves the speed very high and it was rated as one of the best modules. I currently use Block cache alter and it is rated moderate in this review so i am so eager to use Cache Router but i don't know when the Drupal 7 version will be ready. Does anyone know? Or do you suggest any alternative module to be used in addition to Memchace and APC which works like Cache Router?

drupik’s picture

Why not just use mongodb?

thedavidmeister’s picture

Status: Active » Fixed

this looks fixed to me. Discussion seems to be over.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.