Hi

I found interesting Statistics granularity module
http://drupal.org/project/statistics_granularity

Does your module something similar (or will be it added in near future) ?
You know, the best could be to have only one statistics addon with a lot of features (submodules).

thanks
Igor

Comments

rsevero’s picture

I don't really know the Statistics Granularity module but from a fast code scan I believe that you can get all new info provided by it with Statistics Pro.

If this previous statement is right, the Statistics Pro module has an important advantage as it does it with a much smaller performance hit. The Statistics Granularity module introduces an UPDATE or an INSERT per page hit and it increases the row size of the node_counter table. The Statistics Pro module only includes a new cron task and it keeps the extra info on a new separate table.

Again, if my first statement is right, the Statistics Pro module also provides much more extra granularity options than Statistics Granularity as the last seems to provide 3 extra options and Statistics Pro provides 13 extra options.

The Statistics Granularity module certainly has some advantages:

  • it has no requirements except from the obvious one: the core Statistics module
  • it is much simplier so it's easier to be sure it has no bugs

It would be great if you could get the Statistics Granularity maintainer to comment on this issue so we could identify any mistake I might have done on this brief analysis.

rsevero’s picture

Assigned: Unassigned » rsevero
igorik’s picture

Hi, thanks for the feedback. I wrote issue to author of Statistics Granularity about this conversation, so I will hope that he join us here.

thanks
Igor

BassistJimmyJam’s picture

I, the Statistics Granularity maintainer, was not aware of this module when I created it (though I did search and ask around). I am currently on vacation and cannot comment further at this time. However, when I return home I will look into this module so that I can give this issue the attention it deserves.

BassistJimmyJam’s picture

I, the Statistics Granularity maintainer, was not aware of this module when I created it (though I did search and ask around). I am currently on vacation and cannot comment further at this time. However, when I return home I will look into this module so that I can give this issue the attention it deserves.

BassistJimmyJam’s picture

Sorry, don't know why my last comment was published twice. Statistics Granularity adds weekly, monthly and yearly node counts in the existing node_counter table. It does this at the end of each page load (using hook_exit()). I see that Statistics Pro adds these counts as well (in a different table) using a cron job. It looks like is uses the access log, which Statistics Granularity does not use (part of a requirement for the original request). So while Statistics Granularity does introduce some extra processing per page load, it does not require the processing required for the access log (unless the site has enabled the access log anyway). Statistics Granularity does have a Views integration but it does not require any modules other than the core Statistics module. I have also created an issue under Statistics Advanced to either add an integration with Statistics Granularity or merge in its functionality: #770688: Statistics Granularity Integration.

igorik’s picture

Hi BassistJimmyJamm thank you for explaing differences.

have a nice day
Igor

rsevero’s picture

On #811942-6: Diff between this and Statistics Granularity:

I believe your remarks are all correct. You mention that not using the access log was part of the original requirements. I would like to understand this better.

Was it a performance requirement? Something in the line of: the Statistics module with the access log turned on is too slow, we need something faster? Or the requirement had some other motivation?

Trying to analyse the performance side of this: I see that you exchanged one INSERT with 8 fields inserted and 6 function calls (the Statistics module solution) for one UPDATE with 3 fields updated and no function call (the Statistics Granularity solution). It surelly looks like your solution can be faster (I'm not sure about what is the trade off between INSERTs and UPDATEs though). Have you got any benchmarks? I would like to know what kind of difference we can get with the Statistics Granularity trade off.

There should be noted that the Statistics module access log option records much more info besides the ones necessary to reproduce Statistics Granularity extra info as all those extra fields inserted and function calls aren't there for nothing ;) :

  • referer uri,
  • ip address from where the access were made,
  • user uid,
  • session id,
  • timer read representing the time expend by the server to produce the page and finally
  • the time when the log was recorded.

I mention this because I'm afraid that many people that use the Statistics Granularity module might end up also using the Statistics module access log option because of the extra info so they end up with no performance increase at all.

And most important of all, I'm wondering if there really is the need for two Statistics module enhancers. Maybe we could join forces?

BassistJimmyJam’s picture

I agree that the data in the access log can be very useful and I'm not in any way saying that it shouldn't be used. The original request for the module required that it worked without the access log because the client did not use it and had no need for it. They did not want to have to enable the access log in order to get the simple functionality provided by Statistics Granularity.

I did run some benchmarks and using Statistics Granularity with the access log off was faster than just Statistics with the access log on. However, I don't appear to have those numbers anymore.

I agree that one module is better than two. I think if we could find some way to get Statistics Pro to work without using the access log, we could get the best of both worlds. Perhaps using the day count in the node_counter table and getting the difference from the last cron run would work.

Another requirement for Statistics Granularity is that it require as few modules as possible. Would it be possible to not require Charts and not provide some of the reports that are currently provided by default unless it is installed?

rsevero’s picture

Thanks for clarifying the original requirements you dealt with.

I'm sorry to hear you don't have the benchmarks numbers anymore. In case you find them, please let me know.

Statistics Pro can work without the access log. Unfortunately it can't produce statistics equivalent to the ones Statistics Granularity does. With access log or without it. I said before that it could. I was wrong.

But I believe we could make it produce similar statistics. I believe we could have problems with the "difference from the last cron run" method you proposed because we could end up losing some updates because of different run times for cron from the Statistics module and the Statistics Pro module.

My first idea is to create a new table, something like node_counter_day with just 3 fields:

  • nid
  • day
  • counter

This table would hold info on day counts per node for some configurable time. From this data we would be able to produce statistics similar to the ones Statistics Granularity does.

The catch that would make it faster than Statistics + Statistics Granularity is that we would recommend users to completely disable the "Content viewing counter" from the Statistics module. We would stop doing 2 UPDATEs that affect 5 fields and put in place 1 UPDATE changing only 1 field.

To even enhance more we could work with two tables: something like node_counter_today and node_counter_archive. The current updates would happen on node_counter_today which would be a really small table (in MySQL smaller = faster). The node_counter_archive table would be accessed only for reading during statistics presentation and for writing once a day when moving data from node_counter_today to node_counter_archive.

I envision this would be really fast.

We would also need to incorporate Statistics Advanced bug fixes in the code.

About the Charts module requirement, please see #819272: The pie charts at "common" statistics page don't make sense. To resume, I believe the Charts requirement will disappear in the short term. When some kind of chart get properly reimplemented in Statistics Pro, it could certainly be optional.

There might be modules out there that would be affected by the disable the "Content viewing counter" from the Statistics module idea. I really don't know the size of the impact it would have.

cerup’s picture

rsevero, I think your suggestions in your latest comment sound the best.

Ideally, the fewest dependencies possible is best. I even like the idea of being able to remove the 'content viewing counter' entirely; would this mean there would also be a chance of not even needing Statistics module turned on (since that would be great)?

I think the idea of having two tables (day and archive) where days can be saved for X amount (configured by user) and archive is run once per day (like you said) would be a quick and effective implementation.

The charts aspect could probably be broken off into a submodule so that it's not required.

rsevero’s picture

After more thought I'm considering the possibility of using rrdtool as a data consolidation tool.

The main data would be recorded through INSERTs consisting of nid and timestamp. INSERTs in a simple table like this without indexes will be really fast.

For archive, rrdtool would be used.

cerup’s picture

Wouldn't that require used to install rrdtool? If that's the case, it seems we're getting away from what this module was originally intended for. There's already an rrdtool module I believe also.

rsevero’s picture

Wouldn't that require used to install rrdtool?

Yes it would, in some form.

If that's the case, it seems we're getting away from what this module was originally intended for.

Do you think so? Could you please elaborate more on this idea?

There's already an rrdtool module I believe also.

Really good to know. I will take a better look at it latter.

rsevero’s picture

Status: Active » Closed (works as designed)

Please reopen if new info or a patch is available.

chrowe’s picture

Just found this in my search for a statistics module and thought I would suggest using the data module http://drupal.org/project/data to store your data. I know it adds a dependance, but It seems like the point of modules it to brake down functionality into plugable components.