Mediawiki has a fine idea to add the name of the function issuing a query as a SQL comment when running the query. This makes SHOW PROCESSLIST much more usable. Thats the mysql command to see all running queries. Further, this makes it easier to analyze slow query logs and such.

db_query() or _db_query() could do this themselves and not involve any outside code with just a call to debug_backtrace().

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

kbahey’s picture

Status: Active » Needs review
FileSize
746 bytes

This will save me a lot of greps to find the offending queries when doing tuning.

Here is a patch that seems to work.

Don't see why this should not be in Drupal 6. It is non-intrusive at all.

webchick’s picture

Status: Needs review » Needs work

That's not the only place there are queries.

See http://drupal.org/node/168812. In fact, if we could put this in that logic there, it might be nice, so we don't have to do backtraces in two+ places.

moshe weitzman’s picture

Version: 7.x-dev » 6.x-dev

@kbahey - do you think we should do this always, or only if dev_query variable is TRUE? I could go either way. I lean towards within dev_query block.

@webchick. i don't fully understand your comment. we issue queries that don't run through db_query()? ... i don't think consolidating backtrace info will be helpful since we are already in a driver specific code. the driver knows to retreat 2 steps for mysql, for example. i don't feel that strongly though.

webchick’s picture

@moshe:

i don't fully understand your comment. we issue queries that don't run through db_query()?

Yes. Several.

From http://drupal.org/files/issues/db_backtrace_26.patch:

$query_functions = array('db_query', 'pager_query', 'db_query_range', 'db_query_temporary', 'update_sql');
webchick’s picture

You're right that we can't consolidate it though.. the code there only executes on error condition. But you can probably swipe the logic to make sure it prints the right function.

kbahey’s picture

@moshe

I see no harm in having it always there. If you have a large live site that is creaking under load, and no devel, you are out of luck.

The only thing I am not sure of is whether David Strauss' master/slave thing parses the query to determine if it is a read or write one. If it does, then the /* ... */ part has to be stripped out or ignored.

@webchick

You are right about this being scattered over all those functions. Should we go and add these two lines to each of these functions?

moshe weitzman’s picture

I just noticed that Mediawiki adds the logged in user's name as well (if available).

@kbahey - you don't really need devel.module. You just need to set that variable via command line or via previewing a php node or other. Just some thoughts ...

kbahey’s picture

Status: Needs work » Needs review
FileSize
841 bytes

This version addresses moshe's two comments:

- Makes this conditional on the dev_query variable being set.

- Adds the user name before the function name, this includes anonymous too (why not ...)

Need opinions from webchick and others on how best to handle the other functions. We can make these few lines a centralized function and call it from all the other 4 functions.

moshe weitzman’s picture

Status: Needs review » Needs work

If you put the comment in _db_query instead, you will capture other queries like db_query_range()/pager_query()

kbahey’s picture

Status: Needs work » Needs review
FileSize
2.24 KB

Here it is. Seems to work fine for me.

More testing needed.

moshe weitzman’s picture

Title: Add function name as a comment in all SQL queries » Add function name and username as a comment in all SQL queries
Status: Needs review » Reviewed & tested by the community

I verified that this patch doesn't break anything and that the comment has the right data in it. Others can also verify by turning on devel's query log. devel will naturally strip out the comment for presentation after this is committed.

Gábor Hojtsy’s picture

Status: Reviewed & tested by the community » Needs work

Plus:

- Good that we only run this when devel is on.
- SQL rewrites are thankfully already done by this stage, so we don't need to be afraid of these comments messing up the rewrites.

Minus:

- There are some code style problems with concatenation.

kbahey’s picture

FileSize
2.23 KB

Code style fixed.

Should be ready to go.

kbahey’s picture

Status: Needs work » Needs review
moshe weitzman’s picture

Status: Needs review » Reviewed & tested by the community

i retested and all is well.

Gábor Hojtsy’s picture

Priority: Normal » Critical
Status: Reviewed & tested by the community » Active

Doh, although I just committed this patch, it occurred to me that this opens an attack vector with usernames. If I have "Gábor */ DROP TABLE system;" as my username, and the site has devel module turned on for whatever reason, I can drop the table. So let's close this attack option.

moshe weitzman’s picture

Good thinking. Yet neither of those characters are allowed by in my testing - i.e. blocked by user_validate_name(). still need to sanitize the username?

Gábor Hojtsy’s picture

Priority: Critical » Normal

Hm, I also looked into the user name validation code, and it seems these chars are not allowed, so we are safe there. So tasks:

- let's document this in the mysql and mysqli _db_query()
- why was the pgsql code left out of the fun?? pgsql does support /* */ types of comments

DrupalTestbedBot tested kbahey's patch (http://drupal.org/files/issues/mysql-func-name-2.patch), the patch failed. For more information visit http://testing.drupal.org/node/107

moshe weitzman’s picture

It failed because we already applied the patch. In the future, that won't be a problem since we won't do anything until bot says the patch his clean.

DrupalTestbedBot tested kbahey's patch (http://drupal.org/files/issues/mysql-func-name-3.patch), the patch failed. For more information visit http://testing.drupal.org/node/131

kbahey’s picture

Status: Active » Needs review
FileSize
1.83 KB

This patch documents the changes to the two files (mysql, and mysqli).

I don't use PostgreSQL, so someone who does can add that.

moshe weitzman’s picture

Category: feature » bug
Status: Needs review » Reviewed & tested by the community

I massaged the help text slightly.

More importantly, I removed a t() call which was breaking anonymous page views with devel turned on. t() isn't available yet. If Gabor ha a better suggestion that t() [like what the installer does], then please do make the needed change.

moshe weitzman’s picture

FileSize
2.22 KB

oops - patch here.

Heine’s picture

Hm, I also looked into the user name validation code, and it seems these chars are not allowed, so we are safe there.

We should not rely on the username validation code; I'm sure their are various ways to create users (contrib?, imports?) that may not run the validation.

moshe weitzman’s picture

FileSize
2.41 KB

Agreed .. Here is a hardened version. See the str_replace()

Gábor Hojtsy’s picture

Status: Reviewed & tested by the community » Needs work

- If you only need t() for the anonymous name, then this is not a real problem IMHO. Feel free to leave that out, but fix the comment to suggest that it was actually removed from around Anonymous. Now the comment hangs in the air.
- Also to be sure that values are properly sanitized, I would put the str replace around the ternary operator, not only on the username, but also on the anonymous name. There is nothing stopping admins from screwing up all their SQL queries with a fun anonymous user name otherwise.

moshe weitzman’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
2.53 KB

Addressed both of Gabor's points.

Gábor Hojtsy’s picture

Status: Reviewed & tested by the community » Fixed

Thanks, committed.

Anonymous’s picture

Status: Fixed » Closed (fixed)