When updating a 6.16 database to 7.x.dev system_update_7008 proceeds till the update query then breaks.

This is occuring with php.5.29 and mysql 5.1 on xampp

the query uses aliases and was one damien had made an issue with on using aliases in updates but can find it again. The patch in that issue was approved and backported to d6. The current query is

db_query("UPDATE {poll_votes} SET chid = (SELECT chid FROM {poll_choices} c WHERE {poll_votes}.chorder = c.chorder AND {poll_votes}.nid = c.nid)");

The query I tried was

db_query("UPDATE {poll_votes} SET chid = (SELECT chid FROM {poll_choices} WHERE {poll_votes}.chorder = {poll_choices}.chorder AND {poll_votes}.nid = {poll_choices}.nid)"); 

but that didn't work either. I'm not sure if this is a mysql 5.1 issue or not. Right now it is not working, haven't tried it yet on mysql 5.0

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

catch’s picture

This should be db_update(), which I'm pretty sure can handle subqueries.

ctmattice1’s picture

When someone has the time, can you explain what I'm doing wrong?

I understand what

db_query("UPDATE {poll_votes} SET chid = (SELECT chid FROM {poll_choices} c WHERE {poll_votes}.chorder = c.chorder AND {poll_votes}.nid = c.nid)");

does but I'm trying to convert it to the new db functions and am having problems. All I've managed to come up with so far is

$result = db_select('poll_choices','pc')
      ->addField('pc', 'chid')
      ->join('poll_votes', 'pv', 'pc.chorder = pv.chorder AND pc.nid = pv.nid')
      ->condition('pc.chorder', 'pv.chorder')
      ->condition('pc.nid', 'pv.nid');
    db_update('poll_votes')
      ->fields(array('chid' => $result))
      ->condition('pc.chorder', 'pv.chorder')
      ->condition('pc.nid', 'pv.nid')
      ->execute();

Still get errors with " join on a non-object" so I know i'm missing something. It's be easy with a single db table but I can't find anywhere how to set up a query using 2 or more tables. The only thing I can think of, I haven't tried, is using the ->where() clause but that because I haven't figured that out either. Can't seem to locate anything in the docs or when looking at the db api.

AaronBauman’s picture

ctmattice1:
addField returns the field alias (string), breaking the object chain.
you want fields, which returns $this and allows the chain to continue.

ctmattice1’s picture

Status: Active » Needs review
FileSize
2.03 KB

aaronbauman:
Thanks for pointing that out. strange how you miss stuff and try to get too complicated.

Figured it out, here's a patch

catch’s picture

Status: Needs review » Needs work

This is now reverting changes from system_update_7053(), please run cvs up on your head install. Also we should remove the old code rather than commenting it out.

ctmattice1’s picture

Status: Needs work » Needs review
FileSize
1.13 KB

new patch against head. comment removed

moshe weitzman’s picture

Status: Needs review » Needs work

Thats running many queries where we used to run one. I'm pretty sure DBTNG can replicate the single query we had before.

ctmattice1’s picture

$query = db_select('poll_choices','c')
->fields('c', array('chid'))
->join('poll_votes', 'v', 'c.nid = v.nid')
->condition('chorder', 'v.chorder')
->condition('nid', 'v.nid');

Can some please explain what I'm doing wrong, I keep coming up with call to member function ????? on non-object.

Unless I'm missing something I'm selecting the table "poll_choices" and assigning it the alias "c", then I want to select "chid" using the fields() function so it can continue the chain. I then want to inner join "poll_votes" assigning it an alias of "v" with only the rows that match "pole_choices.nid = pole.votes.nid" be used then based on the condition that "chorder" form pole_choices equals v.chorder (pole_votes.chorder using the alias "v" for the table "pole_votes) and also requestion that the select also match "nid" from pole_choices and v.nid

I've tried all kinds of various things including using where() but I always come up with this error. This is why I submitted the patch in #6 that has multiple queries.

Stevel’s picture

The join-statement returns the actual table alias that is generated (when the given alias is already in use), instead of a query object. See also the Dynamic queries documentation at http://drupal.org/node/310075

AaronBauman’s picture

After some trial and error, here's what I came up with to create the unified query:

  $subquery = db_select('poll_choice','pc')
      ->fields('pc', array('chid'))
      // Have to use where() instead of condition() since we're using column names, not values
      ->where('pc.chorder = poll_vote.chorder')
      ->where('pc.nid = poll_vote.nid');
  $query = db_update('poll_vote')
      // Have to use expression() instead of fields(), again since we're using a subquery, not values
      ->expression('chid' => '(' . $subquery . ')'));

I'll roll a patch with this next week if no one else gets around to it.

catch’s picture

Issue tags: +D7 upgrade path
catch’s picture

Component: update system » poll.module
ctmattice1’s picture

Component: poll.module » update system

catch,

Changing tag back to update system since the change require modification to the system.install file and not poll.install

catch’s picture

Component: update system » system.module

In that case it's system.module - the update system is for the actual process of updating Drupal.

ctmattice1’s picture

Tried #10 and many different variations of it today. All of them fatal-ed on ->execute()

aaronbauman: Thanks for pointing out expression()

->expression('chid' => '(' . $subquery . ')'));

it has one too many ending ) though and table names are plural. per the docs I tried a couple of things

1st - ->expression('chid' , '(' . $subquery . ')');
2nd - ->expression(array('chid' => '(' . $subquery . ')'));

neither worked. I attempted to display the $subquery results before running the update query with

  $subquery = db_select('poll_choices','pc')
      ->fields('pc', array('chid'))
      // Have to use where() instead of condition() since we're using column names, not values
      ->where('pc.chorder = poll_votes.chorder')
      ->where('pc.nid = poll_votes.nid');
      ->execute() ; print_r($subquery): die;

the update bombed out on remaining system updates then picks up around sequence #113 out of #117. Also tried it after execution of the update query with same results.

catch’s picture

I think we should go with #6 to fix the upgrade path, it's poll votes, not {node} or {term_node}.

Then also open a new issue for subquery support in updates. This is the issue where it was added for insertQuery #481288: Add support for INSERT INTO ... SELECT FROM ..., I haven't checked that issue but I don't remember it being added for updates, so that might be a separate, critical, omission - but it needs its own issue to sort it out.

chx’s picture

Status: Needs work » Needs review
FileSize
910 bytes

Breaking out of SQL and into PHP is not the best idea. (and there are severe code style issues too) Edit: I wrote a whole another patch instead. Which is not the nicest patch either but it'd be a feature request to support subqueries-in-updates and figuring out table names there would be fun. While not nice it does not breach any 'contracts' -- still the update is ran through db_update and there are no variables in the SELECT statement.

Stevel’s picture

I've managed to get the query from #10 working, so here is another patch which uses db_select for the subquery. Tried to upgrade with a poll, and didn't get any errors, so I assume the query works :)

Status: Needs review » Needs work

The last submitted patch, 817216-update-query-with-subquery.patch, failed testing.

Stevel’s picture

Reroll, don't know what happened here.

Stevel’s picture

Status: Needs work » Needs review
ctmattice1’s picture

#20 works and it doesn't.

If the subquery finds a match then it completes, if no match is found the update query fails and bombs out. This is how I found the issue initially. Probably one of those edge case scenarios that dates back to databases in existence since 4.7 and updated. The particular database I use is one of these edge cases, v4.7 initially solwly updated through 5.22 then updated to 6.17.

Getting 6.17 -> 7 may be a edge case in this circumstance but i'm sure others will have the same situation so i've purposely used it. I really don't use the poll module at all any more but left over tables I kept since the module is still active. It is a small database set with maybe 12 entries in poll_choices on only 2 in poll_votes.

The original values in poll_votes for chorder are -1

no match to chorder in poll_choices, update fails and further system updates stop around sequence #28 then update.php starts again around sequence # 90 or greater.

If I change one of the chorder column values in poll_votes to match then db_update is happy and update.php runs it course as it should except for #826640: Update results page missing session data, causing "Undefined index: update_success" notice and "aborted prematurely" message which i've ignored as a nuisance until update.php is debugged

#6 works period.

haven't tried chx's patch in #17 yet, some time today i'll give it a shot but think it will do the same thing, could be wrong.

chx’s picture

Oh invalid votes! Right. I am not going with Stevel's patch , I used mine. His requires an intimate knowledge of DBTNG innards -- namely that we call down prefxitables on the whole string instead of prefixing on join() -- while mine is uglier but it does not violate encapsulation.

Invalid votes are handled by this patch by allowing NULLs first then deleting them and then changing the table. Oh and the index is added after the second step just for kicks.

chx’s picture

aand less tpyos.

chx’s picture

FileSize
1016 bytes

But this will be superb slow because some engines do not take alter table lightly and do ALTER TABLE with practically rebuilding the table. Let's use COALESCE instead.

Stevel’s picture

Status: Needs review » Reviewed & tested by the community

This looks good. Reviewed and extensively tested.

I wasn't even aware I was relying on table prefixing happening in the tostring of a query object. Thanks chx for pointing that out.

ctmattice1’s picture

I'm not sure we want to delete the votes. In my case these were from anonymous users and are still votes. visit http://authentic-empowerment.net/node/1209

The two votes are displayed properly. If we delete them then poll voting is useless and inaccurate.

Stevel’s picture

ctmattice1: These 'invalid' votes are votes in the database for non-existing choices and should not have existed in the database in the first place. in D6.17 these votes are deleted from the database when the choice is deleted, but this could be a left-over from an older version.

edit: votes from anonymous users are not deleted of course.

ctmattice1’s picture

Status: Reviewed & tested by the community » Needs review
FileSize
1.08 KB

Here's patch #25 without the deleting the votes.

ctmattice1’s picture

Stevel:"but this could be a left-over from an older version."

Yep, these are from somewhere in 5.x, currently 5.22 is one of my production sites which this came from. I updated this to 6.17 which I'm using to run 6 -> 7 update.

Stevel’s picture

ctmattice1: could you give a reason to keep these votes in the database, given that these votes don't correspond to any of the presented options?

The votes that are actually corresponding to one of the choices (like those in your example) are not deleted by #25.

edit: could you check the poll_choices after the D5.22->D6.17 upgrade and see if there is a record with chid == 0 in there?

chx’s picture

Status: Needs review » Reviewed & tested by the community

#25 is still RTBC. There is no point in keeping those votes because there is absolutely no way to figure them out which choice they belong to after chorder is dropped if chid is zero.

Damien Tournoud’s picture

Agreed with #32. It's not like we can do anything with those broken rows.

ctmattice1’s picture

Status: Reviewed & tested by the community » Needs review

My Bad.

I looked at the tables and saw 0 poll_vote. Did not bother looking further. Disregard #29 even with no rows in poll_vote after the update it is still picking up the votes in poll_choice. RTBC #25

Stevel’s picture

Status: Needs review » Reviewed & tested by the community

Back to RTBC then for #25

Damien Tournoud’s picture

I would also would like to see this move out of the system module. I'm happy to help maintain the base system, but the poll module has nothing to do with system :)

ctmattice1’s picture

How this look

Status: Reviewed & tested by the community » Needs work

The last submitted patch, Move_system_update_7008_to poll_update.diff, failed testing.

ctmattice1’s picture

Hmmm,

stupid copy/paste try again

ctmattice1’s picture

Status: Needs work » Needs review
FileSize
2.97 KB

sorry forgot to change status and shortened name

Status: Needs review » Needs work

The last submitted patch, Move_system_update_7008.diff, failed testing.

ctmattice1’s picture

Status: Needs work » Needs review
FileSize
3.02 KB

Worked on my local copy, strange.

Status: Needs review » Needs work

The last submitted patch, Move_system_update_7008_1.diff, failed testing.

ctmattice1’s picture

Status: Needs work » Needs review
FileSize
3.04 KB

I don't believe you Mr. Bot, see if you like this one

marcingy’s picture

Status: Needs review » Reviewed & tested by the community

Is the patch in #25 that was marked RTBC previously plus the existing code in system install moved to poll install.

Stevel’s picture

Status: Reviewed & tested by the community » Needs work

Only a small style remark: there is some extra whitespace at the end of the third line of the comment.

And not sure if the comment for eg "rename table" is necessary, as the function calls are pretty self-explanatory.

ctmattice1’s picture

Status: Needs work » Needs review
FileSize
3 KB

Re-rolled with comment changes

Damien Tournoud’s picture

Cannot we safely assume that the tables exist if we move the update function to the poll module where they belong?

ctmattice1’s picture

for some strange reason testing bot did not like poll_update 7001 when I left the db_rename_table functions as they were originally, when I placed then inside the if statement it passed. Seems back in #582948: Improve safety of schema manipulation this was raised and on a failed modify it throws an exception. It worked that way on my local machine, might be a bot issue.

ctmattice1’s picture

re-rolled as damien suggested. Ran a fresh install of HEAD and new update. everything worked as expected, let's see what the testing bot thinks this time.

Status: Needs review » Needs work

The last submitted patch, Move_system_update_7008_4.diff, failed testing.

ctmattice1’s picture

Status: Needs work » Needs review
FileSize
2.84 KB

Last attempt. Double checked, triple checked. Bot on prior failss have bunked me at 1794 so left the system description as was and comment was place inside function. Let's see what happens with this one.

Damien Tournoud’s picture

@ctmattice1: you should not try to edit your patches manually. The patch in #50 doesn't apply because part of the context is missing in the system.install hunk.

ctmattice1’s picture

@Damien: I guess I need to learn netbeans better. I make changes there, copy the files to my drupal cvs folder then use toroiseCVS to run diffs, copy /paste into editpad to make sure it's in unix format. Know there's got to be a easier way just haven't figured it out yet. Blame it on my old line item editor days I guess using the old skool green screen terminals I grew up on. Sheesh you think I'd have learned a better way by now.

Damien Tournoud’s picture

Status: Needs review » Reviewed & tested by the community

#52 works for me.

webchick’s picture

Status: Reviewed & tested by the community » Fixed

Committed to HEAD. Thanks!

ctmattice1’s picture

Title: system_update_7008 does not work » poll_update_7001 does not work
Component: system.module » poll.module
Status: Fixed » Active

Due to a recent commit to table prefixing in databse.inc this update is broke again. If damien or chx is around please look at database.inc and see if it is in the prefixTables function recently committed. The line that gets caught is 399 and has to do with $this->prefixes part of the line. Can't remember the complete error. If this is the proper way to do it then any suggestions as to how to fix chx's fix on the sql statement

db_update('poll_votes')
    ->expression('chid', DatabaseConnection::prefixTables('COALESCE((SELECT chid FROM {poll_choices} c WHERE {poll_votes}.chorder = c.chorder AND {poll_votes}.nid = c.nid), 0)'))
    ->execute();

would be appreciated. I also received database exceptions for db_add_field so will need to re-roll with the if db_table_exists statements.

chx’s picture

Database::getConnection()->prefixTables() I think.

Stevel’s picture

Status: Active » Needs review
FileSize
1.06 KB

Here's a patch

ctmattice1’s picture

Status: Needs review » Reviewed & tested by the community

Works for me

webchick’s picture

Status: Reviewed & tested by the community » Fixed

Ok, committed to HEAD.

But this means we even more urgently need docs for #195416: Table prefixes should be per database connection. We're breaking our own code already, which means we're probably breaking contrib's all over the place as well.

Status: Fixed » Closed (fixed)
Issue tags: -D7 upgrade path

Automatically closed -- issue fixed for 2 weeks with no activity.