When updating sites from version A to version D there is a chance an index was created in version B and removed again in version D. PostgreSQL throws an error if a non-existent index is used in a DROP INDEX. This can be avoided by checking the index exists. PostgreSQL 8.2 introduced an IF EXISTS clause to DROP INDEX.

Ref:
DROP INDEX (8.2)
DROP INDEX (8.1)

Patch to follow.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

sammys’s picture

sammys’s picture

Status: Active » Needs review
FileSize
1.74 KB

Turns out db_drop_primary_key() and db_drop_unique_key() are also affected. I've added _db_index_exists() and used that in all three functions.

We need some reviewing of the patch since it's larger than trivial.

sammys’s picture

Patched code didn't work. Fixed it and rerolled.

webchick’s picture

Version: 6.9 » 7.x-dev
Status: Needs review » Needs work

Ran across the need for this today in #363262: Missing index on url_alias table.

1. Needs to be fixed in D7 first.
2. We already have db_column_exists() and db_table_exists(), so there's no reason to prefix this function with an underscore.
3. The function is missing PHPDoc.

Crell’s picture

webchick asked for my input here. I've no problem with an indexExists() method on the schema API, with a procedural wrapper. It should follow the same logical design as the rest of schema, of course, but I like having a rich set of primitives. :-)

sammys’s picture

Here is a rerolled patch with underscore removed and docs added.

Crell: Are you adding indexExists() or shall I assist?

sammys’s picture

Issue tags: +PostgreSQL

Tagging

Crell’s picture

You can go ahead. :-) A D7 implementation, however, must put the actual logic back in the schema class, and then provide a DB-specific implementation in each DB's schema class. Then add a wrapper function that mirrors the other functions in database.inc.

sammys’s picture

Ok... i'll play with it over the weekend. Been waiting for another opportunity to get my hands dirty on your new DB stuff :)

sammys’s picture

Reposting the patch with the leading underscore removed on the calls to db_index_exists().

Damien Tournoud’s picture

Issue tags: -PostgreSQL

I fail to see how this is database specific. Isn't MySQL also returning some kind of error when you try to DROP a non existing index?

Plus, I'm -1 on this approach. Every inconsistency should throw a warning during the update process. It's the job of the administrator to choose to ignore them or not.

sammys’s picture

Damien,

I agree some feedback is required when the index you want to drop doesn't exist. It could mean duplicate indexes or that some column change operations will fail. In the case of duplicate indexes, it is a performance hit and doesn't break the system. In the case of failing column operations, the failed column operation (such as allowing dupes in a column with unique constraint) would fail showing an error.

To me, the issue is really about where to put the responsibility of error/warning reporting. Update code is the only part knowing the operations being performed and having the ability to report the condition appropriately.

Perhaps we don't bypass reporting of the failed index removal and make it a warning instead. This will allow an administrator to still see there was something fishy. An error on a following operation can also be associated with the warning without looking at code.

How does that sound?

sammys’s picture

I did a little dev on the D7 implementation of db_index_exists(). From my reading MySQL associates indexes with tables and PostgreSQL have them as independent entities. This will mean we have a function prototype of:

db_index_exists($table, $name)

Most of it is implemented. Couldn't easily find information for sqlite. Can anyone offer some suggestions on this?

sammys’s picture

Status: Needs work » Needs review
FileSize
4.12 KB

Here's a preliminary patch. I haven't tested the code yet.

sammys’s picture

Title: DROP INDEX throws an error if index doesn't exist on PostgreSQL » Adding db_index_exists() to DBTNG and updates handling missing indexes properly

Changing the title to match what's now going on in this issue. Still waiting on a review.

Crell’s picture

Status: Needs review » Needs work

db_result() is deprecated. Use ->fetchField() instead.

sammys’s picture

Status: Needs work » Needs review
FileSize
4.17 KB

Rerolled using ->fetchField().

catch’s picture

Issue tags: +Needs tests

Seems like this should add to the database tests. Just checking for an existing and non-existing index ought to work no?

Dries’s picture

Status: Needs review » Needs work

The code looks good to me, but as catch suggests, it needs tests.

sammys’s picture

I'll get to it as soon as I can.

markus_petrux’s picture

Bug in latest patch: $table needs to be escaped somehow when enclosed between {}. Otherwise table prefix substitution will fail.

Fix:

-  return db_query("SHOW INDEX FROM {$table} WHERE key_name = '$name'")->fetchField();
+  return db_query("SHOW INDEX FROM {" . $table . "} WHERE key_name = '$name'")->fetchField();
Josh Waihi’s picture

Issue tags: +PostgreSQL Code Sprint

subscribing and promoting to PostgreSQL Sprint

markus_petrux’s picture

I implemented this method to check for existence of indexes in CCK, which is available in CCK 2.5. Reference: #231453: Allow indexing columns.

But it has been reported (see #562260: content_db_index_exists has wrong syntax for postgres) that the following query doesn't work in PostgreSQL.

db_query("SELECT COUNT(indexname) FROM pg_indexes WHERE indexname = '$name'")

That it should look like this:

db_query("SELECT COUNT(indexname) FROM pg_indexes WHERE indexname = '" . $table . '_' . $name . "_idx'")

Is it possible that the contents of the indexname column varies depending on PostgreSQL versions? I've been unable to find information about how PostgreSQL stores data in this column, so I cannot really tell. :(

I hope the experience collected by the use of such a feature in CCK could potentially help provide a more reliable method for D7.

Does anyone knows more about the format of the indexname column in pg_indexes table throughout the different PostgreSQL versions supported by Drupal itself?

markus_petrux’s picture

LOL... it was sooo much easy. The fact is that the Schema API for PostgreSQL is building the index names like that, so the statement to check if an index exists should be built using the same pattern for index names as the one used when creating the index. The following syntax is then correct:

db_query("SELECT COUNT(indexname) FROM pg_indexes WHERE indexname = '" . $table . '_' . $name . "_idx'")

And the one implemented in the patch in #17 is not, which is like this:

db_query("SELECT COUNT(indexname) FROM pg_indexes WHERE indexname = '$name'")

For references, see methods dropIndex() and _createIndexSql() in class DatabaseSchema_pgsql, implemented in file includes/database/pgsql/schema.inc

I hope that helps here. ;-)

Josh Waihi’s picture

Priority: Normal » Critical
Status: Needs work » Postponed

I think this will be easily done with #302327: Support cross-schema/database prefixing like we claim to in. So lets wait for that. I'm sure this will go in after the freeze too.

sun’s picture

Title: Adding db_index_exists() to DBTNG and updates handling missing indexes properly » db_index_exists() missing, module updates cannot handle indexes properly
Status: Postponed » Active

Absolutely required. Also for the module update numbering scheme we teach in http://api.drupal.org/api/function/hook_update_N/7

markus_petrux’s picture

If that helps, here's the code that's using CCK for a few months already:

/**
 * Checks if an index exists.
 *
 * @todo: May we remove this funcion when implemented by Drupal core itself?
 * @link http ://drupal.org/node/360854
 * @link http ://dev.mysql.com/doc/refman/5.0/en/extended-show.html
 *
 * @param $table
 *   Name of the table.
 * @param $name
 *   Name of the index.
 * @return
 *   TRUE if the table exists. Otherwise FALSE.
 */
function content_db_index_exists($table, $name) {
  global $db_type;
  if ($db_type == 'mysql' || $db_type == 'mysqli') {
    if (version_compare(db_version(), '5.0.3') < 0) {
      // Earlier versions of MySQL don't support a WHERE clause for SHOW.
      $result = db_query('SHOW INDEX FROM {'. $table .'}');
      while ($row = db_fetch_array($result)) {
        if ($row['Key_name'] == $name) {
          return TRUE;
        }
      }
      return FALSE;
    }
    return (bool)db_result(db_query("SHOW INDEX FROM {". $table ."} WHERE key_name = '$name'"));
  }
  elseif ($db_type == 'pgsql') {
    // Note that the index names in Schema API for PostgreSQL are prefixed by
    // the table name and suffixed by '_idx'.
    return (bool)db_result(db_query("SELECT COUNT(indexname) FROM pg_indexes WHERE indexname = '{". $table ."}_{$name}_idx'"));
  }
  return FALSE;
}
moshe weitzman’s picture

use schema api instead of querying db directly

Crell’s picture

Status: Active » Needs review
FileSize
11.21 KB

Chasing HEAD from #17, tidied the code to use the DB API properly (internal calls should always use $connection->query(), never db_query()), and adds unit tests.

Dries’s picture

+++ modules/simpletest/tests/schema.test	2009-12-14 02:47:45 +0000
@@ -1,4 +1,4 @@
-<?php
+g<?php

Looks wrong. I'll correct that manually before committing. Interestingly, all tests pass.

Dries’s picture

Status: Needs review » Fixed

Fixed, and committed to CVS HEAD. Thanks!

Crell’s picture

Weird, but thanks. :-)

Status: Fixed » Closed (fixed)
Issue tags: -Needs tests, -PostgreSQL Code Sprint

Automatically closed -- issue fixed for 2 weeks with no activity.