While developing the linkchecker module I thought about a way how to clean tables very quickly + resetting the auto increment field.

This is all no problem in D5 and D6 with TRUNCATE TABLE 'foo' and MySQL and PgSQL support TRUNCATE. But if it comes to SQLite - this database does not support the truncate statement and you need to delete all items with DELETE what is often suboptimal and very slow + it does not reset the auto increment field.

I don't like to write DELETE statements for tables having 1.000.000 or more entries - for clearing a table and I think we need to have a function like db_truncate_table('foo') that makes it easy to have a truncate that is DB independent. We could use TRUNCATE for all servers supporting it and DELETE on SQLite. Otherwise we need to write DB depended statements :-(. I don't like to do this...

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Crell’s picture

I'm open to this, although of course it would have to be done in the same trivial-wrapper approach as other DB actions. We'd have to document that the reset-of-autoinc behavior is required, so SQLite can simulate it however it wants to.

hass’s picture

I have NULL experience with SQLite and no idea how to reset the auto-increment plus I'm not able to provide a patch as I'm not yet worked with anything of the D7 DB changes, yet. :-(

Damien Tournoud’s picture

Status: Active » Needs review
FileSize
3.48 KB

SQLite does the right thing: if you issue a DELETE without a WHERE clause, it just drops the table and recreates it, just what MySQL and PostgreSQL do with TRUNCATE.

It is better in my opinion to simply special case the "DELETE without a WHERE clause" argument to use TRUNCATE on engines that support it.

Crell’s picture

Status: Needs review » Needs work

Is TRUNCATE part of the SQL 2003 spec, or is it a common extension? If the latter, then I'd rather put the delete-only implementation in the base class and the TRUNCATE version in the db-specific drivers. If the former, then the current setup is fine.

We do want new unit tests for the new code path either way.

Damien Tournoud’s picture

Status: Needs work » Needs review
FileSize
4.99 KB

Added a test that the serial columns are reset. This failed on SQLite (fixed). Should fail on PostgreSQL, that doesn't reset serial columns when TRUNCATE is used (todo: fix that).

Damien Tournoud’s picture

@Crell: TRUNCATE is part of the very new SQL:2008, nothing before that.

dropcube’s picture

Reviewed the patch, seems OK, but won't be good to have also an utility db_truncate_table() to explicitly truncate (or empty) a table and reset the serial columns?

Dries’s picture

I think that the practical differences between 'DELETE FROM foo' and 'TRUNCATE TABLE foo' warrant that we make this 2 functions.

- If the auto increments are not reset when DELETE is used, than that might be a feature that we like to maintain. There might be a valid case for deleting everything and maintaining the auto increments?

- The MySQL implementation of TRUNCATE TABLE ignores locks, whereas DELETE doesn’t. According to SQL:2008, truncate is a schema-change statement hence it should not respect locking. This might have unwanted side-effects. For example, if db_delete() uses TRUNCATE TABLE, than db_delete() might cause the transaction support to fail?

- Truncate requires different permissions -- in MySQL, it requires DROP-permission; e.g. schema level permissions.

- The SQL:2008 specification says that TRUNCATE TABLE is illegal on a referenced table of a foreign key. If we're adding support for foreign keys, and we're attempting to do so, DamZ's patch might have some issues.

Damien Tournoud’s picture

Discussed that with Dries, and we concluded that due to implementation differences between engines (and other smaller concerns, like keeping the autoincrement and requirement a different set of permissions), we cannot simple special case DELETE with no condition.

There are some use cases for an explicit truncate operation (like dropping huge search tables, or cache tables), so we decided to implement it. Here is the patch, already tested on SQLite.

Status: Needs review » Needs work

The last submitted patch failed testing.

dropcube’s picture

Status: Needs work » Needs review
FileSize
6.21 KB

There is a minor syntax error in the query:

TRUNCATE FROM {test} - Array ( ) SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'FROM simpletest591157test' at line 1

The patch fixes it.

Dries’s picture

This looks good to me.

I'm not sure truncate is supposed to return status codes -- on MySQL it does but on PostgreSQL it might not. It is not a show stopper for me -- just wanted to point it out. We might be able to remove the return option stuff.

Crell’s picture

Status: Needs review » Needs work

The docblock for the $truncateClass variable still says "Delete class". Other than that this looks good on visual inspection.

We don't currently deal with status codes in most other situations, since even with PDO they're still not as standardized as I'd like. I believe merge queries also return "whatever the DB gives us" and then documents the return as "undefined", so I'm OK with doing the same here. Let's fix the docblock and then this has my +1 to RTBC.

Damien Tournoud’s picture

Status: Needs work » Needs review
FileSize
6.04 KB

Fixed the docblock. Seems ready to go.

moshe weitzman’s picture

Code looks good. Do we have any use cases in core? I can't think of one.

Dries’s picture

Status: Needs review » Fixed

Great. Committed to CVS HEAD. We can tweak as necessary. Thanks!

hass’s picture

Title: Implement db_truncate_table() » Implement db_truncate()

Great, THX!

catch’s picture

For use cases in core I can only think of:

      if ($cid == '*') {
        db_delete($table)->execute();
      }
in cache_clear_all().

#452936: Use db_truncate() in cache_clear_all()

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.