While developing the linkchecker module I thought about a way how to clean tables very quickly + resetting the auto increment field.
This is all no problem in D5 and D6 with TRUNCATE TABLE 'foo'
and MySQL and PgSQL support TRUNCATE. But if it comes to SQLite - this database does not support the truncate statement and you need to delete all items with DELETE
what is often suboptimal and very slow + it does not reset the auto increment field.
I don't like to write DELETE statements for tables having 1.000.000 or more entries - for clearing a table and I think we need to have a function like db_truncate_table('foo')
that makes it easy to have a truncate that is DB independent. We could use TRUNCATE for all servers supporting it and DELETE on SQLite. Otherwise we need to write DB depended statements :-(. I don't like to do this...
Comment | File | Size | Author |
---|---|---|---|
#14 | 396578-implement-truncate-query.patch | 6.04 KB | Damien Tournoud |
#11 | 396578-implement-truncate-query-11.patch | 6.21 KB | dropcube |
#9 | 396578-implement-truncate-query.patch | 6.05 KB | Damien Tournoud |
#5 | 396578-truncate-query.patch | 4.99 KB | Damien Tournoud |
#3 | 396578-truncate-query.patch | 3.48 KB | Damien Tournoud |
Comments
Comment #1
Crell CreditAttribution: Crell commentedI'm open to this, although of course it would have to be done in the same trivial-wrapper approach as other DB actions. We'd have to document that the reset-of-autoinc behavior is required, so SQLite can simulate it however it wants to.
Comment #2
hass CreditAttribution: hass commentedI have NULL experience with SQLite and no idea how to reset the auto-increment plus I'm not able to provide a patch as I'm not yet worked with anything of the D7 DB changes, yet. :-(
Comment #3
Damien Tournoud CreditAttribution: Damien Tournoud commentedSQLite does the right thing: if you issue a DELETE without a WHERE clause, it just drops the table and recreates it, just what MySQL and PostgreSQL do with TRUNCATE.
It is better in my opinion to simply special case the "DELETE without a WHERE clause" argument to use TRUNCATE on engines that support it.
Comment #4
Crell CreditAttribution: Crell commentedIs TRUNCATE part of the SQL 2003 spec, or is it a common extension? If the latter, then I'd rather put the delete-only implementation in the base class and the TRUNCATE version in the db-specific drivers. If the former, then the current setup is fine.
We do want new unit tests for the new code path either way.
Comment #5
Damien Tournoud CreditAttribution: Damien Tournoud commentedAdded a test that the serial columns are reset. This failed on SQLite (fixed). Should fail on PostgreSQL, that doesn't reset serial columns when TRUNCATE is used (todo: fix that).
Comment #6
Damien Tournoud CreditAttribution: Damien Tournoud commented@Crell: TRUNCATE is part of the very new SQL:2008, nothing before that.
Comment #7
dropcube CreditAttribution: dropcube commentedReviewed the patch, seems OK, but won't be good to have also an utility db_truncate_table() to explicitly truncate (or empty) a table and reset the serial columns?
Comment #8
Dries CreditAttribution: Dries commentedI think that the practical differences between 'DELETE FROM foo' and 'TRUNCATE TABLE foo' warrant that we make this 2 functions.
- If the auto increments are not reset when DELETE is used, than that might be a feature that we like to maintain. There might be a valid case for deleting everything and maintaining the auto increments?
- The MySQL implementation of TRUNCATE TABLE ignores locks, whereas DELETE doesn’t. According to SQL:2008, truncate is a schema-change statement hence it should not respect locking. This might have unwanted side-effects. For example, if db_delete() uses TRUNCATE TABLE, than db_delete() might cause the transaction support to fail?
- Truncate requires different permissions -- in MySQL, it requires DROP-permission; e.g. schema level permissions.
- The SQL:2008 specification says that TRUNCATE TABLE is illegal on a referenced table of a foreign key. If we're adding support for foreign keys, and we're attempting to do so, DamZ's patch might have some issues.
Comment #9
Damien Tournoud CreditAttribution: Damien Tournoud commentedDiscussed that with Dries, and we concluded that due to implementation differences between engines (and other smaller concerns, like keeping the autoincrement and requirement a different set of permissions), we cannot simple special case DELETE with no condition.
There are some use cases for an explicit truncate operation (like dropping huge search tables, or cache tables), so we decided to implement it. Here is the patch, already tested on SQLite.
Comment #11
dropcube CreditAttribution: dropcube commentedThere is a minor syntax error in the query:
TRUNCATE FROM {test} - Array ( ) SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'FROM simpletest591157test' at line 1
The patch fixes it.
Comment #12
Dries CreditAttribution: Dries commentedThis looks good to me.
I'm not sure truncate is supposed to return status codes -- on MySQL it does but on PostgreSQL it might not. It is not a show stopper for me -- just wanted to point it out. We might be able to remove the return option stuff.
Comment #13
Crell CreditAttribution: Crell commentedThe docblock for the $truncateClass variable still says "Delete class". Other than that this looks good on visual inspection.
We don't currently deal with status codes in most other situations, since even with PDO they're still not as standardized as I'd like. I believe merge queries also return "whatever the DB gives us" and then documents the return as "undefined", so I'm OK with doing the same here. Let's fix the docblock and then this has my +1 to RTBC.
Comment #14
Damien Tournoud CreditAttribution: Damien Tournoud commentedFixed the docblock. Seems ready to go.
Comment #15
moshe weitzman CreditAttribution: moshe weitzman commentedCode looks good. Do we have any use cases in core? I can't think of one.
Comment #16
Dries CreditAttribution: Dries commentedGreat. Committed to CVS HEAD. We can tweak as necessary. Thanks!
Comment #17
hass CreditAttribution: hass commentedGreat, THX!
Comment #18
catchFor use cases in core I can only think of:
#452936: Use db_truncate() in cache_clear_all()