Image Module Update #7004 Fail

Moving to Core.

I've also got the same issue since upgrading from 7.10 to 7.12

#1353030: Increase length of "alt" and "title" text for images

It hasn't check ALT & TITLE states before updates.

Were these image fields created without alt / title columns? Is that even possible? The schema describes these columns being created so it should be fitting in with them.

Also, marking as major atleast until we figure out what is going on.

I'm trying to test this out and I haven't run into issues so far; I tried creating a fresh 7.10 site install and generated 5k nodes, 250k comments, and 255k images for it; the upgrade didn't choke or shoot the above error. I tried the same by going from 7.0 to 7.12 (atbeit with a much smaller number of images to speed up the test) and didn't run into the issue. When the image field is created, it creates the appropriate alt and title fields as part of the schema to go with it. I have not tried a 6.x -> 7.x upgrade, however. Were the sites that are getting these issues previously Drupal 6 sites?

I tried an upgrade from 6.x -> 7.x. The field schema seems to migrate over as well (though I ran into errors when trying to migrate the field when I first went from 6.24 to 7.0. Once I upgraded to 7.12, the fields migrated correctly). Because I cannot replicate the issue, I cannot figure out what or why this is happening. @Triumphent or @iancm, would you be able to post the schema of your image fields (it should be field_data_field_image) to try and narrow down the issue?

Same problem. Empty page.
Looking at logs.
Logs saying: PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 46277076 bytes) in /srv/www/php/xxx/xxxxxxxx/dev2/includes/theme.inc on line 1398

What it says? All object of Drupal (since version 7? don't know) are stocked in database as serialized PHP object (and binaries mysql fields).
Due to the structure of a PHP serialized object, the number of characters are stocked too. Arrays' size, objects properties count too, and so and so...
Here? It cand find the values's termination of a propertie or key or something else...
So... looking side theme.inc... maybe could say more

@sbree, it does not sound like you have having the same issue; this issue has to do with 'DatabaseSchemaObjectDoesNotExistException' and not allowed memory size error. Try to increase the amount of memory you have allocated for you site.

Et merde...
The returned line (1398) is : return ob_get_clean();
....
Ok, looking MySQL

@BTMash Oh yes sorry, that's not the good post...
But my error is not a misconfiguration of php or mysql, just deserialization of a serialized object, that's the subject of PHP return error "PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 46277076 bytes)."
That's my problem.
Objects are serialized (like theme preference, and others) and stocked in database. Characters re-encoding could break a serialized object.
Re-encoded characters could take 2 bytes in place of 1. That's due to the serialized structure of an object.

Yours is that something is wrong with the update of your database. And with the update engine of your Drupal installation...

@BTMash Do you have access to a database manager?
The field you're trying to change does not exist...
Could post SQL query to create it.

@BTMash That's not correct, but here it is :
ALTER TABLE `field_data_field_image` ADD `field_image_alt` VARCHAR( 128 ) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT 'Alternative image text, for the image’s ’alt’ attribute.'

@sbree, I'm not getting that error. I've been *trying* to get that error because a few others have run into that issue with update_7004 from the image module. If I can replicate it, then there is a chance to figure out what is going on.

@BTMash Ok sorry.
Don't have.
But a problem on serialized object...

What are the modules installed?

@Triumphent, that helps a little, thank you. Would you be post the schema for your image field is (the table name should be 'field_data_')? It would be a lot of help in trying to figure out what is going on.

Upgrade path bugs are critical IMHO.

I just tried to get the behavior that you are describing with no luck. Installed a Drupal 7.9 site (standard install), generated a bunch of content off the core install, updated the codebase to 7.12 and ran upgrade.php. No issues.

The error I receive is similar but it looks like it might be ubercart:

image module

Update #7004
Failed: DatabaseSchemaObjectDoesNotExistException: Cannot change the definition of field <em class="placeholder">field_data_uc_catalog_image</em>.<em class="placeholder">uc_catalog_image_alt</em>: field doesn't exist. in DatabaseSchema_mysql->changeField() (line 449 of...

@iancm, its definitely the same type of error that @Triumphent is getting. Its a field with the ubercart name that has been affected this time. Would either of you be able to list a set of modules that you have installed? I've been hitting walls as far as trying to replicate the issue :(

try to do this:

image.install LINE 428

    foreach ($tables as $table) {
      debug($table);
      debug($alt_column);
      debug($title_column);
      db_change_field($table, $alt_column, $alt_column, $alt_spec);
      db_change_field($table, $title_column, $title_column, $title_spec);
    }

login with phpmyadmin or other tools, export table scheme (without data)

and the error message show immediately ??

Modules currently installed:
admin_menu
block_titlelink
captcha
colorbox
ctools
date
devel
ds (display suite)
entity
field_format_settings
field_multiple_limit
google_analytics
image_resize_filter
insert
megamenu
metatag
module_filter
nivo_slider
omega_tools
pathauto
rules
superfish
taxonomy_display
taxonomy_menu
token
ubercart
views
wysiwyg

@droplet, I added the code to the image.install file and ran the update.php again and nothing appeared on screen. I'm not sure where the debug information is supposed to be displayed or recorded.

@iancm, try this instead:

foreach ($tables as $table) {
drupal_set_message(print_r($table, TRUE));
drupal_set_message(print_r($alt_column, TRUE));
drupal_set_message(print_r($title_column, TRUE));
db_change_field($table, $alt_column, $alt_column, $alt_spec);
db_change_field($table, $title_column, $title_column, $title_spec);
}

Until we get clear steps on how to reproduce using just Drupal core, marking this as "postponed (maintainer needs more info)".

It sounds like this might be coming in from a contributed module that extends core image fields?

Printed from drupal_set_message:

field_data_field_image
field_image_alt
field_image_title
field_revision_field_image
field_image_alt
field_image_title
field_data_uc_product_image
uc_product_image_alt
uc_product_image_title
field_revision_uc_product_image
uc_product_image_alt
uc_product_image_title
field_data_uc_catalog_image
uc_catalog_image_alt
uc_catalog_image_title

@iancm, First off, thank you for trying to help narrow down where the problem might be coming from. You've been patient on the matter and I appreciate it very much. I still cannot replicate the issue but your example proves one thing: the issue is not consistent. All fields prior to uc_catalog_image were likely successful in the migration (or the issue would have started from up field_image. So there is a possibility that the field did not get created according to the field schema. Why or how it is did do this, I am not yet sure. I tried to programmatically create an image field in Drupal 7.0 and 7.9 and they do create the alt and title columns in the schema (with the field name prepended).

Would you be able to try the following for the drupal_set_message stuff that was asked?

foreach ($tables as $table) {
$table_schema = drupal_get_schema($table);
drupal_set_message(print_r($table, TRUE));
drupal_set_message(print_r($table_schema, TRUE));
}

@BTMAsh, I don't mind trying to help out at all. I came to find out the solution to my problem and found out someone already had the same issue. As long as this helps others in the long run I don't mind at all.

Here is the results from the next drupal_set_message:

field_data_field_image
Array ( [fields] => Array ( [entity_type] => Array ( [type] => varchar [length] => 128 [not null] => 1 [default] => ) [bundle] => Array ( [type] => varchar [length] => 128 [not null] => 1 [default] => ) [deleted] => Array ( [type] => int [size] => tiny [not null] => 1 [default] => 0 ) [entity_id] => Array ( [type] => int [unsigned] => 1 [not null] => 1 ) [revision_id] => Array ( [type] => int [unsigned] => 1 [not null] => ) [language] => Array ( [type] => varchar [length] => 32 [not null] => 1 [default] => ) [delta] => Array ( [type] => int [unsigned] => 1 [not null] => 1 ) [field_image_fid] => Array ( [type] => int [not null] => [unsigned] => 1 ) [field_image_alt] => Array ( [type] => varchar [length] => 512 [not null] => ) [field_image_title] => Array ( [type] => varchar [length] => 1024 [not null] => ) [field_image_width] => Array ( [type] => int [unsigned] => 1 ) [field_image_height] => Array ( [type] => int [unsigned] => 1 ) ) [primary key] => Array ( [0] => entity_type [1] => entity_id [2] => deleted [3] => delta [4] => language ) [indexes] => Array ( [entity_type] => Array ( [0] => entity_type ) [bundle] => Array ( [0] => bundle ) [deleted] => Array ( [0] => deleted ) [entity_id] => Array ( [0] => entity_id ) [revision_id] => Array ( [0] => revision_id ) [language] => Array ( [0] => language ) [field_image_fid] => Array ( [0] => field_image_fid ) ) [foreign keys] => Array ( [field_image_fid] => Array ( [table] => file_managed [columns] => Array ( [field_image_fid] => fid ) ) ) [module] => field_sql_storage [name] => field_data_field_image )

field_revision_field_image
Array ( [fields] => Array ( [entity_type] => Array ( [type] => varchar [length] => 128 [not null] => 1 [default] => ) [bundle] => Array ( [type] => varchar [length] => 128 [not null] => 1 [default] => ) [deleted] => Array ( [type] => int [size] => tiny [not null] => 1 [default] => 0 ) [entity_id] => Array ( [type] => int [unsigned] => 1 [not null] => 1 ) [revision_id] => Array ( [type] => int [unsigned] => 1 [not null] => 1 ) [language] => Array ( [type] => varchar [length] => 32 [not null] => 1 [default] => ) [delta] => Array ( [type] => int [unsigned] => 1 [not null] => 1 ) [field_image_fid] => Array ( [type] => int [not null] => [unsigned] => 1 ) [field_image_alt] => Array ( [type] => varchar [length] => 512 [not null] => ) [field_image_title] => Array ( [type] => varchar [length] => 1024 [not null] => ) [field_image_width] => Array ( [type] => int [unsigned] => 1 ) [field_image_height] => Array ( [type] => int [unsigned] => 1 ) ) [primary key] => Array ( [0] => entity_type [1] => entity_id [2] => revision_id [3] => deleted [4] => delta [5] => language ) [indexes] => Array ( [entity_type] => Array ( [0] => entity_type ) [bundle] => Array ( [0] => bundle ) [deleted] => Array ( [0] => deleted ) [entity_id] => Array ( [0] => entity_id ) [revision_id] => Array ( [0] => revision_id ) [language] => Array ( [0] => language ) [field_image_fid] => Array ( [0] => field_image_fid ) ) [foreign keys] => Array ( [field_image_fid] => Array ( [table] => file_managed [columns] => Array ( [field_image_fid] => fid ) ) ) [module] => field_sql_storage [name] => field_revision_field_image )

field_data_uc_product_image
Array ( [fields] => Array ( [entity_type] => Array ( [type] => varchar [length] => 128 [not null] => 1 [default] => ) [bundle] => Array ( [type] => varchar [length] => 128 [not null] => 1 [default] => ) [deleted] => Array ( [type] => int [size] => tiny [not null] => 1 [default] => 0 ) [entity_id] => Array ( [type] => int [unsigned] => 1 [not null] => 1 ) [revision_id] => Array ( [type] => int [unsigned] => 1 [not null] => ) [language] => Array ( [type] => varchar [length] => 32 [not null] => 1 [default] => ) [delta] => Array ( [type] => int [unsigned] => 1 [not null] => 1 ) [uc_product_image_fid] => Array ( [type] => int [not null] => [unsigned] => 1 ) [uc_product_image_alt] => Array ( [type] => varchar [length] => 512 [not null] => ) [uc_product_image_title] => Array ( [type] => varchar [length] => 1024 [not null] => ) [uc_product_image_width] => Array ( [type] => int [unsigned] => 1 ) [uc_product_image_height] => Array ( [type] => int [unsigned] => 1 ) ) [primary key] => Array ( [0] => entity_type [1] => entity_id [2] => deleted [3] => delta [4] => language ) [indexes] => Array ( [entity_type] => Array ( [0] => entity_type ) [bundle] => Array ( [0] => bundle ) [deleted] => Array ( [0] => deleted ) [entity_id] => Array ( [0] => entity_id ) [revision_id] => Array ( [0] => revision_id ) [language] => Array ( [0] => language ) [uc_product_image_fid] => Array ( [0] => uc_product_image_fid ) ) [foreign keys] => Array ( [uc_product_image_fid] => Array ( [table] => file_managed [columns] => Array ( [uc_product_image_fid] => fid ) ) ) [module] => field_sql_storage [name] => field_data_uc_product_image )

field_revision_uc_product_image
Array ( [fields] => Array ( [entity_type] => Array ( [type] => varchar [length] => 128 [not null] => 1 [default] => ) [bundle] => Array ( [type] => varchar [length] => 128 [not null] => 1 [default] => ) [deleted] => Array ( [type] => int [size] => tiny [not null] => 1 [default] => 0 ) [entity_id] => Array ( [type] => int [unsigned] => 1 [not null] => 1 ) [revision_id] => Array ( [type] => int [unsigned] => 1 [not null] => 1 ) [language] => Array ( [type] => varchar [length] => 32 [not null] => 1 [default] => ) [delta] => Array ( [type] => int [unsigned] => 1 [not null] => 1 ) [uc_product_image_fid] => Array ( [type] => int [not null] => [unsigned] => 1 ) [uc_product_image_alt] => Array ( [type] => varchar [length] => 512 [not null] => ) [uc_product_image_title] => Array ( [type] => varchar [length] => 1024 [not null] => ) [uc_product_image_width] => Array ( [type] => int [unsigned] => 1 ) [uc_product_image_height] => Array ( [type] => int [unsigned] => 1 ) ) [primary key] => Array ( [0] => entity_type [1] => entity_id [2] => revision_id [3] => deleted [4] => delta [5] => language ) [indexes] => Array ( [entity_type] => Array ( [0] => entity_type ) [bundle] => Array ( [0] => bundle ) [deleted] => Array ( [0] => deleted ) [entity_id] => Array ( [0] => entity_id ) [revision_id] => Array ( [0] => revision_id ) [language] => Array ( [0] => language ) [uc_product_image_fid] => Array ( [0] => uc_product_image_fid ) ) [foreign keys] => Array ( [uc_product_image_fid] => Array ( [table] => file_managed [columns] => Array ( [uc_product_image_fid] => fid ) ) ) [module] => field_sql_storage [name] => field_revision_uc_product_image )
field_data_uc_catalog_image
Array ( [fields] => Array ( [entity_type] => Array ( [type] => varchar [length] => 128 [not null] => 1 [default] => ) [bundle] => Array ( [type] => varchar [length] => 128 [not null] => 1 [default] => ) [deleted] => Array ( [type] => int [size] => tiny [not null] => 1 [default] => 0 ) [entity_id] => Array ( [type] => int [unsigned] => 1 [not null] => 1 ) [revision_id] => Array ( [type] => int [unsigned] => 1 [not null] => ) [language] => Array ( [type] => varchar [length] => 32 [not null] => 1 [default] => ) [delta] => Array ( [type] => int [unsigned] => 1 [not null] => 1 ) [uc_catalog_image_fid] => Array ( [type] => int [not null] => [unsigned] => 1 ) [uc_catalog_image_alt] => Array ( [type] => varchar [length] => 512 [not null] => ) [uc_catalog_image_title] => Array ( [type] => varchar [length] => 1024 [not null] => ) [uc_catalog_image_width] => Array ( [type] => int [unsigned] => 1 ) [uc_catalog_image_height] => Array ( [type] => int [unsigned] => 1 ) ) [primary key] => Array ( [0] => entity_type [1] => entity_id [2] => deleted [3] => delta [4] => language ) [indexes] => Array ( [entity_type] => Array ( [0] => entity_type ) [bundle] => Array ( [0] => bundle ) [deleted] => Array ( [0] => deleted ) [entity_id] => Array ( [0] => entity_id ) [revision_id] => Array ( [0] => revision_id ) [language] => Array ( [0] => language ) [uc_catalog_image_fid] => Array ( [0] => uc_catalog_image_fid ) ) [foreign keys] => Array ( [uc_catalog_image_fid] => Array ( [table] => file_managed [columns] => Array ( [uc_catalog_image_fid] => fid ) ) ) [module] => field_sql_storage [name] => field_data_uc_catalog_image )

field_revision_uc_catalog_image
Array ( [fields] => Array ( [entity_type] => Array ( [type] => varchar [length] => 128 [not null] => 1 [default] => ) [bundle] => Array ( [type] => varchar [length] => 128 [not null] => 1 [default] => ) [deleted] => Array ( [type] => int [size] => tiny [not null] => 1 [default] => 0 ) [entity_id] => Array ( [type] => int [unsigned] => 1 [not null] => 1 ) [revision_id] => Array ( [type] => int [unsigned] => 1 [not null] => 1 ) [language] => Array ( [type] => varchar [length] => 32 [not null] => 1 [default] => ) [delta] => Array ( [type] => int [unsigned] => 1 [not null] => 1 ) [uc_catalog_image_fid] => Array ( [type] => int [not null] => [unsigned] => 1 ) [uc_catalog_image_alt] => Array ( [type] => varchar [length] => 512 [not null] => ) [uc_catalog_image_title] => Array ( [type] => varchar [length] => 1024 [not null] => ) [uc_catalog_image_width] => Array ( [type] => int [unsigned] => 1 ) [uc_catalog_image_height] => Array ( [type] => int [unsigned] => 1 ) ) [primary key] => Array ( [0] => entity_type [1] => entity_id [2] => revision_id [3] => deleted [4] => delta [5] => language ) [indexes] => Array ( [entity_type] => Array ( [0] => entity_type ) [bundle] => Array ( [0] => bundle ) [deleted] => Array ( [0] => deleted ) [entity_id] => Array ( [0] => entity_id ) [revision_id] => Array ( [0] => revision_id ) [language] => Array ( [0] => language ) [uc_catalog_image_fid] => Array ( [0] => uc_catalog_image_fid ) ) [foreign keys] => Array ( [uc_catalog_image_fid] => Array ( [table] => file_managed [columns] => Array ( [uc_catalog_image_fid] => fid ) ) ) [module] => field_sql_storage [name] => field_revision_uc_catalog_image )

Hmm, this is completely confusing me. I've formatted something more readable from the table that brought about this issue, field_data_uc_catalog_image:

Array (
  [fields] => Array (
    [entity_type] => Array (
      [type] => varchar
      [length] => 128
      [not null] => 1
      [default] =>
    )
    [bundle] => Array (
      [type] => varchar
      [length] => 128
      [not null] => 1
      [default] =>
    )
    [deleted] => Array (
      [type] => int
      [size] => tiny
      [not null] => 1
      [default] => 0
    )
    [entity_id] => Array (
      [type] => int
      [unsigned] => 1
      [not null] => 1
    )
    [revision_id] => Array (
      [type] => int
      [unsigned] => 1
      [not null] =>
    )
    [language] => Array (
      [type] => varchar
      [length] => 32
      [not null] => 1
      [default] =>
    )
    [delta] => Array (
      [type] => int
      [unsigned] => 1
      [not null] => 1
    )
    [uc_catalog_image_fid] => Array (
      [type] => int
      [not null] =>
      [unsigned] => 1
    )
    [uc_catalog_image_alt] => Array (
      [type] => varchar
      [length] => 512
      [not null] =>
    )
    [uc_catalog_image_title] => Array (
      [type] => varchar
      [length] => 1024
      [not null] =>
    )
    [uc_catalog_image_width] => Array (
      [type] => int
      [unsigned] => 1
    )
    [uc_catalog_image_height] => Array (
      [type] => int
      [unsigned] => 1
    )
  )
  [primary key] => Array (
    [0] => entity_type
    [1] => entity_id
    [2] => deleted
    [3] => delta
    [4] => language
  )
  [indexes] => Array (
    [entity_type] => Array (
      [0] => entity_type
    )
    [bundle] => Array (
      [0] => bundle
    )
    [deleted] => Array (
      [0] => deleted
    )
    [entity_id] => Array (
      [0] => entity_id
    )
    [revision_id] => Array (
      [0] => revision_id
    )
    [language] => Array (
      [0] => language
    )
    [uc_catalog_image_fid] => Array (
      [0] => uc_catalog_image_fid
    )
  )
  [foreign keys] => Array (
    [uc_catalog_image_fid] => Array (
      [table] => file_managed
      [columns] => Array (
        [uc_catalog_image_fid] => fid
      )
    )
  )
  [module] => field_sql_storage
  [name] => field_data_uc_catalog_image
)

Based off the schema, uc_catalog_image_alt and uc_catalog_image_title are supposed to exist. I can only think it could be a contrib module that is doing something funky but nothing from the list you provided stands out :(

@Triumphent, would you be able to list the schema for field_data_field_image? That's the one that seems to have caused issues on your upgrade.

Now this is interesting...the table does not exist? So in the site that where this issue is happening...is there a field called 'field_image'? You should be able to see a list of fields on your site by going to /admin/reports/fields. I'll be testing out what happens when a field is deleted.

removed field will be field_deleted_data_***

try to backup your site and remove all contri modules and then run updates.

Or try my module, you can update single field:
http://drupal.org/sandbox/droplet/1416898

@iancm's data is updated.

@droplet yep, the remove field is field_delete_data_FIELDID. I tried it out and the update process (with a field_deleted table and without it after all its data has been cleared) worked correctly. I don't know if @iancm's data was updated. The schema shows as such but does the schema actually reflect what is in the table? afaik, drupal_get_schema checks against its schemas internally instead of against the db table directly, doesn't it?

Just to add to the mix - I also ran into this - the instance was a deleted image field (in this case field_migrate_example_image) which had been moved to field_deleted_data_9 and field_deleted_revision_9

Cloning an existing image field and changing the appropriate column names (manually recreating the field_deleted_data_9 and field_deleted_revision_9) allowed the update to succeed - but perhaps a sanity check needs to be put in place that will first look if the tables still exist in the database - it certainly appears that something is being missed when fields are deleted in the system but not having worked on the code much on this site or the admin, not sure when or how this field was created and deleted.

In the meantime, this is a workaround.

If you have another field_image - you can clone it's structure in the database giving the new table the name of the first item in the error message and then renaming the columns. In my example, the error I was receiving was:

Cannot change the definition of field <em class="placeholder">field_deleted_data_9</em>.<em class="placeholder">field_migrate_example_image_alt</em>: field doesn't exist.

This gave me enough information to know the missing table was named field_deleted_data_9, and the field names were going to be field_migrate_example_image_x, so I cloned another field_data_image table's structure and then for each of the following columns:

• field_XXX_image_fid
• field_XXX_image_alt
• field_XXX_image_title
• field_XXX_image_width
• field_XXX_image_height

I replaced the XXX with the name of the missing fields columns:

• field_migrate_example_image_fid
• field_migrate_example_image_alt
• field_migrate_example_image_title
• field_migrate_example_image_width
• field_migrate_example_image_height

Once I had done that, I simply cloned this empty field_deleted_data_9 table with the name field_deleted_revision_9 and the update was able to succeed getting rid of the ugly error message for my clients.

I know this isn't a long term solution, but hopefully it will be helpful in the meantime.

We just upgraded from 7.8 to 7.12 and in doing so, update.php spits out this error:

DatabaseSchemaObjectDoesNotExistException: Cannot change the definition of field <em class="placeholder">field_deleted_data_7</em>.<em class="placeholder">field_migrate_example_image_alt</em>: field doesn't exist. in DatabaseSchema_mysql->changeField() (line 449 of E:\inetpub\wwwroot\includes\database\mysql\schema.inc).

So in trying to get to a point where I could perhaps re-install the image module, I went to remove a field that is using the image module, and while it did delete the field, it threw this error:

PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'gattondrupal.field_deleted_data_7' doesn't exist: SELECT DISTINCT field_deleted_data_70.entity_type AS entity_type, field_deleted_data_70.entity_id AS entity_id, field_deleted_data_70.revision_id AS revision_id, field_deleted_data_70.bundle AS bundle FROM {field_deleted_data_7} field_deleted_data_70 WHERE (field_deleted_data_70.deleted = :db_condition_placeholder_0) AND (field_deleted_data_70.bundle = :db_condition_placeholder_1) LIMIT 10 OFFSET 0; Array ( [:db_condition_placeholder_0] => 1 [:db_condition_placeholder_1] => migrate_example_beer ) in field_sql_storage_field_storage_query() (line 577 of E:\inetpub\wwwroot\modules\field\modules\field_sql_storage\field_sql_storage.module.)

Any help would be appreciated.

Thanks,

@Triumphent, ok..that is strange. You have a 'field_image' field but you don't have the corresponding table (field_data_field_image) in your site? I'm curious about where the data is being saved.

@jeanac, did you delete the field tables from the database or did you delete the field using the delete button? Its very strange to be getting this type of error and it sounds like something that wasn't supposed to be done was.

@BTMash - I deleted the field in my content type that was using it.

I did what bonked did and went through all the error messages and recreated the tables:
field_data_field_taxonomy_photos,
field_data_uc_catalog_image,
field_revision_field_taxonomy_photos,
and field_revision_uc_catalog_image

Once I did all that then I was able to run the update without a failure message.

@iancm, so then were these tables missing from your database? If they were there, what was their prior schema like?

Many thanks to frob for input on this. He lead me to this trail. I voiced my strong suspicions to him at the Long Beach Drupal meeting just now.

My suspicions are below. Basically, I believe it to be a database abstraction issue in Drupal Core code.

Confirmed: This is not a MySQL error, as includes/database/mysql/schema.inc line 449 is a PHP throw, and this means the passed in arguments to function changeField are highly suspect, as the "next" line is the MySQL command invocation. This next line is never executed, so it can be the MySQL client/server software returning this error.

Suspect: I suspect the arguments $field and $field_new are not matching, when they should, meaning one of these two arguments was not set correctly by the mysql code. This assumes a core code hiccup in 7.12, and that the databases involved were not corrupted. The hiccup could be caused by the supplied Alter statement being parsed incorrectly, due one of many reasons. Or the query of the table schema field names was incorrect. I will check these out.

Possible Suspects:
Parsing errors could be caused by php release level of the function doing the parsing (which I have yet to look into, as I thought it important to post now).
The supplied Alter statment could have non display ASCII characters embedded in it, that do not display in many commands (unlikely - I will check for this).

This means I will not be making manual changes to 7.10 databases to try to duplicate this message, though that could be the problem.

Last Thoughts: Need more info from posters with this error as follows:

Mysql release level
PHP release level
Operating System and it's release level
settings.php driver is 'mysql' or 'mysqli'

Original and Upgraded table schema (example below):

describe field_data_field_image;
+--------------------+------------------+------+-----+---------+-------+
| Field              | Type             | Null | Key | Default | Extra |
+--------------------+------------------+------+-----+---------+-------+
| entity_type        | varchar(128)     | NO   | PRI |         |       |
| bundle             | varchar(128)     | NO   | MUL |         |       |
| deleted            | tinyint(4)       | NO   | PRI | 0       |       |
| entity_id          | int(10) unsigned | NO   | PRI | NULL    |       |
| revision_id        | int(10) unsigned | YES  | MUL | NULL    |       |
| language           | varchar(32)      | NO   | PRI |         |       |
| delta              | int(10) unsigned | NO   | PRI | NULL    |       |
| field_image_fid    | int(10) unsigned | YES  | MUL | NULL    |       |
| field_image_alt    | varchar(128)     | YES  |     | NULL    |       |
| field_image_title  | varchar(128)     | YES  |     | NULL    |       |
| field_image_width  | int(10) unsigned | YES  |     | NULL    |       |
| field_image_height | int(10) unsigned | YES  |     | NULL    |       |
+--------------------+------------------+------+-----+---------+-------+
12 rows in set (0.00 sec)

I could be all wet with this direction, reviewing all the involved code lines, but I feel it's worth putting before those trying to solve this. This post concerns the supplied 7.12 code the generates the error message, 'field does not exist'. While BTmash has looked at manual alters to the 7.10 database, I have pursued a problem with the code, php interpretor, or similar site specific, or one time error. This idea is based upon only 3 people reporting this error, and none of them stating they "duplicated" the error from restoring backups, etc.

Note: I'm suspecting this issue is a 'one off' due to site specific "hiccup", like a cosmic ray that the CPUs error correction could not fix. For me to continue, feeling this error is "otherwise", the posters would have to state they have duplicated this error, at least one more time, and post the requested info above, as well as more info, like CPU type, fixes installed for the chipset, and what date and time and timezone they got the 7.12 upgrade, and what method (drush, ftp, http, git, ???), as frob has proposed the hiccup could be a corrupted download, corrupted by one of many methods. A flipped bit at one of many places, some listed below, could cause this error.

Looking at the "code" involved, I see the supplied 7.12 module Image install files has the below excerpt.

  <?php
foreach ($fields as $field_name => $field) {
    $tables = array(
      _field_sql_storage_tablename($field),
      _field_sql_storage_revision_tablename($field),
    );
    $alt_column = $field['field_name'] . '_alt';
    $title_column = $field['field_name'] . '_title';
    foreach ($tables as $table) {
      db_change_field($table, $alt_column, $alt_column, $alt_spec);
      db_change_field($table, $title_column, $title_column, $title_spec);
    }
  }
?>

The function db_change_field is below:

function db_change_field($table, $field, $field_new, $spec, $keys_new = array()) {
  return Database::getConnection()->schema()->changeField($table, $field, $field_new, $spec, $keys_new);
}

The function 'changeField' code (for mysql) is below:

  public function changeField($table, $field, $field_new, $spec, $keys_new = array()) {
    if (!$this->fieldExists($table, $field)) {
      throw new DatabaseSchemaObjectDoesNotExistException(t("Cannot change the definition of field %table.%name: field doesn't exist.", array('%table' => $table, '%name' => $field)));
    }
    if (($field != $field_new) && $this->fieldExists($table, $field_new)) {
      throw new DatabaseSchemaObjectExistsException(t("Cannot rename field %table.%name to %name_new: target field already exists.", array('%table' => $table, '%name' => $field, '%name_new' => $field_new)));
    }

    $sql = 'ALTER TABLE {' . $table . '} CHANGE `' . $field . '` ' . $this->createFieldSql($field_new, $this->processField($spec));
    if ($keys_sql = $this->createKeysSql($keys_new)) {
      $sql .= ', ADD ' . implode(', ADD ', $keys_sql);
    }
    $this->connection->query($sql);
  }

I've included the comment from author of function fieldExists, under the assumption they are far wiser than I am in MySQL functionality, and there might be 'other' reasons that MySQL returns a "false" negative.

  public function fieldExists($table, $column) {
    // The information_schema table is very slow to query under MySQL 5.0.
    // Instead, we try to select from the table and field in question. If it
    // fails, the most likely reason is that it does not exist. That is
    // dramatically faster than using information_schema.
    // @link http://bugs.mysql.com/bug.php?id=19588
    // @todo: This override should be removed once we require a version of MySQL
    // that has that bug fixed.
    try {
      $this->connection->queryRange("SELECT $column FROM {" . $table . "}", 0, 1);
      return TRUE;
    }
    catch (Exception $e) {
      return FALSE;
    }
  }

So after testing this, I can only replicate this error by deleting the field "field_image_alt"

I did three clean installs from 7.9, 7.10, and 7.11 and couldn't reproduce this error without manually editing the database.

There could be any number of reasons that could cause this, I cannot think that any of them have anything to do with an error in the update code.

I added the field back and the error went away.

Make sure the field exists and then attempt the update again.

Other pages reporting the same error messaged, field not existing, are:

http://drupal.org/node/1054104
http://drupal.org/node/1016940

Found via this google URL/keywords: https://www.google.com/search?q=drupal+inurl%3Anode+"field+doesn't+exist"

My conclusion is to roll a patch to change from the 'fast' SELECT trap to test if a field exists, to the solid, but slow, function db_field_exists, which for the purpose of doing an 'upgrade' of Drupal Core, being slow is acceptable, but being "sometimes" wrong is not. (see comment I point out in my previous post).

Rolling such a patch, would not be able to be "tested", without one of the 3 posters being able to reliably duplicate the error, and test the patch, and the error is gone. If they can duplicate the problem, then they should also be able to provide a "describe" on the involved tables, where the fieldname should, ah, 'not' be there.

Philosophically, using an accurate function to test if a field exists is superior to a "suspect" method. Looking into why the 7.12 method is suspect might be hard? Asking the author of the comment is an easy way? How to find that author?

This issue might have such a patch justified under philosophical rationale? I'd like to hear from more expert in Drupal core upgrade and database abstraction coding regarding justification, and past precedent.

There are several possible 'times' where this missing field could occur.

1) 7.12 upgrade deletes the field, or renames it.

2) It was missing even before 7.12 upgrade was attempted, and the _alt field was never queried, as IMG ALT tags were "turned off". Otherwise, this missing field would have generated error messages each time an image was displayed (the field was queried) - and this was not mentioned.

3) Other 'times' as others think of them.

Now, we need to hear from the 3 posters how busy are their sites, were ALT tags "turned off"?

I traced the function db_field_exists and found it goes to the same fieldExists function that uses the SELECT fieldname from tablename LIMIT 1,0 query. So, the existing abstraction for testing if a field exists for MySQL does not invoke a supplied MySQL function call. While mysql's information_schema query to get all field names is commented as being slow in MySQL 5.0, does this include query for a single fieldname? I started reading the dev.mysql.com manual for 5.0, but it's late.

Point is, I mentioned a patch could use function db_field_exists, and that is not so, I found out.

the method should be rewritten to fall back to the slow way if the quick way returns FALSE. That way the slow way is only used to make sure the fast way knows what it is talking about.

Re-postponing because:

• It appears there is still no way to reproduce the issue through the UI
• While the issue has been reproduced by deleting a database table manually, we can't support when that happens
• My suspicion is that some contrib module is causing this, and if so, we really need to find out what it is.

Tagging for steps to reproduce. Ideally the steps to reproduce should start from either "Install Drupal 6.24" or "Install Drupal 7.10."

Also, this issue could use a good issue summary.

I don't have a copy of the original install but I believe it was either 7.7 or 7.9. My install was (7.7 || 7.9)->7.10->7.12 and all I did was create and then delete those image fields mentioned in my previous post. The creation and deletion was done prior to the upgrades.

@iancm, maybe I missed it; what fields did you create and delete? Could you describe a little more what you did? If so, we can have someone install 7.7 and see if the reproduce the same thing by creating an image field, then deleting it, then running the upgrade.

I just tested this with no problems.

Install 7.7
create new image field
Update 7.9
create new image2 field
Update 7.10
delete image2 field
Update 7.12

No errors.

One big question. Should this even fail if the field doesn't exist. This fluke problem would go away if the field where added if it didn't exist.

My expectation is that this is the function that is causing the fluke problem.

/**
 * Utility function: fetch all the field definitions from the database.
 *
 * Warning: unlike the field_read_fields() API function, this function returns
 * all fields by default, including deleted and inactive fields, unless
 * specified otherwise in the $conditions parameter.
 *
 * @param $conditions
 *   An array of conditions to limit the select query to.
 * @param $key
 *   The name of the field property the return array is indexed by. Using
 *   anything else than 'id' might cause incomplete results if the $conditions
 *   do not filter out deleted fields.
 *
 * @return
 *   An array of fields matching $conditions, keyed by the property specified
 *   by the $key parameter.
 */
function _update_7000_field_read_fields(array $conditions = array(), $key = 'id') {

Notice the Warning

* Warning: unlike the field_read_fields() API function, this function returns
* all fields by default, including deleted and inactive fields, unless
* specified otherwise in the $conditions parameter.

Thus this will return if the fields even if they have been deleted or possibly made inactive somehow.

@iancm, if you don't have the old db could you please post a list of the enabled module. My guess is there is one that has modified the image field table fields.

I had a Drupal 6.4 or so database, working on day one, and the next day it was missing several, maybe many whole tables. I just started over. I never did find out what happened. It was on a VPS, OpenVZ, and I'd like to blame sync to disk and journaling system. I've seen the journaling be very slow. And buffering can take hours, for a tail command to show log entries. Very strange. I did not set up the OpenVZ.

That's the only time I have seen some, many Drupal tables just disappear off the disk.

Given Triumphent experienced something quite similar, though in D7, and unknown OS (if VPS... then we might be safe to assume it's an issue with the Host OS doing weird stuff in a 100% RAM used, or 100% disk used situation ...), and overcame it by re-creating the site. iancm and xjm have spoken up, we know now this issue will not be "recreated" by anyone involved, now or in the future.

Can we assume iancm and xjm will not have info on OS and if it was VPS? VPS are known to have problems, as is any OS when the disk fills up, when writes to disk are too fast, files get lost. A crashed machine in this situation, with the file just opened, and the file pointer set at zero, will result in the file being truncated, but still in the directory tree, just zero in size.

I'm thinking the remaining key point is how did the field, or associated tables, and perhaps unrelated tables, go missing? Yes, I'd like to think we might one day find out. I know in my case, it's not just unlikely, it's not going to happen.

What happens to an issue when this happens? Closed, can not reproduce? How soon after this unable to reproduce is known is the issue closed? I'm asking for feedback to learn how an issue is closed. I'll report this thread at our next Long Beach Contrib meeting on March 28, 2012.

I have a similar problem (I believe) that I have been unable to resolve. I upgraded to 7.12 and ubercart 3.0-rc4. I can no longer make modifications to my product pages (error message below). I am not a database person so a bit confused on how this has been resolved per previous post. Could you please provide a bit more detail - seems like I need to create fields in the `dr_field_data_uc_product_image`table? I have the following related fields in the table now
uc_product_image_fid
uc_product_image_alt
uc_product_image_title

Do I need to create:
uc_product_image_width
uc_product_image_height
and then run update.php? how do i find the attributes of the fields to create?

PDOException: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'uc_product_image_width' in 'field list': INSERT INTO {field_data_uc_product_image} (entity_type, entity_id, revision_id, bundle, delta, language, uc_product_image_fid, uc_product_image_alt, ...

Is it an Ubercart bug, remove field with an inappropriate way??

2 reports here are related to UberCart.

@r_wel,
either upgrade 7.12 or 3.0-rc4 . don't upgrade them at same time. can you report us the new result. thanks.

Tried fix as described, didn't have any effect.

Installed previous version of ubercart (Ubercart 3.0-beta3, 2011-5-26) which I have been running in production into 7.12 site - ran update.php. Same error occurred. Tried production system (drupal 7.8 and Ubercart 3.0-beta3, 2011-5-26) - did not fail.

I have updated site in a couple of months until last week but I recall that I had tried updating Ubercart module in November and had failures (could have been the same but can't recall) - I did not have time to troubleshoot back then.

Have searched forums/google for issue - found several references of folks indicating same error (different versions of drupal I believe) but no responses on a resolution.

I know you mentioned what you use...but have you run update.php? This is somewhat unrelated as the issue is stemming out from update_7002 which is what created the width and height columns (and that came out in Drupal 7.10).

yep, I run update.php all the time (just to be sure, I just ran again) - same error.

I tried adding the two fields (width and height) to field_data_field_image as described above by bvirtual - no effect, same error.

Also followed the suggestions in issue 'image update #7002 failed.' The only image tables I found with 'width' and 'height' were the 2 in which I had added the fields. Put site in maintenance mode, deleted the fields in the 2 tables, ran update.php, cleared cash, ran cron - no effect, same error.

Can you please give me some ideas on a workaround for this - I have to get this up and running this weekend. At the moment, all I can think of is to install previous versions of drupal (or just back down to 7.8). I didn't want to do this because of the security fixes deployed since then although I am not sure how critical these issues were.

@r_wel sorry about this :( I had crashed and burned by the end of Friday and didn't even notice this issue :(

The thing I see that seems to have happened is that an update or set of updates seem to have skipped. I'm not sure how. But basically, try the following if you have phpmyadmin or access to mysql (and back up your database!):

UPDATE system SET version = 7001 where name = 'image' and type = 'module';

And run update.php. This *should* create the width, height columns and run the various other updates as needed.

But please...back up your db. Any other suggestions from folk would be helpful as I'm trying to think of ways to clear up this issue when in a bind.

I cut & pasted your code (UPDATE system SET version = 7001 where name = 'image' and type = 'module';)
into the sql form with the selected db. I get the following error code.
#1146 - Table 'efficie7_drp9.system' doesn't exist

As I said in an earlier post, I have am not a database guy so I'm a bit lost on on to proceed. Could you please clarify.

The table is called dr_system in my db. There is a field called schema_version. I changed your statement to reflect these and got it to run. After running update.php (put in maintenance mode first), I get the following when trying to save a content page:

PDOException: SQLSTATE[HY000]: General error: 1366 Incorrect integer value: '' for column 'uc_product_image_width' at row 1: INSERT INTO {field_data_uc_product_image} (entity_type, entity_id, revision_id, bundle, delta, language, uc_product_image_fid, uc_product_image_alt, uc_product_image_title, uc_product_image_width, uc_product_image_height) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6, :db_insert_placeholder_7, :db_insert_placeholder_8, :db_insert_placeholder_9, :db_insert_placeholder_10); Array ( [:db_insert_placeholder_0] => node [:db_insert_placeholder_1] => 27 [:db_insert_placeholder_2] => 27 [:db_insert_placeholder_3] => product [:db_insert_placeholder_4] => 0 [:db_insert_placeholder_5] => und [:db_insert_placeholder_6] => 26 [:db_insert_placeholder_7] => [:db_insert_placeholder_8] => [:db_insert_placeholder_9] => [:db_insert_placeholder_10] => ) in field_sql_storage_field_storage_write() (line 448 of /home1/efficie7/public_html/drupal/modules/field/modules/field_sql_storage/field_sql_storage.module)

I tried the process again with slight mod. After re-installing db back-up, I made the value change in the dr_system table (1 row effected - modules/image/image.module). I did not put the system into maintenance mode, ran update.php then ran cron. This time, things seem to work and I've been able to successfully edit/save a number of pages.

Not sure why this last try made any difference but am very happy that things are up and running. I appreciate your help.

@r_wel, I'm glad your issue worked out. It does seem like the updates did not run as they should have on your system. That is interesting to know re: the update.php followed by cron. Cron might have cleared some particulars? But please post any updates as you come across them; your patience is much appreciated.

Post #40 by bonked solution worked for me. I updated website from 6 to 7 before this thread started...I'm so happy now!

So for some reason, it is trying to change the definition of a field in deleted data?

That makes sense to me actually - the Field API keeps deleted fields around in the database until the data has been completely and properly purged, so it should be fine to continue updating that data until the purge is finished.

So the question is, why does the {field_config} table believe the field is still around in the database on your site, even though the {field_deleted_data_19} database table is actually gone?

I feel like there might have been a more generic field API issue about this kind of inconsistency somewhere else, but I'm not sure.

Also, this issue should probably be in either the image.module or database update system components for now (since it's about the upgrade path) - yes it's confusing but update.module is something else.

updated the CSS Injector module. keep on trying to update the database, and getting

The following updates returned messages
css_injector module
Update #7001

    Failed: DatabaseSchemaObjectExistsException: Cannot add field <em class="placeholder">css_injector_rule</em>.<em class="placeholder">enabled</em>: field already exists. in DatabaseSchema_mysql->addField() (line 328 of /home1/raphaelt/public_html/includes/database/mysql/schema.inc).

I am strictly an amateur -- this is way above my head, and frankly I haven't the faintest idea what the problem or risk is. I know it's bad form for the ignorant to panic, but if I could figure out how to revert to the previous version of the module and never perform an update again, I would probably do that.

I copied the tables from another database into this one, then set the value for image in the system table to the same update value as that one (7004). Voila!