Unable to add new content with empty upgraded image/file field

I'm getting exactly the same error in exactly the same situation. Did you get to the bottom of it?

Similar: #905108: Integrity constraint violation: 1048 Column 'fid' cannot be null

My error:

PDOException: SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'fid' cannot be null: INSERT INTO {file_usage} (fid, module, type, id, count) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4); Array ( [:db_insert_placeholder_0] => [:db_insert_placeholder_1] => file [:db_insert_placeholder_2] => node [:db_insert_placeholder_3] => 12459 [:db_insert_placeholder_4] => 1 ) in file_usage_add() (line 661 of /Users/kenorb/Sites/XXX/docroot/includes/file.inc).

But I'm not using image, just a file field.
It happens when submitting the node using ctools wizard multistep.
None of the files were posted.

This could be related as well:
#1163740: PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '' for key 2: INSERT INTO {file_managed}

Following code (from Drupal core) in my specified case:

function file_usage_add(stdClass $file, $module, $type, $id, $count = 1) {
  if (is_null($file->fid)) {
    var_dump($file, $module, $type, $id, $count); exit;
  } 
  db_merge('file_usage')
    ->key(array(
      'fid' => $file->fid,
...

returns:

object(stdClass)[311] // <!-- 1st argument is EMPTY object
string 'file' (length=4)
string 'node' (length=4)
string '12461' (length=5)
int 1

Backtrace:

'function' => string 'file_usage_add' (length=14)
'function' => string 'file_field_insert' (length=17)
'function' => string '_field_invoke' (length=13)
'function' => string 'field_attach_insert' (length=19) (node)
'function' => string 'node_save' (length=9) (my module)

Problem:
node which is saved contains somehow empty/null value:

public 'field_file' => 
    array
      'und' => 
        array
          0 => null

This is because:
When I'm submitting step which has file field on the page, the following variable is _POSTed to $form_state:

array
  'previous' => string 'Back' (length=4)
  'next' => string 'Continue' (length=8)
  'field_file' => 
    array
      'und' => 
        array
          0 => 
            array
              '_weight' => string '0' (length=1)
              'fid' => int 0
              'display' => string '1' (length=1)
              'description' => string '' (length=0)
              'upload_button' => string 'Upload' (length=6)
              'remove_button' => string 'Remove' (length=6)
              'upload' => string '' (length=0)
  'form_build_id' => string 'form-xxx' (length=48)
  'form_token' => string 'xxx' (length=43)
  'form_id' => string 'xxx' (length=31)
  'op' => string 'Continue' (length=8)

, after that I'm copying values from field_file to my $node object, so at this point everything is perfect.
The problem is when those details are loaded from cache and are damaged by _field_invoke_multiple().

It happens when I'm doing preview of the node by following code (#1622952: The right way to programatically render the preview of node object before the creation):

  $node = ctools_object_cache_get('foo', 'node');
  $node->in_preview = TRUE;
  _field_invoke_multiple('load', 'node', array(NULL => $node)); // THIS function did some changes to $node object !
  field_attach_prepare_view('node', array(NULL => &$node), 'full'); // lets field types and formatters load additional data needed for display
  $build = @node_view($node, 'full');
// in different function I've:
$node = ctools_object_cache_get('foo', 'node');
node_save($node);
// here I've the fatal error, because the data in $node has been overridden by _field_invoke_multiple()

and following line in that function (_field_invoke_multiple):

$entity->{$field_name}[$langcode] = $grouped_items[$field_id][$langcode][$id];

is overriding the field values, changing field_file array to NULL.

XDebug snapshot:

$langcode = 'und' modules/field/field.attach.inc:363
$entity = class stdClass { public $type = 'xxx'; public $workflow = 5; public $status = 1; public $promote = 0; public $sticky = 0; public $uid = '181'; public $created = 1341591323; public $revision = FALSE; public $comment = '0'; public $field_file = array ('und' => array (0 => array ('_weight' => '0', 'fid' => 0, 'display' => '1', 'description' => '', 'upload_button' => 'Upload', 'remove_button' => 'Remove', 'upload' => ''))); public $in_preview = TRUE } modules/field/field.attach.inc:364
$id = '' modules/field/field.attach.inc:364
=> $entity->field_file['und'] = array (0 => NULL) modules/field/field.attach.inc:366

I've fixed my problem by cloning ctools cached the object, See: #1676696: ctools_object_cache_get() returns reference, instead of clone
But still the question is, why _field_invoke_multiple is overriding the node values (some of them to NULL) when doing node preview.

Deescription suggests a bug, changing category.

This could be related to:
#846356: node_preview() calls _field_invoke_multiple()

Possibly related:
#1164240: Notice: Undefined offset: 188 in _field_invoke_multiple() (line 351 of /home/.../field.attach.inc)
#1169426: Field invoke multiple does not handle field language correctly
#549710: Fields not rendered properly during comment preview
#1089174: Prepare view hooks do not receive the language parameter

Hi Guys, I'm now working on the same site @aangel found this problem on, and I'd like to get to the bottom of it. To recap, we are working with a D7 site that has been upgraded from D6. Creating a node with a migrated image field on it throws errors when the image field is left empty:

PHP Notice

Notice: Undefined property: stdClass::$fid in includes/file.inc on line 654

PDO Fatal:

PDOException: SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'fid' cannot be null: INSERT INTO {file_usage} (fid, module, type, id, count) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4); Array ( [:db_insert_placeholder_0] => [:db_insert_placeholder_1] => file [:db_insert_placeholder_2] => node [:db_insert_placeholder_3] => 361 [:db_insert_placeholder_4] => 1 ) in file_usage_add() (line 661 of includes/file.inc).

I'm getting the same error. Drupal site was migrated from Drupal 6, and I've used imceimage before. Now it is exactly as in #2. I can not have file field, which was migrated from imceimage field, to be empty, for it gives error message as in Issue Summary

I'm up to 7.17 now, and having this problem on files as well as images. Any file or image field that has been upgraded from D6 to D7 throws PDO errors when left blank. I'm not sure what the difference is between the D6 upgraded fields, and why this isn't an issue for newly-created fields in D7. But what is happening is that a "file" made up of the array below ends up getting passed into file_field_insert().

Array
(
    [embed] => 
    [value] => 
)

In my case, that "file" was passed in from _field_invoke and in the case of #6 it was passed in from _field_invoke_multiple.

better title :)

It looks like there is already a hook-like function that allows field types to define their primary field, and allow the field to be set as "empty" when that primary field is blank. File module uses it: file_field_is_empty.

Unfortunately, this function is only called from one place (_field_filter_items) and that function is never called in my case, and I expect not for #6 as well.

The reason this function is never called is because _field_filter_items is only called from within the default field functions for the two ops validate and submit, but those functions only get called when the $options array passed into _field_invoke contains 'default' => TRUE.

The default value for 'default' is FALSE. When the default is false, it causes Drupal to search for module-specific op functions like image_field_validate and file_field_validate (which don't exist) instead of using field_default_validate.

Since these functions don't exist, no function is run on this data at all, and the entire contents of the array get sent along to the next step.

I think that #1443158: file_field_presave assumes that a file object has been loaded may resolve this, likely this is a duplicate of that issue.

No, it's not related.

At the time file_field_presave is run $items is just an empty array for me, so the patch #1443158: file_field_presave assumes that a file object has been loaded makes no difference in this case.

Maybe I can use the same approach for file_field_insert though, since it's not safe to assume that $items are properly formatted before they get here.

This patch seems to do the trick.

If at the time of presave $items is an empty array, but at insert, it's not, then I'm not sure what core should actually be doing. Something is injecting invalid data inbetween those times. Presave should be the proper place to fix it...

Well, the only module I am using that implements hook_field_insert is media, and it's implementation is almost exactly the same as file_field_insert. (none implement hook_field_presave).

What comes between presave and insert?

Testing #15.

I could easily reproduce it via: drush -y generate-content --types=advert 10
Now instead of:

array_flip(): Can only flip STRING and INTEGER values! entity.inc:178                                                                                                                                                                                              [warning]
array_flip(): Can only flip STRING and INTEGER values! entity.inc:178                                                                                                                                                                                              [warning]
WD node: PDOException: SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'fid' cannot be null: INSERT INTO {file_usage} (fid, module, type, id, count) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2,            [error]
:db_insert_placeholder_3, :db_insert_placeholder_4); Array
(
    [:db_insert_placeholder_0] => 
    [:db_insert_placeholder_1] => file
    [:db_insert_placeholder_2] => node
    [:db_insert_placeholder_3] => 3
    [:db_insert_placeholder_4] => 1
)
 in file_usage_add() (line 661 of /home/ubuntu/docroot/includes/file.inc).

I see:

array_flip(): Can only flip STRING and INTEGER values! entity.inc:178                                                                                                                                                                                              [warning]
array_flip(): Can only flip STRING and INTEGER values! entity.inc:178                                                                                                                                                                                              [warning]
array_flip(): Can only flip STRING and INTEGER values! entity.inc:178                                                                                                                                                                                              [warning]
WD node: PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '' for key 'uri': INSERT INTO {file_managed} (filesize, status, timestamp) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2);     [error]
Array
(
    [:db_insert_placeholder_0] => 0
    [:db_insert_placeholder_1] => 1
    [:db_insert_placeholder_2] => 1355846014
)
 in drupal_write_record() (line 7036 of /home/ubuntu/docroot/includes/common.inc).

Try to combine these two patches:
http://drupal.org/files/core-save_upgraded_empty_fields-please-1627860-1...
http://drupal.org/files/file.field_.inc-1163740_0.patch
I think it seems to work.

Source:
http://drupal.org/node/1627860#comment-6788650
http://drupal.org/node/1163740#comment-6102250

UPDATE:
New patch available here:
http://drupal.org/node/1443158#comment-6955596

^ these patches merged together can be found here:
http://drupal.org/node/1163740#comment-6944850

#1163740: PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '' for key 2: INSERT INTO {file_managed}

Duplicate of:
#1163740: PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '' for key 2: INSERT INTO {file_managed}
#1443158: file_field_presave assumes that a file object has been loaded

I can confirm that #15 solves the problem for med on a migrated file field from D6 --> D7. That patch seems to fix problems for some people. Shouldn't it just be commited for it self in stead of combining patches which fixes several problems (which some I do not experience?)? No 15 is RTBC for me.

I still think that something inserting an invalid value between hook_file_presave() and hook_file_insert() means you have something else wrong that shouldn't be hidden by core.

same problem here

I used #15 with success, however I had to apply the principle to hook_field_update as well

Tested with a few hundred nodes.. failed again .. :-( at least it doesn't fail after 10 nodes.. now I'm getting:
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '' for key 2

Noticed in the logs that it was some object based issue, so I added a wrapper call to "is_object" before trying it, worked on all 10k odd nodes! YAY No more errors!

function addImageToNode(&$node, $imagePATH) {
    $image_data = file_get_contents($imagePATH);
    if (strlen($image_data) > 1) {
        if (REDOEMALL) {
            $image_file = file_save_data($image_data, 'public://', FILE_EXISTS_REPLACE);
        } else {
            $image_file = file_save_data($image_data, 'public://', FILE_EXISTS_ERROR);
        }
//file_save_data returns false on error or ERROR, so this call is skipped and the image isn't attached.
        if (is_object($image_file)) {
            try {
//We just saved a new file, lets attach it correctly to the node.
                $node->field_product_photos[$node->language][0] = (array) $image_file;
                field_attach_validate('node', $node);
            } catch (FieldValidationException $e) {
                if (DEBUG)
                    print $e->getMessage();
                return false;
            }
            return true;
        }
    }

    return false;
}

Needs a bit of work to attach multiple images.. but its getting there. (Note, calls to is_readable etc are in a parent function, this simply attaches the image, maybe I just have to remove the 0.. ;-)

fixing code formatting

I can confirm that #15 fixed the issue for me

There have been several reviews that say #15 solves the problem. Any chance we can get this committed, or maybe get some more feedback that might suggest an alternative approach, or a reason why this bugfix is not acceptable? Would love not to have to apply this patch again to 7.26 :)

I am so looking forward to this being fixed. Now I have to manually remember this issue, find this post, and apply the #15 fix afterwards. Of course this always happens after someone creates a new post and get an error, losing their work :-(

#15 worked for me as well. Thanks jenlampton.
Note: the issue marked as a duplicate did not fix the problem for me.
It patched hook_field_presave() whereas #15 is patching hook_field_insert() with the same kind of fix. It looks as if both patches are needed.
It would be good if this could be committed so we don't have to patch core on every new release.

#15 worked for me. Thanks.

I agree with Andrew. This is the only module I use that requires manual fixing. It would be great to commit this patch.

Some more information about this problem, and a suggested solution that removes the need to patch the code.

I have the same problem described in #12:
- a content type that in D6 had an "Embedded Image" field that has been migrated to a D7 file field.
- If that field is left empty and the node is saved, I get the PDOexception shown at the top of the thread.
- The exception happens when a "file" made up of the array below ends up getting passed into file_field_insert().

Array
(
    [embed] =>
    [value] =>
)

The node has another file field ("Upload", the file attachments field) that can be left blank without any problem.

Looking through the backtrace, when I enter node_save(), neither of the file fields are present on the $node. Same when it calls field_attach_insert(), but after that calls _field_invoke_default(), the bad field is added to the node, with contents:

Array
(
    [embed] =>
    [value] =>
)

The problem happens in field_get_default_value(), which has the lines:

  elseif (!empty($instance['default_value'])) {
    $items = $instance['default_value'];
  }

That 'default_value' is stored in the table field_config_instance, in the data column.

If I look at bad file field's entry, the data column includes:

    [default_value] => Array
        (
            [0] => Array
                (
                    [embed] => 
                    [value] => 
                )

If I look at the good file field's entry, there is no '[default_value]'.

I think it's a bit of rubbish left over from the D6->D7 migration, so I've used the following code to delete the '[default_value]' section from that field:

  $field_name='YOUR_FIELD';
  $bundle='YOUR_BUNDLE';
  $data = db_query('SELECT data FROM {field_config_instance} WHERE field_name = :field_name AND bundle = :bundle', array(':field_name' => $field_name, ':bundle' => $bundle))->fetchField();
  $data = unserialize($data);

  unset($data['default_value']);

  db_update('field_config_instance')
    ->fields(array('data' => serialize($data)))
    ->condition('field_name', $field_name)
    ->condition('bundle', $bundle)
    ->execute();

After that I can save a node with the field left empty, and there isn't any exception.

Thanks davidhk - the previous comment is much more useful than the patch

I upgraded my website after previously patching like this - then the patch disappeared and hence the problem reappeared. As far as i know this still isnt in core so you really need to fix the root of the problem not patch

I made a new page with input type "php" to enable me to do this in a simple way - you then just save the node and the page should load, but depending on where you're editing it from you may have to then visit the page.

In case you were confused as to what the YOUR_FIELD and YOUR_BUNDLE needed to be - visit your node type and find the field names for any image / media type fields, the field is the machine name for the field, the bundle is the machine name for the content type. Go to the field_config_instance table in your database and you can check there too.

i also flushed the cache which i think might sometimes be necessary, here's some php which helps print out the $data so you can check all is well - this is what is in my node - completely copied from the great post by davidhk, with a bit of output:

// from this article
// https://www.drupal.org/node/1627860#comment-9238389

$field_name='YOUR_FIELD';
  $bundle='YOUR_BUNDLE';
  $data = db_query('SELECT data FROM {field_config_instance} WHERE field_name = :field_name AND bundle = :bundle', array(':field_name' => $field_name, ':bundle' => $bundle))->fetchField();
  $data = unserialize($data);
  unset($data['default_value']);


  db_update('field_config_instance')
    ->fields(array('data' => serialize($data)))
    ->condition('field_name', $field_name)
    ->condition('bundle', $bundle)
    ->execute();
print('<pre>');
print_r($data);
print('</pre>');
print('done it');

Thanks davidhk and amcc... your posts saved me a lot of time today.

For anyone who is just hitting this problem you can also save the above code as a php file (field_fix.php) and run it from drush with the following command:

drush php-script field_fix.php

Then follow with a cache clear:
drush cc all

I highly recommend doing this on a test version of your site first just to make sure that you don't break something else.

Ken Colwell

Let's reopen this given that the other issue is fixed but didn't seem to fully solve the issue for everyone.

However I agree with the questions above by Dave Reid and others - why would we need to do this in file_field_insert() if we're already doing it in file_field_presave()? It doesn't seem like that's been answered... if some other module is inserting bad data into the entity after presave has already run, isn't that a bug in that module instead?

Based on #36, if that's the source of the problem for everyone (not sure if it is), that would suggest this needs to be fixed via an update function rather than via runtime code?

Given that, I'm restoring the earlier title for now.

Well, I've written an update hook as per #36 but now I'm not sure this belongs as part of core either. Where is the right place for this hook, as part of CCK migrate, perhaps?

Here's the update (currently written as part of file module) in case anyone else needs it before we decide where to put it.

/**
 * Cleanup left over data from D6 upgrade.
 */
function file_update_7001(){
  $fields = field_info_fields();
  foreach ($fields as $field_info) {
    if ($field_info['type'] == 'image' || $field_info['type'] == 'file') {
      foreach ($field_info['bundles'] as $entity_type) {
        foreach ($entity_type as $bundle) {
          $data = db_query('SELECT data
                            FROM {field_config_instance}
                            WHERE field_name = :field_name
                            AND bundle = :bundle',
                          array(
                            ':field_name' => $field_info['field_name'],
                            ':bundle' => $bundle))
                    ->fetchField();

          $data = unserialize($data);
          if (isset($data['default_value'])) {
            unset($data['default_value']);
            $data = serialize($data);

            db_update('field_config_instance')
              ->fields(array('data' => $data))
              ->condition('field_name', $field_info['field_name'])
              ->condition('bundle', $bundle)
              ->execute();
          }
        }
      }
    }
  }
}

Thanks for the update hook in #36, @jenlampton! This error suddenly cropped up on a D7 site I recently inherited that had been upgraded from D6; adding the hook to modules/file/file.install and running the resultant database update fixed the problem.

I'll note that the error occurred whenever a node was created that included an upgraded image (file) field, whether or not the field was empty. The error did not occur when editing an existing node, only when first saving a new node.

I can confirm that patch from #15 fixes the issue for me. Thanks a lot!

I did #38, but the code from #37 I had to wrap in <?php ... ?>, only then it worked for me and solved the problem. Thanks, @davidhk, @amcc and @kcolwell.