Come together with the global Drupal community in Rotterdam, 28 Sept – 1 Oct 2026. Sessions, contribution, connection, and Early Bird savings until 8 June.I am dealing with a module (CMIS) that is trying to insert content into the database with the wrong charset. I was getting the following two errors:
Warning: htmlspecialchars(): Invalid multibyte sequence in argument in check_plain() (line 1545 of /var/www/drupal/includes/bootstrap.inc).
PDOException: in field_sql_storage_field_storage_write() (line 448 of /var/www/drupal/modules/field/modules/field_sql_storage/field_sql_storage.module).
However, the exception handling code in errors.inc combined with check_plain() means that the error message is completely lost in the PDOException. Note that the exception above has no message or trace.
That is, in includes/errors.inc#_drupal_decode_exception(), if $exception->args() contains invalid characters such as from file content, the displayed exception in the logs is simply:
PDOException: in field_sql_storage_field_storage_write()
If I remove print_r($exception->args, TRUE) from the $message, then the exception becomes (infinitely more useful):
PDOException: SQLSTATE[HY000]: General error: 1366 Incorrect string value: '\xD0\xCF\x11\xE0\xA1\xB1...' for column 'body_value' at row 1: INSERT INTO {field_data_body} (entity_type, entity_id, revision_id, bundle, delta, language, body_value, body_summary, body_format) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6, :db_insert_placeholder_7, :db_insert_placeholder_8); in field_sql_storage_field_storage_write() (line 448 of C:\hubnet\drupal\modules\field\modules\field_sql_storage\field_sql_storage.module).
This is definitely due to the if ($exception instanceof PDOException) block, because if I remove it, the exception becomes (still useful):
PDOException: SQLSTATE[HY000]: General error: 1366 Incorrect string value: '\xD0\xCF\x11\xE0\xA1\xB1...' for column 'body_value' at row 1 in PDOStatement->execute() (line 2139 of C:\hubnet\drupal\includes\database\database.inc).
I'm not sure what the best solution is, since in most cases you'd want to keep the arguments. Is there a way to check that a given string is valid w.r.t. a given charset before passing it to check_plain() - perhaps with drupal_validate_utf8()?
This seems to almost be a duplicate of "Error messages not generated in UTF8 in some situations" - but this is occurring on two different platforms (Linux, Windows) with MySQL.
| Comment | File | Size | Author |
|---|---|---|---|
| #27 | reroll_diff_14-27.txt | 944 bytes | sahil.goyal |
| #27 | 1929862-27.patch | 793 bytes | sahil.goyal |
| #14 | 1929862-14.patch | 675 bytes | rpayanm |
Issue fork drupal-1929862
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
compare
Comments
Comment #1
soundasleep commentedAttached is a proposed patch that replaces invalid UTF-8 with simply [invalid UTF-8].
Comment #2
David_Rothstein commentedThis looks like a useful patch for debugging these kinds of situations. I guess it's kind of a hack but I can't think of anything better at the moment...
Here is a reroll that applies correctly from the Drupal root.
Would need to go in Drupal 8 too if it's actually going to be committed, but I needed the Drupal 7 patch myself now and only had time to reroll that one.
Comment #3
David_Rothstein commentedComment #5
David_Rothstein commentedOh, I think it failed because I put the issue against Drupal 8. Moving back to Drupal 7 (temporarily) to let the testbot run.
Comment #6
David_Rothstein commented#2: pdo-exception-invalid-utf8-1929862-2.patch queued for re-testing.
Comment #7
David_Rothstein commentedTests pass - here's a completely untested Drupal 8 version.
Comment #8
mskicker commented#7 work for me
really tanx and very imp please add to dev ASAP
Comment #9
jhedstromPatch no longer applies. Would it be possible to add a test for this?
Comment #10
nitesh sethia commentedRerolled the patch as per the latest release.
Comment #11
daffie commented@Nitesh Sethia: How is this just a reroll of the previous patch? The previous patch is 648 bytes and your patch is 2970 bytes?
Comment #12
sharique commentedThe file error.inc is changed very much, perhaps re-written completely, so this issue need to be tested manually to see if it is still an issue or it is resolved.
Comment #13
rpayanmComment #14
rpayanmComment #15
dalinGreat to see a reroll, but it looks like we still need some unit tests.
Comment #27
sahil.goyal commentedHi, Seen that last patch was too much old and i tried as well to apply on my local environment 9.4.x but it doesn't get apply properly so making a reroll patch for the same and adding a reroll file.
Comment #28
sahil.goyal commentedComment #29
rpayanmComment #31
smustgrave commentedThis issue is being reviewed by the kind folks in Slack, #needs-review-queue-initiative. We are working to keep the size of Needs Review queue [2700+ issues] to around 400 (1 month or less), following Review a patch or merge request as a guide.
In addition to the manual testing this will need a test case to show the issue.
Comment #33
smustgrave commentedBelieve this change now qualifies for the new policy of being small enough to not warrant tests. but definitely needs an issue summary update.
Comment #34
Anonymous (not verified) commentedtpham1023 made their first commit to this issue’s fork.