Fields locales_source.source and locales_target.translation is not suitable for long texts and huge config objects

I have the same issue.

The size of locale_sources' source field is too small to hold the data you're inserting. Either don't insert so much data, or make the source field take more data:

       'source' => array(
         'type' => 'text',
         'mysql_type' => 'blob',
         'size' => 'big',
         'not null' => TRUE,
         'description' => 'The original string in English.',
       ),

function locale_update_7006() {
  db_change_field('locales_source', 'source', 'source', array(
    'type' => 'text',
    'mysql_type' => 'blob',
    'size' => 'big',
    'not null' => TRUE,
    'description' => 'The original string in English.',
  ));
}

The lack of patch is because it's more seeing if this helps the Drupal 7 folk first. The fix would need to get into Drupal 8 before backporting to 7.

This is in response to comment #2 only:

I believe that the potential negative performance impact of increasing the size of the blob outweighs the benefit. The purpose of the Locale module is to support translations in the interface in which case 16kb would appear to be more then sufficient. I would not recommend making the change suggested in comment #2.

Hi all,

I had this issue, and not even #2 could help me.
I had a very large string in watchdog, and dblog page failed to load because the string was forced to be saved into locales_source.source.

Not having enough time for debugging how to eliminate the large string (deleting from watchdog did not do it) I had no choice.
My only remained solution was to patch the core (I know, never ever even think of it) temporary:

if(strlen($string) > 50000) {
    return $string;
}

/modules/locale/locale.module line 671

After some time I will remove this and see if survived the large string, but now it saved me a lot of debugging time.
I Hope this helps someone someday.

This still happens (of course. since it hasn't been fixed/backported yet).
During development of a module, which logs some events to watchdog I made a mistake and a complete view including 25 full nodes was var_dumped into the watchdog message. Since I was on a multilingual site and currently in another language interface than English, the "Show Recent Log Posts" page fails with the above error. It works fine when switching to English.

I am experiencing this issue now, but @kardave's temporary fix doesn't seem to work any more... Does anyone know how to fix it, also even if only temporary?

See this #2057615: Increase size of source field to hold long URLs

This is because mysql limit for blob field is 64k
Faced with that in d8 where config object is bigger than that

BTW looks the same limit we have for translation

The attached patch is port of #4 to Drupal 8. All it does is store an empty string in the 'locales_source' and 'locales_target' table if the source string is greater than 64KB.

This is only a temporary solution. The most immediate side-effect I am seeing is when translating config the source string no longer appears because it is empty.

Here is the patch and I am setting it to needs review just get the testbot to review it.

The watchdog is the most likely place where too long strings get passed to t(), but it any code may make this “mistake”. The translation system should guard itself for this types of errors.

I see two options: 1. Truncate the string; 2. Throw an exception.

I agree that increasing the table size is not the solution. Exceptional cases like these cannot justify any performance this change may have.

I just ran into this issue on 8.9.13.

#14 applied and works.

Rerolled the patch in #14, thanks!

1. +++ b/core/modules/locale/src/StringDatabaseStorage.php
   @@ -463,6 +464,14 @@ protected function dbStringSelect(array $conditions, array $options = []) {
   +    if (Unicode::strlen($string->getString()) > 64000) {
   

   There's no such function in D9, see https://www.drupal.org/node/2850048

2. +++ b/core/modules/locale/src/StringDatabaseStorage.php
   @@ -463,6 +464,14 @@ protected function dbStringSelect(array $conditions, array $options = []) {
   +      $string->setString('');
   

   This event needs to be logged in watchdog (warning at least) so users can catch this bugs

Addressed #28.1, thanks.

Note: in my tests I did not have this issue with PostgreSQL, however I came across it with MySQL v8.0.28

so far I was able to mitigate my use case by:
#2844452: Export configuration YAML strings as multiline
#3271926: mysql 1406 Data too long for column 'source' at row 1 insert into locales_source

With those two related items that @joseph.olstad referenced now fixed, is this still an issue

Curiously enough, for some reason I noticed that webforms configuration that is edited by ckeditor generates \r\n even on the recent core release 9.3.16.

Postgresql has a larger blob, doesn't have this issue but with MySQL variants I had issues importing large webform config yml , not sure why ckeditor would be outputing \r\n or where to turn this bad ckeditor behavior off as it affects the processing of translations that eventually go into the locales table.

Sorry not much in dept here but I did check out the blob sizes for MySQL:

TINYBLOB   : L < 2^8  =           256 Bytes
BLOB       : L < 2^16 =        65,536 Bytes
MEDIUMBLOB : L < 2^24 =    16,777,216 Bytes
LONGBLOB   : L < 2^32 = 4,294,967,296 Bytes

MySQL seems to blow up on 64K limits , the blob type is used in a few places.

Drupal running PostgreSQL 12 that I also use doesn't have this issue because it's blob field has different /higher limits.

As a workaround I manually edited my exported YML, replaced \r\n with \n and everything imported correctly with MySQL, however today someone brought up to me a large webform that is unable to save a webform translation in the ui after a certain limit. Seems like two different but similar issues going on, haven't done a deep dive on this yet.

Wonder if this should be reopened?

I've got this issue in Drupal 9 and Drupal 10.
Having the body field or content fields with both editorial content as well as full HTML on larger pages causes this local field validation to fail every single time.

I have to patch each release so that the 10,000+ records and updates will actually complete in my builds.

Most recent Patch attached.
This commits to both Drupal 9.5.10 and Drupal 10.1.2

Maybe increasing size of blob for MySQL could help here as both Sqlite and Pgsql has limit for blobs 1GB but Mysql only 64k

It would be worthwhile looking into increasing the blob size. With that said, 64kb for a locale string is very large.

When I manually replaced \r\n with \n and everything imported correctly with MySQL.
The issue is that the ckeditor outputs \r\n, I'm guessing that a filter needs to be modified in core for the locale processing that treats \r\n the same as \n.

Ideally the common web interface tools should be handled and understood by the locale system.

It seems strange that we'd be translating strings of 64 kilobytes when we could have line items as was originally intended.

With that said, if all of this is too much trouble then please switch the default recommended database system for Drupal from MySQL to PostgreSQL.

Alternatively, perhaps the config system could automatically convert \r\n to \n and then maybe the locale system perhaps would work correctly. Might be easy to create this patch.

Seems like this should be moved out of PNMI.

Tested by changing source field 'mysql_type' from 'blob' to 'longblob'. Why this could not be an option?

Can the issue summary be updated to use a standard template please :)

Also was previously tagged for tests still needed I'm assuming?

I'm going to make some reasonable changes to the merge request. Also, it's not just the source column, it's also the translation column in locales_target that needs it.

Tests are passing.

Rationale: move forward from rediculously low 64kb blob size limit over to a much less restricting 16M mediumblob for mysql/mariadb types (16M is way overkill for this but there's nothing inbetween and mediumblob resolves the issues permanently)

The Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

ok something was done to the main branch that broke the patch, this new MR will not roll on anything older than bleeding edge. Seperate patch will have to be rolled now for D11.4.x and D11.3.x for those unable to run pre-release core dev branch today (most of us).

see comment #54

IMHO this is a won't fix.

The locales tables are intended for translating UI messages. Those aren't expected to be blobs of texts.

If webform or even core are including huge blobs IMHO it's a problem in the consumers, the "API contract" is not expecting that.

As it is triggered in code and a site builder cannot do anything to fix it, I don't think we can repurpose this to improve validation to prevent the problem. A static analysis tool should flag this instead.

@penyaskito , after 13 years open, still open.

Those with PostgreSQL are blissfully unaware of this issue and will never hit the limit because their default blob size is basically not limited to 64kb.

It's very bad UX to have the webform translation crash AND also have drush config import crash on large webform translations resulting in an unusable ui.

No one has figured out a better way.

With that said, I'm not sure why \r\n is still being generated by the various components in core+contrib. Would be nice if that was permanently flushed out in favour of \n (then maybe this would "fix" the glitch?).

Agree with

It's very bad UX to have the webform translation crash AND also have drush config import crash on large webform translations resulting in an unusable ui.

But this is fixing a symptom, not the root cause: this should not be stored where it is right now.

I've mentioned this previously. I am able to work around the console crash during drush config import by replacing \r\n with \n globally in the config/sync/language/de/webform.webform.machine_name.yml however without the patch the ui interface will always crash when saving while \r\n is always re-generated via the GUI interface save operation.

With that said, client has a budget, doesn't care about the details, all the client wants is the issue resolved. When switching the two columns from blob to mediumblob , the issue is resolved, client doesn't care or know enough to care about the plumbing involved with the apparatus. Until there's a better solution, there isn't a better solution.

With that said, client has a budget, doesn't care about the details, all the client wants is the issue resolved. When switching the two columns from blob to mediumblob , the issue is resolved, client doesn't care or know enough to care about the plumbing involved with the apparatus. Until there's a better solution, there isn't a better solution.

If a hack works for you, good for you!
But this is not a good solution for core. I'm closing this as won't fix.

@penyaskito , wow so just close the issue as if it doesn't exist! Maybe it will go away by it'self?

btw, I'm looking into another possible solution relating to the \r\n however a custom module I wrote hasn't actually fixed the web ui crash yet when testing against the blob type with 64kb limit.

As of now, the only known full solution is the patch I uploaded and the merge request.

The rationale for closing isn't entirely clear to me.

One point that still seems unresolved is that PostgreSQL already supports substantially larger values for this field, while MySQL/MariaDB are currently limited by the use of a 64 KB "BLOB". As a result, identical Drupal functionality behaves differently depending on the database backend.

changing the MySQL schema from "BLOB" to "MEDIUMBLOB"

- It significantly narrows the capability gap between MySQL/MariaDB and PostgreSQL.
- It is backward compatible from an application/API perspective, as it only increases the maximum storage capacity.
- It only incurs an additional one-byte length prefix for larger values rather than fundamentally changing how Drupal interacts with the field.

I have discovered 5 known options going forward. The easiest option (other than patching) is to migrate to PostgreSQL using pgloader and abandon mysql. There are a couple of patch options also.

1) patch core
2) patch webform module which affects core schema.

The CRLF stripping/replacement that I discussed earlier relating to switching from CRLF to LF might help for some contrib modules and scenarios but not for others. In the case of the webform 6.3.x module it helps drush cim however does not help the webform translation save which crashes /WSOD on translations larger than 64kb. The webform translation save treats the entire webform translation as one locale and saves it to the locale_target.translation .

This is all fine in PostgreSQL with it's vastly higher default blob limit.

In my test case I used a new custom module (attached to this comment) called locale_newline_normalizer for debugging:

Type	locale_newline_normalizer
Date	Sunday, August 9, 2026 - 15:53
User	joseph.olstad
Location	https://mysecret-drupal/en/admin/structure/webform/manage/example/translate/fr/edit
Referrer	https://mysecret-drupal/en/admin/structure/webform/manage/example/translate
Message	Intercepted webform.webform.example/fr. Source: 91163 -> 91163 bytes, CRLF=0, standalone CR=0, LF=1849. Translation: 70600 -> 70600 bytes, CRLF=0, standalone CR=0, LF=453.
Severity	Notice

The source is 91,163 bytes. That exceeds MySQL's BLOB limit of 65,535 bytes by over 25 KB.
The translation is 70,600 bytes. That also exceeds the  mysql BLOB limit.

The webform translation save GUI failure has nothing to do with CRLF.
So we can retire the CRLF hypothesis for this particular webform translation save failure.

Currently the only known fixes for this scenario is to alter the affected columns blob type from blob to mediumblob OR to migrate to PostgreSQL using pgloader.

The most likely options

1. a core patch ( a variant of MR 12196 )
2. a webform module patch that does exactly what the core patch does
3. rewrite the webform module so that it stores translations into it's own schema instead of using core locale_target.translation , declare a mediumblob for it's schema instead of a blob when dealing with mysql.
4. abandon language support for webforms. (least favourable)
5. migrate from mysql over to PostgreSQL and take advantage of PostgreSQL extremely high default blob limit. pgloader is the de facto standard for migrating MySQL to PostgreSQL. We have successfully and recently done this for a mysql drupal site which involved around 20,000 nodes and approximately 200,000 revisions of these nodes. Most favourable option since the core maintainers are unfavourable towards a change in core.

13 year old issue just won't go away by it'self...