We discussed this at the Berlin i18n sprint that textgroups have various issues for i18n. They are controlled by a single permission, they don't have rich input widgets, they have inconsistent validation. i18n needs to work around filter formats so that it will not make dangerous things translatable accidentally. Then there is the problem that the user contributed things are not always in the site default language, but we still assume that. Finally if people change the site default langugae, all hell breaks loose because there is no practical way to switch around the data, and we don't do anything about it.

We discussed implementing a simple storage mechanism which would be keyed off of the current context value (and/or pieces thereof). i18n already augments the locale tables with a table on its own for this data, so relying on that would not be a groundbreaking change. However, relating data to those would allow us to move the "default language" to the same levels like the others, so if you change the default language, nothing special would happen, it would just work (wow). Also, we could drop most of the insert-time format handling code since we could just handle it on edit, ensuring that via our own UI. i18n module already mostly generates its own UIs for translation with the new translation tabs, so we'd only need to implement a central translation table to mimic the missing locale functionality.

Finally, really the only negative of this migration will be that gettext .po import/export will be lost. That is in fact not widely used (had/has lots of bugs with Drupal 6 and 7). However, we could augment the new i18n storage with a refactored gettext API that could take data from i18n's data sources or push data there for storage.

Tagging for Drupal 8 Multilingual initiative, since this will be crucial for the D8 migration path, so that when we can safely drop textgroups without loosing data.

Related: core issue to get rid of textgroups in Drupal 8 #1188430: Rip out textgroup support from locale module
Related: core issue to refactor gettext API #1189184: OOP & PSR-0-ify gettext .po file parsing and generation - should be made available as a D7 module too

Parent issue (in terms fo Drupal 8 migration path)

#1260628: META: Better translation management API in locale module

Comments

Jose Reyero’s picture

Priority: Normal » Critical
Issue tags: +di18nscb

Since this will be important for forward-compatibility and I wouldn't like to end up maitaining two branches of the 7.x module, I am marking this as critical for first stable release.

Also this is pre-requisite for fixing the 'language default issues'. So I think we should be fixing these two as the last remaining important issues for a 1.0 release.

Steps:
- We need to create a 'i18n_string_text' table and move all the translations there, included the source language. So we'll have a 'i18n_string' with string metadata and 'i18n_string_text' with source text and translation for every language.
- Rework the data storage and retrieving functions that will be simplified by this new schema. Luckily most of this has been encapsulated now on i18n_string_textgroup_default class.
- Note: i18n_string_text table will need a 'format' field, so we can fix too that text format related issues.

klonos’s picture

I don't fully understand all of this issue's goals, but I know that it blocks #1170850: Switching of default language causes change of source language (and the now dupe of it #1164322: Allow users to select what the source language of created vocabularies will be (like we do with nodes).), so following in order to be updated on their progress/status.

renat’s picture

Subscribe.

jox’s picture

Subscribe.

zambrey’s picture

subscribe

Jose Reyero’s picture

Priority: Critical » Normal
Status: Active » Postponed

After giving it a try, this would be a major rework of i18n_strings. I'm afraid not for now if we want a stable release anytime soon.

Also I'm thinking building it as a different module that properly handles *all* strings so we can later replace it. Other option would be creating a 2.x branch if someone else is willing to work on the patch.

Gábor Hojtsy’s picture

That is totally understandable. I did mean this as a proposal for later, not exactly for 1.0. I do work on gettextapi module to provide a generic string API, but I don't think that would blend well with providing any of the i18n string handling code, so that is definitely not the right place for it.

Ashraf Amayreh’s picture

A quick question, is there any dirty tricks to the language default problem? As Gabor stated, assuming the language in all fields is the default language is not right at all, it doesn't make sense to re-edit all field names, block titles, etc when switching the default language. Not to mention, who wants to search for strings in Arabic or Hebrew and translate them to English?

I'd like to also ask why this problem doesn't happen with regular strings? For example, when editing a t() surrounded string, you still get Arabic in the translation options although the default site language is Arabic. Shouldn't the same behavior apply?

Gábor Hojtsy’s picture

@ominds: per definition t() should not be used on user provided or possibly user editable strings. It should be used on code provided strings. If you use it differently, that is wrong. Using t() on user provided strings easily leads to permission escalation and other issues. Therefore we assume strings in t() are always English.

@Jose: update: textgroups are now removed in Drupal 8 #1188430: Rip out textgroup support from locale module

Jose Reyero’s picture

Component: Code » Blocks

Ok, thanks for the updates.

About this, I don't think we are reworking (again) i18n_strings, we really need to look for something better/simpler and do some fresh implementation. The plan goes on like this:

1. Create some 'string API' for modules to create/get/set/update/retrieve strings which would have its own storage.
2. Use it as a replacemente for all i18n string backend (keep the i18n_string_* API though)
3. Propose it as an option to handle D8 strings.

Jose Reyero’s picture

Component: Blocks » Strings

This is the first try for a string API that may replace i18n_string backend later, http://drupal.org/sandbox/reyero/1290346

Jose Reyero’s picture

Version: 7.x-1.x-dev » master

I don't think we'll have the time to address this for 7.x version.

Gábor Hojtsy’s picture

I18n will need an upgrade path though for whatever happens in Drupal 8, eventually.

Gábor Hojtsy’s picture

Issue summary: View changes

Adding parent