Entity translation fallback, several language fallbacks + country dependent fallback + Fallback languages for views

Thanks, this looks promising

1/ Nice
2/ Now the fallback is cascading for strings, so this works: ga > en-ie > en-uk > en it will keep searching till it finds a translation.
3/ I like the idea of country specific fallback, it makes sense for more countries as well, but maybe better in a submodule

There are a lot of dependencies added to the info file, they have to be optional or in a submodule and we should limit them to what is absolutely needed: field_collection, multiple_selects

I had a quick look at the zip, but it contains a patch for bootstrap.inc, is this necessary?

Is it possible to upload a patch against the latest dev version? I think it's best to createe 2 submodules: one for entities and one for the country fallback, so people can still use only for string translation fallback.

The module here is try to make it possible for fallbacks like: ga > en > es, which is different from drupal's default logic that ends with en.
* the patch is for interface translation fallback
* to make submodules, we need to build core language_fallback module as kinda api, a hook_language_fallback_fallbacks_alter to allow submodules changes the fallbacks, and handle the settings form.

I think it's better to build language_fallback module as
* a core language_fallback api module which will accept a fallbacks array, and make the fallbacks. Also it will be great to cover entity_translations fallback
* a sub language_fallback_example module which for single language fallback feature

and we can create dedicated projects like language_fallback_country

"fallbacks like: ga > en > es" for string translations it works like that, the only thing is that in the end you'll always end up with the default language, so it is "fallbacks like: ga > en > es > [default language]"

The translations of strings is completely different from entity translation, so we can not use the localeWithFallback class for it, we can on the other hand re use the "$fallbacks = language_fallback_fallback($langcode);" like you did in language_fallback_language_fallback_candidates_alter, but that we need to move that part to another module 'language_fallback_locale', so we can do something similar for ET (language_fallback_entity_translatation) and maybe in the future for content translation (language_fallback_content_translation). The all will use language_fallback_fallback to get possible translation, but we will be able to manage the dependencies.

Since the country fallback can be used for all types of fallbacks, we probably need our own hook so the language_fallback_country can change it if needed.

Does this makes sense? If so can we start by adding the language fallback for entities, and once that's in add the country specific things?

Example, using fallback ga > en > es

Visitor language is ga and visiting node/1
node/1 exists in ga + en + es ==> visitor sees the node in ga
node/1 exists in en + es ==> visitor sees the node in en
t('xyzzy') translated in ga + en + es ==> visitor sees xyzzy in ga
t('xyzzy') not translated ==> visitor sees xyzzy (whatever the language may be)
node/1 has field_xyzzy in ga + en + es ==> visitor sees the field in ga
node/1 has field_xyzzy in es ==> visitor sees the field in es
node/1 has field_xyzzy but not translated ==> visitor sees the field as is (whatever the language may be)

Visitor language is es and visiting node/1, country of user is set to Spain
node/1 exists in ga + en + es ==> visitor sees the node in ga
node/1 exists in en + es ==> visitor sees the node in es (country fallback)
t('xyzzy') translated in ga + en + es ==> visitor sees xyzzy in ga
t('xyzzy') translated in en + es ==> visitor sees xyzzy in es (country fallback)
t('xyzzy') not translated ==> visitor sees xyzzy (whatever the language may be)
node/1 has field_xyzzy in ga + en + es ==> visitor sees the field in ga
node/1 has field_xyzzy in es ==> visitor sees the field in es
node/1 has field_xyzzy but not translated ==> visitor sees the field as is (whatever the language may be)

Can you upload a patch against the latest dev version as well, it will make it easier for me to review?

#7 I've been very busy, but I had a quick look: it looks good, but - unless I'm wrong - the form on the language settings is removed? Does this mean people will have to use one of the sub modules?

#9 Tough problem, I think we have to choose one storage mechanism for both, uninstalling one to use the other one will complicate things and will likely confuse users.

Globalbility asked me to sort this out so his changes can be committed back to the official release. He specifically requested particular features so I would like to ask for your opinion on implementation details, so I can provide code that will satisfy both of you :)

1. Unique fallback chains per language. Currently the module cascades through all fallback languages until it finds translation or reaches default language. It's OK for simple sites, but may lead to trouble on more complex sites. For example Norwegian may fall back to Danish and at the same time Danish may be set to fallback to Norwegian. So to avoid endless loops and provide accurate fallbacks for each language the unique fallback chains for each language would be needed. Is it OK for you to introduce such a change?
2. Country specific fallback. Globalbility needs different fallbacks according to the visitor country. So even the same language may have different fallbacks depending on the country the visitor is in. This requires additional fallback settings for each language. The problem is that country specific stuff should be optional as not everyone needs it. This could be implemented as a separate sub-module, but overriding base fallback settings form in a separate module or creating API just for this one thing seems overcomplicated. So to keep things simple I would like to build country specific settings into the base module but make that an optional feature that can be enabled if needed.
3. User defined fallback. This is yet another feature requested by Globalbility. Like with country specific fallbacks I would also like to implement this as a part of the base language fallback module. Anyone with "create own language fallback" permission would be able to set their own fallback chains for the language they chose.
4. Settings storage mechanism. Currently the modified version of the module uses entities to store it's settings. I'm not a great fan of using entities for that purpose but it seems that this is the new hot thing for Drupal that everyone is so excited about. Because it is already done, we can stick with that. I think that your problems with settings and making all that stuff work together were because the person that did the modifications moved all the configuration related stuff into country fallback module. If you won't enable it, nothing will work correctly. I'll fix that but I would like to know what is your opinion in this matter.
5. Module structure. You requested to split new features into separate modules. As you probably noticed I would rather incorporate parts related to fallback configuration into one module. I'm not sure if we need that much of fragmentation that there would be need for three or even four modules to implement these simple things.

Please let me know what you think about that so I would know what to do next to make the modified module work the way Globalbility needs.

#11

Thanks for stopping time, I'm abroad for the moment, but some quick responses

1/ AFAIK endless loops are detected in the code, if not it is a bug that needs be fixed, regardless of this issue
2/ Country specific fallback can be included in the main module if it makes the implementation easier, as long as it is an optinal feature.
3/ see 2, but great idea
4/ I had the same reaction, but I think we need to store everything in a database table, and if we do that, I would like it to be easily exportable, I think the ctools approach will be the easiest.
5/ I advocate for separate modules unless it complicates the implementation, if the code is still easy to understand when everything is inside one module, and if all extra's can easily be disabled, I don't mind using one module.

Thanks again!

Thanks for your input. I'll probably change entities to database storage as there is a lot of parameters to store and retrieve. I think that in this particular case entities make it less readable and also less efficient.

Have a look at ctools exportables it gives us an easy way to import and export the configuration.

Please take a look at this version of the module. I've changed storage mechanism from entities to database. The core fallback "engine" is still very simple, only the UI got a bit more complicated. At least in comparison to your original version.

The entity translation and country specific fallbacks are disabled by default. User has to visit module's configuration page to enable these features.

I think that this should be released as the 2.0 version, because we changed the way the fallback chains work and this is significant change. There is no upgrade path yet - I don't think that we would need ctools for that but I'm open for suggestions.

User defined fallbacks are not implemented yet. I would like to know your opinion on the module before I add more features.

Any change you can upload this as a patch as well, it will make it easier for me to review the changes?

Patch against 1.x-dev

I've Fixed few bugs and added country selection block for sites that do not use smart IP for country detection.
No user defined fallbacks yet.

This is yet another and hopefully final version of the modified module.
Some fixes were applied and user defined fallbacks are also implemented.
Let me know what do you think of these changes.

I tried applying the patch, but it failed, can you use git diff to create a new one?

Also can you check the coding standards, the indentation is off, should be 2 spaces for each level.

PS: Sorry for the late reaction

+++ language_fallback-7.x-2.x-dev//language_fallback.admin.inc	2013-08-20 20:46:54.299079074 +0000
@@ -0,0 +1,376 @@
+	//if ($form_id == 'locale_languages_predefined_form' || $form_id == 'locale_languages_custom_form' || $form_id == 'locale_languages_edit_form') {
+
+	// There was a trouble making AJAX to work on "add language" form so it has been disabled.

can you remove commented out code

+++ language_fallback-7.x-2.x-dev//language_fallback.admin.inc	2013-08-20 20:46:54.299079074 +0000
@@ -0,0 +1,376 @@
+		/**/

unneeded comment

+++ language_fallback-7.x-2.x-dev//language_fallback.admin.inc	2013-08-20 20:46:54.299079074 +0000
@@ -0,0 +1,376 @@
+				//'#field_suffix' => '<a href="#" id="language-fallback-remove-' . $ccode . '-' . $i .'">X</a>' ¶

trailing white space

+++ language_fallback-7.x-2.x-dev//language_fallback.admin.inc	2013-08-20 20:46:54.299079074 +0000
@@ -0,0 +1,376 @@
+	 * This also applies the the "remove" button below.

the the --> the

+++ language_fallback-7.x-2.x-dev//language_fallback.admin.inc	2013-08-20 20:46:54.299079074 +0000
@@ -0,0 +1,376 @@
+//	dpm($form_state);

debug code

+++ language_fallback-7.x-2.x-dev//language_fallback.install	2013-08-20 20:46:54.299079074 +0000
@@ -1,49 +1,120 @@
+						->expression('name', "REPLACE(name, '_locale_custom_strings_', 'locale_custom_strings_')")

did you test this on all supported databases?

+++ language_fallback-7.x-2.x-dev//language_fallback.module	2013-08-20 20:46:54.299079074 +0000
@@ -1,65 +1,188 @@
+		$user_cahin = language_fallback_get_user_chain();
+		

$user_cahin --> $user_chain

+++ language_fallback-7.x-2.x-dev//language_fallback.module	2013-08-20 20:46:54.299079074 +0000
@@ -1,65 +1,188 @@
+function language_fallback_get_user_chain() {
+	if (isset($_SESSION['language_fallback']['user_chain'])
+		&& is_array($_SESSION['language_fallback']['user_chain'])
+		&& count($_SESSION['language_fallback']['user_chain']) > 0)
+			return $_SESSION['language_fallback']['user_chain'];

is _SESSION needed? Will this affect performance for anonymous users?

Also, always use {} for if statements

Sorry for the delay. I've created new patch using git.

Sorry for the indentation - my Eclipse is jerking around constantly :( I hope this is fixed now.

did you test this on all supported databases?

No I did not test this with other databases. If you have any suggestions for alternative solutions, then please share. Basically, this part of the code is required even for your basic version of the module, because the module leaves references to the "localeWithFallback" class inside variables table. If you disable the module, drupal no longer loads the file that contains class definition and you end up with fatal error.

is _SESSION needed? Will this affect performance for anonymous users?

Well, we assume that even anonymous users can set up their own fallback chains, so there is no other way to do it. Besides there is an optional smart_ip module in use, which also stores data inside the $_SESSION variable... and if it didn't then that would be a massive performance hit :)

So yes, in this use case session variables are required.

Thanks for the feedback, I'm still struggling with the SESSION because it means that caching (varnish, nginx) will be no effective, can we make this part optional so site administrators can opt in/opt out

Regarding the smart_ip module, my servers are running on top of barracuda and there the IP to country resolution is done by nginx by using http headers: X-GeoIP-Country-Code, X-GeoIP-Country-Name so it would be nice if that can be used as well

I've changed the way $_SESSION is used. Now it will only be set if one of these is true:

1. smart_ip module is enabled. It would be very unwise not to cache the geolocation results that came from external sources.
2. One of the module blocks is enabled and user will set his own fallback chain or set his own country. If these blocks are not used the session variable is also not used.

There is also one new feature. The module offers new language filter option for views. You can select "Fallback languages" and filter will use all languages from the list of fallback candidates offered by core language_fallback_get_candidates() function.

Sorry, wrong patch uploaded. This one is OK.

Hi, can you please let us know if you are going to apply this patch?

#26 Yes, I'm planning to create a 7.x-2.x version containing this patch, but there are a lot of whitespace and EOL errors, I fixed most of them but can you have a look at the code.

I've cleaned up leading and trailing whitespaces and also fixed few comments here and there to make them more drupalish. This is a patch against head.

Great feature to have language fallback in views. But it seems this is not working?

I tried #23 and #28 patches. I had to create language_fallback table manually, because language_fallback_schema() didn't create the table.
Then I had none of the patches showed up the new language filter option for views : "You can select "Fallback languages" ..." .

Could you tell me if there is a way to get this working ?

As a matter of fact there isn't any new filter. The module just extends existing language filter by adding another option to chose from when setting up the language filter. Just add ordinary language filter and you should see new option like the one on the screenshot.

Thanks SiliconMind. This is exactly what I meant and understood, but despite many installing/uninstalling/patching tries ... the option Fallback Language never appeared.

AFAIK, to get your module work at this point (as it is completely re-writed), I have to apply patch #28 on dev version, and only then install the patched project... if not, the install process is not fully performed. And anyway doing so, database table is not created, as explained in #29.

So can you give me once a step by step process, or a git branch I could use, to ensure I'm doing right ?

Thanks in advance.

You can get fully working module from github repo.

If current patch is not working it might be due to the fact that dev release of language fallback module changes once in a while. I don't know if maintainer is still willing to add these features, so it's kind of pointless to re-roll those patches over and over again. Use the repo from github. It's tested and works fine.

All,

I'll try to have another look to see if we can merge it, I somehow missed #28

Thanks a lot attiks, SiliconMind, for your time and answers. I'll use github repo then, and follow this issue future posts :)

@attiks:
Any updated on this issue?

We mamaged to get a unified system for handeling fallback languages in Drupal 8:
https://www.drupal.org/node/2122175

If you add the patch so we can continue development of this module and improve it together, it will also make it easier to port these functions later to Drupal 8 module.

Hope this issue will not take another year...

I'll ask Jelle to have a look tomorrow

The patch can not be committed as-is, since there is no upgrade path (no hook_update_N() that creates the new table, migrates old settings to the new table, removes the fallback field from the languages table).

Jelle, I can write the update code, I'ts not a problem, but the fundamental question is: are you actually willing to commit this code? It would be pointless to waste time for creating update code and get rejected after that anyway.

#38 It was always the idea to merge this, I only lost track, my bad

OK, thanks for the info. I'll figure out something and post a patch with upgrade path within few days.

This patch includes update code for 1.x branch. Should work fine but it would be great if someone currently running 1.x version could test this too.

I did a quick patch review, there are a lot of code style issues, but these can be fixed be a follow-up patch, so I propose we commit this to a new 7.x-2.x branch so people can start testing.

1. +++ b/language_fallback.admin.inc
   @@ -0,0 +1,351 @@
   +      // Get all countries that we have a fallback for this language
   

   indentation

2. +++ b/language_fallback.admin.inc
   @@ -0,0 +1,351 @@
   +        while($c = $result->fetchField()){
   

   space after while and before {

3. +++ b/language_fallback.admin.inc
   @@ -0,0 +1,351 @@
   +      // Unset countries that are in use
   

   missing dot

4. +++ b/language_fallback.admin.inc
   @@ -0,0 +1,351 @@
   +            'callback' => 'language_fallback_country_add_remove_ajax_callback',
   +            'wrapper' => 'countries-fieldset-wrapper',
   

   indentation

5. +++ b/language_fallback.admin.inc
   @@ -0,0 +1,351 @@
   +      //$options = array('' => t('Default language (@lang)', array('@lang' => $default_lang->name))) + $options;
   +      //unset($options[$default_lang->language]);
   

   I assume this can be removed

6. +++ b/language_fallback.admin.inc
   @@ -0,0 +1,351 @@
   +      '#submit' => array('language_fallback_country_add_language_add_another'), // This is shared across all countries
   

   no inline comments

7. +++ b/language_fallback.admin.inc
   @@ -0,0 +1,351 @@
   + * @param unknown $form
   + * @param unknown $form_state
   

   these are arrays

8. +++ b/language_fallback.block.inc
   @@ -0,0 +1,147 @@
   +  $prop = 'name'; // Change to "native" to use native names;
   

   should this be an option?

9. +++ b/language_fallback.block.inc
   @@ -0,0 +1,147 @@
   +  } else {
   

   split in 2 lines

10. +++ b/language_fallback.module
    @@ -1,64 +1,230 @@
    +module_load_include('inc', 'language_fallback', 'language_fallback.admin');
    +module_load_include('inc', 'language_fallback', 'language_fallback.block');
    

    better to load when needed

Pushed to 7.x-2.x with coding style issues addressed as well.
I'll create a dev release so people can start testing.

Cool! :)