$node->foo[$what][$delta]['value'] about 99% would write LANGUAGE_NONE in place of $what. I wouldn't know what else to write. There's field_get_items with this super helpful doxygen:

<?php
 
* @param $langcode
 
*   (optional) The language code $entity->{$field_name} has to be displayed in.
 *  
Defaults to the current language.
?>

There's also the minor problem of 1 call to field_get_items in core altogether (in file.module).

There should be a field_set_items as well and a way better understanding of wtf that $langcode should be.

Files: 
CommentFileSizeAuthor
#63 drupal-improvedocs-1232120-62.patch518 bytesRyan Weal
PASSED: [[SimpleTest]]: [MySQL] 36,273 pass(es).
[ View ]
#61 drupal-improvedocs-1232120-61.patch594 bytesRyan Weal
PASSED: [[SimpleTest]]: [MySQL] 36,280 pass(es).
[ View ]
#45 drupal-improvedocs-1232120-5792176.patch2.61 KBRyan Weal
PASSED: [[SimpleTest]]: [MySQL] 35,914 pass(es).
[ View ]
#43 drupal-improvedocs-1232120-5792114.patch2.65 KBRyan Weal
PASSED: [[SimpleTest]]: [MySQL] 38,633 pass(es).
[ View ]
#42 drupal-improvedocs-1232120-5792070.patch2.61 KBRyan Weal
PASSED: [[SimpleTest]]: [MySQL] 35,914 pass(es).
[ View ]
#40 drupal-improvedocs-1232120-5784662.patch3.88 KBRyan Weal
PASSED: [[SimpleTest]]: [MySQL] 35,899 pass(es).
[ View ]
#37 field-improvedocs-1232120-c37.patch4.24 KBYesCT
PASSED: [[SimpleTest]]: [MySQL] 35,909 pass(es).
[ View ]
#35 field-improvedocs-1232120-c35.patch4.24 KBYesCT
PASSED: [[SimpleTest]]: [MySQL] 35,899 pass(es).
[ View ]
#35 diff-c29-c35.txt2.2 KBYesCT
#29 field-improvedocs-1232120-5775454.patch4.32 KBRyan Weal
PASSED: [[SimpleTest]]: [MySQL] 35,718 pass(es).
[ View ]

Comments

Gábor Hojtsy’s picture

Version:7.x-dev» 8.x-dev
Issue tags:+D8MI
chx’s picture

Version:8.x-dev» 7.x-dev
Issue tags:-D8MI

Earlier plach told me that $langcode is not simply optional but most of the time it's not needed. Documenting that is a good first step.

chx’s picture

Version:7.x-dev» 8.x-dev
Issue tags:+needs backport to D7, +D8MI

Ops crosspost

plach’s picture

Issue tags:-needs backport to D7

I'd need to know what in http://api.drupal.org/api/drupal/modules--field--field.multilingual.inc/... needs to be improved (high level descritpion, internal descritpions, missing pieces), and please don't answer everything. That would not help ;)

plach’s picture

Issue tags:+needs backport to D7

crosspost :)

mikl’s picture

Yeah, the data access has become so convoluted that I see many devs using EntityDrupalWrapper (from the Entity API module) to avoid the mess of language prefixes, etc. Would be nice if there was a cleaner API for it.

webchick’s picture

chx, can you please tone down the rhetoric? It's not constructive, and is tearing down hard work that folks like plach have done. Post suggestions for improvement, not criticisms.

Here's a shot at a skeleton for that page, which totally needs work, but is a place to start (I know we can't do headings in doxygen but this is just to organize.)

About Multilingual Fields

Fields natively implement multilingual support, which is optionally set per-field. This is useful in situations where ... . Situations where you might not want a field translatable are ... .

(Basically, provide an overview of the feature and the thinking behind it.)

Multilingual Field Structure

All fields use the following structure:

$entity->{$field_name}[$langcode][$delta][$column_name]

* For untranslatable fields $langcode is ...
* For translatable fields $langcode is ...

Accessing Multilingual Fields

As a module or theme developer, here's the proper way to access a field's property.

// Example code, and rationale.

Multilingual Field API

// The stuff about _field_invoke() etc.

xjm’s picture

Title:Remove field multiple language system or document it» Improve documentation of field multiple language system
larowlan’s picture

Didn't know field_get_items existed.
/me refactors his code.
Definite +1

Boobaa’s picture

Having a single-value userreference called field_user, who needs to be added to a multi-value userreference called field_user_history, but only if s/he is not yet there. The best place to achieve this seems to be nook_node_presave(). I was struggling with forging the set by cycling through $node->field_user_history[$node->language], when I was pointed to a blog post of @davereid. Then I switched to using field_get_items('node', $node, 'field_user_history') which I do understand is a more straightforward way to do so. Now I can properly build the set for my aforementioned use case.

OTOH the question remains unanswered: what is the setter counterpart of it? IOW: how to properly assing value to a field? I was trying $node->field_user_history[$node->language] = $set, but this didn't work (I have even tried hook_field_attach_presave() and basically every node API hook I thought to be to-the-point, without any success). Then I tried $node->field_user_history[LANGUAGE_NONE] = $set and my code began working, even as simple as in a hook_node_update().

So: how should one assign value to a field properly? and why should one do it that way?

tim.plunkett’s picture

In IRC, davereid suggested $node->field_foo[field_valid_language($node->language)] = $items.
Also, subscribe.

Boobaa’s picture

As I see field_valid_language($node->language) is only something like a safety measurement to ensure that specific language code (still) does exist, but it does not explain me too much. Why should I be using $random_language (safeguarded by this function or not), or why am I enforced to use LANGUAGE_NONE? Basically that's what I want to know from the smallish coder's point of view.

Gábor Hojtsy’s picture

The best docs I found on this so far are in http://api.drupal.org/api/drupal/modules--field--field.multilingual.inc/... (its one of the few topics from the API navigation screen).

First thing to note from there is that there can be two types of fields: translatable and not translatable. Unless you have the entity_translation module enabled and you run the latest D7 release (where related bugs where fixed), all entities will be created untranslatable. Older Drupal 7 releases had a bug where fields got created translatable.

A key thing to understand from there is that the set of languages you might be able to save data to depends on a setting on the field (whether it is translatable or not). If it is translatable, then it also depends on another set of settings (the list of languages on the site). So when saving a field, you need to consider the field type and then whether the language you are saving into is on the site (if a translatable field).

http://api.drupal.org/api/drupal/modules--field--field.multilingual.inc/... is then the API function to gather a list of languages that one field can be saved in. It considers if the field is translatable and if it is, will consider the available languages on the site. So you get a list of language codes that you can save your field in.

Then which one of these you save your field in depends on your context. If you know the language of the data that is being saved, use that. Informed from node_form() (http://api.drupal.org/api/drupal/modules--node--node.pages.inc/function/...) (or locale module's field translation code), you can see that $node->language is used to gather input on the node data in core. That is since Drupal core does not have a UI for entering translated fields, so it will always display a UI for the field in the node's language. The entity_translation module has a UI and hooks to save fields in other languages than the main language of the node.

At this point I'm not sure you actually want this in a node form submission process, but if you do, entity_translation module should be investiagated for how it allows you to plug into that process (and in that case I'm sure it will tell you the exact language code to use, like node_form does for field_attach_form()).

So in short:

<?php
$list_of_language_codes
= field_available_languages('node', $field_data);
$language_code = somehow_pick_what_i_want($list_of_language_codes);
$node->field_foo[$language_code] = $items;
?>

Now, field_valid_language() is interesting in that it would default to the content display language selected for the page . That information in itself is useful for somehow_pick_what_i_want() but depending on the nature of the data, it could be wrong. If you have a CSV with data in 5 languages, and you need to import that, you just need to know the 5 language codes and use them specifically. If you have some kind of custom form that then updates a field value on an entity, the page content language (global $langugae_content) selected for that page sounds like a safe bet to default to though.

However, the sole use of field_valid_language($node->language) is not good, since for non-translatable fields, the field value cannot be saved in arbitrary languages, it will always be LANGUAGE_NONE. And whether a field is translatable or not is a per-field configuration thing, which field_valid_language() does not care about.

Strong disclaimer: I did not participate in the building of this feature, neither I say I understand how it works, this is merely a recollection of what I'm seeing in the existing code and docs and would welcome corrections. I'm definitely not an authoritative source. Awaiting verification.

Gábor Hojtsy’s picture

#1214472: Undefined index: und in taxonomy_access_form_alter() -- incorrect assignment of language key seems like a good issue where better understanding of the system would be very helpful. Just throwing in as a use case to cover in docs.

webchick’s picture

Category:bug» task

Not to get pedantic but this is a task, not a bug.

plach’s picture

Thanks for posting examples here, they are really useful to understand which are the scenarios in which the Field API is being used. I'd like to see more code exploiting the Field API to craft better examples on how to manage multilingual fields correctly.

No time to expand on this now, just wanted to say that #13 is more or less correct. I should be able to provide details later today.

xjm’s picture

Patterns for specific use-cases would be very helpful. E.g., how to properly handle language codes:

  • On generic entities (not necc. nodes)
  • In a form alter
  • During form validation and submission
  • In hook_field_*()
  • In hook_node_*()
  • Etc.
xjm’s picture

Re: Examples of how Field API gets used: I have some half-broken examples and some crazy workarounds in TAC. They're probably too big to post snippets here, but I attempt to handle language codes in the following places. (Warning: high WTFs.)

drupalcontrib.org seems to be a month or more stale, so I can't link them there. Repo link:
http://drupalcode.org/project/taxonomy_access.git/blob/refs/heads/7.x-1....

xjm’s picture

One thing that I got seriously stuck on (hours of debugging) is that one needs to pass an array keyed by langcode to use form_set_value() on a field element in a form. Quite different from D6 there.

Gábor Hojtsy’s picture

Issue tags:+documention

Tagging.

Gábor Hojtsy’s picture

Uh, tag fix.

plach’s picture

Issue tags:+translatable fields

I'd like to stress on one thing that maybe obvious, but perhaps that has not been sufficiently outlined: fields may hold multiple items array, one for each available language. The details are explained in the opening paragraph of the current documentation, but the point is that if a field is translatable any operation, except viewing and adding an array of items to it for a particular language, needs to be done on every language available in the field structure. That's how the Field API works: if you pass it an $entity object where fields are populated with as much as languages as you want, it will know how to handle it, as it will simply act on every value available. The only exceptions to this behavior are:

  • field_attach_view(), as one usually needs to display one language value at the time. It accepts an optional language parameter, if this is not present the current language is used. field_get_items() is the per-field counterpart, and should be used in the view/render phase, since it defaults to the language the field would be dispayed in.
  • field_attach_form(), since our core UIs allow only one value to be entered for a particular language at a single time. Altough multilanguage field forms could be introduced in contrib, this is by far the most common use case. Here we are missing a per-field counterpart, basically because the assumption is that one knows in which language a value is being entered, but this is true only when editing values, not when adding them. A field_set_items() function might help here in providing a sensible default for the source language if it is unknown/unspecified.

Another thing to keep in mind is that nothing prevents any field to eventually become translatable, it's just matter of switching a flag in the database. AAMOF translatable (and this can be misleading terminology) means that it can have a different value for each available language, not that its type is inherently translatable: a map might not be "translatable" in itself but I might want to have a different map for every available language. Hence every field should be treated as potentially translatable, relying on it being created as non-translatable might lead to having to fix its behavior later.

However, after talking with @chx in IRC some days ago, we agreed that to bring people's understanding of field multilingual support to an acceptable level we need several steps:

  • Improve the available documentation, the suggestion in #7 seems like a good start.
  • Provide code samples on d.o. explaining the available best practices.
  • Try to fix core tests, which are really a poor example at the moment, since they uniformly use hard-coded LANGUAGE_NONE values almost everywhere. This might not be feasible after all, since if you know which language the field is supposed to have, it's correct to use it: in most core tests we don't enable Locale, hence LANGUAGE_NONE is the only possibile value while asserting results.
  • Improve Field API introducing field_set_items(), we need a separate issue for that.

Meanwhile, if people has more example to post, they are really welcome. They will be useful to design examples and field_set_items() itself.

plach’s picture

Component:language system» documentation
Issue tags:-documentation
jhodgdon’s picture

Can someone please update the issue summary so I don't have to read all the comments to figure out what the problem and proposed solution is here? Thanks...

xjm’s picture

Er, tagging?

plach’s picture

Issue tags:+tfdx

API improvements issue: #1260640: Improve field language API DX.

Gábor Hojtsy’s picture

Issue tags:+language-content

Tagging for the content handling leg of D8MI.

Ryan Weal’s picture

Assigned:Unassigned» Ryan Weal
Ryan Weal’s picture

Status:Active» Needs review
StatusFileSize
new4.32 KB
PASSED: [[SimpleTest]]: [MySQL] 35,718 pass(es).
[ View ]

Hey everyone, here is a patch for the field.multilingual.inc file to provide complete documentation. Gabor and I spent a lot of time on this, please let me know if there are any revisions as it is my first attempt at doxygen formatting.

The new text is as follows (slightly adapted from doxygen, sorry about the word wrap) -

Handling of multilingual fields.

Fields natively implement multilingual support by holding multiple values
of the field array. All fields use the following structure:

$entity->{$field_name}[$langcode][$delta][$column_name]

By default a single-language site in English without the locale module
enabled uses LANGUAGE_NONE as the $langcode, and there is always only
one language variant for each field.

$entity->body[und][0][value];
$entity->langcode = 'und';

With Locale module enabled (as well as with Content Translation), fields
will not have their own language information, but LANGUAGE_NONE is used
to signify that we don't have that specified. The whole entity with all
its properties and fields is assumed to have the entity language code
assigned (conceptually).

$entity->body[und][0][value];
$entity->langcode = 'en';

Field translation is pluggable, but the only known implementation
is the contributed Entity translation module that provides a user
interface to field-translation enable entities and change the
translatability on fields.

When we enable field translation, a field that does not have translation
enabled will always be LANGUAGE_NONE, and no other language values will be
present. The meaning of this is different from the above. It means that the
field will have the same value across all translations. This is also called
a shared field. The $entity->langcode value will persist to represent the
language of the properties of the entity, such as the title, author as well
as signify the original language that the entity was saved with. That
information can be useful for permission and workflow purposes.

$entity->body[und][0][value]; // not translatable (shared)
$entity->langcode = 'en'; // applies to properties of the entity

For a field with translation enabled in field settings, multiple variants
of the value may be available in any of the enabled languages. Data that was
created prior to enabling the translation will remain, however, it will be
ignored by Drupal. The LANGUAGE_NONE version of the field in this case
serves only to preserve the historic configuration and can be safely deleted
when it is no longer used.

$entity->body[en][0][value]; // translatble
$entity->body[und][0][value]; // ignored by Drupal
$entity->langcode = 'en'; // applies to properties of the entity

The available languages for a particular field are returned by
field_available_languages(). Whether a field is translatable is determined by
calling field_is_translatable(), which checks the $field['translatable']
property returned by field_info_field(), and whether there is at least one
translation handler available for the field. A translation handler is a
module registering itself via hook_entity_info() to handle field
translations.

Multilingual Field API

By default, _field_invoke() and _field_invoke_multiple() are processing a
field in all available languages, unless they are given a language
suggestion. Based on that suggestion, _field_language_suggestion() determines
the languages to act on.

Most field_attach_*() functions act on all available languages, except for
the following:
- field_attach_form() only takes a single language code, specifying which
language the field values will be submitted in.
- field_attach_view() requires the language the entity will be displayed in.
Since it is unknown whether a field translation exists for the requested
language, the translation handler is responsible for performing one of the
following actions:
- Ignore missing translations, i.e. do not show any field values for the
requested language. For example, see locale_field_language_alter().
- Provide a value in a different language as fallback. By default, the
fallback logic is applied separately to each field to ensure that there
is a value for each field to display.
The field language fallback logic relies on the global language fallback
configuration. Therefore, the displayed field values can be in the
requested language, but may be different if no values for the requested
language are available. The default language fallback rules inspect all the
enabled languages ordered by their weight. This behavior can be altered or
even disabled by modules implementing hook_field_language_alter(), making
it possible to choose the first approach. The display language for each
field is returned by field_language().

Accessing Multilingual Fields

@see field_get_items()

See @link field Field API @endlink for information about the other parts of
the Field API.

andypost’s picture

Status:Needs review» Needs work
+++ b/core/modules/field/field.multilingual.incundefined
@@ -10,20 +10,59 @@
+ * Fields natively implement multilingual support by holding multiple values ¶

trailing whitespace... a lot of

YesCT’s picture

I'm going to fix some spelling and whitespace

Kristen Pol’s picture

Status:Needs work» Needs review

Good job Ryan! I have some feedback below:

Handling of multilingual fields.

Fields natively implement multilingual support by holding multiple values of the field array. All fields use the following structure:
$entity->{$field_name}[$langcode][$delta][$column_name]

By default a single-language site in English without the locale module enabled uses LANGUAGE_NONE as the $langcode, and there is always only one language variant for each field.
$entity->body[und][0][value];
$entity->langcode = 'und';

This looks good, but I would change "locale" to "Locale" for consistency with the next section and to use the human-readable module name.

With Locale module enabled (as well as with Content Translation), fields will not have their own language information, but LANGUAGE_NONE is used to signify that we don't have that specified. The whole entity with all its properties and fields is assumed to have the entity language code
assigned (conceptually).
$entity->body[und][0][value];
$entity->langcode = 'en';

1) Minor: I would change "With Locale module" to "With the Locale module".

2) I'm a bit confused with this part of the first sentence:

fields will not have their own language information, but LANGUAGE_NONE is used to signify that we don't have that specified

This makes me think that either:

* it should be "...fields will have..." (because of the "but")
or
* it should be "..., and LANGUAGE_NONE is used..." (because of the "will not")

My guess is it is the latter.

Field translation is pluggable, but the only known implementation is the contributed Entity translation module that provides a user interface to field-translation enable entities and change the translatability on fields.

Looks good I would probably change the last part to:

...provides a user interface to enable field translation for entities and change the translatability on fields.

When we enable field translation, a field that does not have translation enabled will always be LANGUAGE_NONE, and no other language values will be present. The meaning of this is different from the above. It means that the field will have the same value across all translations. This is also called a shared field. The $entity->langcode value will persist to represent the language of the properties of the entity, such as the title, author as well as signify the original language that the entity was saved with. That information can be useful for permission and workflow purposes.
$entity->body[und][0][value]; // not translatable (shared)
$entity->langcode = 'en'; // applies to properties of the entity

Minor:

Formatting for "entity, such as the title, author as well as" should probably change to something like "entity, such as the title and author, as well as".

Also:

Using the "title" as a property example might be misleading as often we would enable the Title module in order to get a translatable title field (which is separate from the property title) for node entities. You might want to use "author and status" as examples for properties.

For a field with translation enabled in field settings, multiple variants of the value may be available in any of the enabled languages. Data that was created prior to enabling the translation will remain, however, it will be ignored by Drupal. The LANGUAGE_NONE version of the field in this case
serves only to preserve the historic configuration and can be safely deleted when it is no longer used.
$entity->body[en][0][value]; // translatble
$entity->body[und][0][value]; // ignored by Drupal
$entity->langcode = 'en'; // applies to properties of the entity

Looks good except "translatble" should be "translatable" in comment. Interesting to know about the historic 'und' data!

The available languages for a particular field are returned by field_available_languages(). Whether a field is translatable is determined by calling field_is_translatable(), which checks the $field['translatable'] property returned by field_info_field(), and whether there is at least one
translation handler available for the field. A translation handler is a module registering itself via hook_entity_info() to handle field translations.

Looks good.

Multilingual Field API

By default, _field_invoke() and _field_invoke_multiple() are processing a field in all available languages, unless they are given a language suggestion. Based on that suggestion, _field_language_suggestion() determines the languages to act on.

I would change "are processing" to "process".

Most field_attach_*() functions act on all available languages, except for the following:

- field_attach_form() only takes a single language code, specifying which language the field values will be submitted in.

- field_attach_view() requires the language the entity will be displayed in.

Since it is unknown whether a field translation exists for the requested language, the translation handler is responsible for performing one of the following actions:

- Ignore missing translations, i.e. do not show any field values for the requested language. For example, see locale_field_language_alter().

- Provide a value in a different language as fallback. By default, the fallback logic is applied separately to each field to ensure that there is a value for each field to display.

The field language fallback logic relies on the global language fallback configuration. Therefore, the displayed field values can be in the requested language, but may be different if no values for the requested language are available. The default language fallback rules inspect all the enabled languages ordered by their weight. This behavior can be altered or even disabled by modules implementing hook_field_language_alter(), making it possible to choose the first approach. The display language for each field is returned by field_language().

It is not obvious to me what "the first approach" is in the second-to-last sentence.

Accessing Multilingual Fields

@see field_get_items()

See @link field Field API @endlink for information about the other parts of the Field API.

Looks good.

:)

Gábor Hojtsy’s picture

Status:Needs review» Needs work
+++ b/core/modules/field/field.multilingual.incundefined
@@ -10,20 +10,59 @@
+ * to signify that we don't have that specified. The whole entity with all ¶
+ * its properties and fields is assumed to have the entity language code ¶
...
+ * translatability on fields. ¶
...
+ * When we enable field translation, a field that does not have translation ¶
...
+ * language of the properties of the entity, such as the title, author as well ¶
+ * as signify the original language that the entity was saved with. That ¶

Line end whitespace, among other places.

+++ b/core/modules/field/field.multilingual.incundefined
@@ -10,20 +10,59 @@
+ * Data that was created prior to enabling the translation will remain, however, it will be ignored
+ * by Drupal. The LANGUAGE_NONE version of the field in this case serves only to preserve the ¶

These are over 80 chars.

dawehner’s picture

+++ b/core/modules/field/field.multilingual.incundefined
@@ -10,20 +10,59 @@
+ * enabled uses LANGUAGE_NONE as the $langcode, and there is always only

lets use LANGUAGE_NOT_SPECIFIED as that's how it's named in d8.

YesCT’s picture

StatusFileSize
new2.2 KB
new4.24 KB
PASSED: [[SimpleTest]]: [MySQL] 35,899 pass(es).
[ View ]

only fixed whitespace and spelling from comment 29. this patch does not fix the things pointed out in comments 32 or 34.

YesCT’s picture

still has a line wrap problem

YesCT’s picture

StatusFileSize
new4.24 KB
PASSED: [[SimpleTest]]: [MySQL] 35,909 pass(es).
[ View ]

line wrap fixed

Gábor Hojtsy’s picture

Kristen brought together the above docs text with two figures I've created today to help document this and posted at http://drupal.org/node/1500308 - I've made some slight edits to the final part on the API functions to try and make some more sense of it. I think we should cycle back any improvements from here that are not there yet and then decide if we want to have it all in the API docs as well or having it on d.o is better and should be linked from the API docs instead.

Also, of course, several things might be better explained then our current version does, I think the a way better understanding of wtf that $langcode should be that chx was looking for in the opener should be achieved with the two added figures and the explanation :) I hope :)

jhodgdon’s picture

I would suggest that since that doc on d.o has images etc., and it's difficult to maintain documentation in two places, that we have a minimal version in the API docs, with a link to that page.

Ryan Weal’s picture

Status:Needs work» Needs review
StatusFileSize
new3.88 KB
PASSED: [[SimpleTest]]: [MySQL] 35,899 pass(es).
[ View ]

Here is a patch to the field module for D8. I am naming the file using "drupal" as the project, please correct me if this is not the best practice. I'm putting it this way because the URL for the "project" is drupal.org/project/drupal . Thanks!

Ryan Weal’s picture

Status:Needs review» Needs work

Setting back to needs work since it includes two files when it should contain one. Sorry guys.

Ryan Weal’s picture

Status:Needs work» Needs review
StatusFileSize
new2.61 KB
PASSED: [[SimpleTest]]: [MySQL] 35,914 pass(es).
[ View ]

Here is the patch for D8 again. Apologies again for that patch that does too many things at once :-/ I'm still new at this.

Ryan Weal’s picture

Version:8.x-dev» 7.x-dev
StatusFileSize
new2.65 KB
PASSED: [[SimpleTest]]: [MySQL] 38,633 pass(es).
[ View ]

Rolled a backport to D7

tim.plunkett’s picture

Version:7.x-dev» 8.x-dev

No worries :)

Leave for D8, it can be rerolled once committed. If you'd like to keep uploading them, append -do-not-test.patch to the end.

Ryan Weal’s picture

StatusFileSize
new2.61 KB
PASSED: [[SimpleTest]]: [MySQL] 35,914 pass(es).
[ View ]

Reposting that D8 patch because I was not aware changing the issue status to D7 messes up testbot. ;)

Kristen Pol’s picture

Status:Needs review» Reviewed & tested by the community

Looks good! I looked at the documentation and applied the patch with no issues.

Gábor Hojtsy’s picture

Issue tags:+sprint

Put on the sprint tracker for D8MI.

Dries’s picture

As someone who spends a lot of time working in off-line mode (e.g. airplanes), I'm not a big fan of moving documentation out of the code to drupal.org. I'm not sure what the rationale is; it isn't mentioned in the comments or in the issue summary.

Gábor Hojtsy’s picture

@Dries: we posted the text that was originally destined for the API docs inside code at http://drupal.org/node/1500308 with two figures that should help understanding the concepts a huge deal. It makes sense to "duplicate" it in our API docs, but we don't have any way/process to include supporting figured in our API docs shipped with core, so we need to keep the d.o page in any case. I've thrown up this question of whether we want to have the text in the Drupal-shipped API docs (which I agree would be valuable), and jhodgdon commented it looks best to have it at one place and link there. See http://drupal.org/node/1232120#comment-5782440. We did not get this back to the summary. What do you think?

jhodgdon’s picture

My take on drupal.org vs. in-code/api.drupal.org is this:

Function basics (params, return, description of what it does, etc.) definitely belongs in-code, and needs to be maintained with the code. This also applies to classes, methods, constants, hooks, etc.

Reference information, such as the much-loved and much-hated Form API reference, also belong in the code and on api.drupal.org.

Several-paragraph overviews of groups of functions also belong in-code, to give "road maps" (this uses @defgroup, and makes the Topics list on api.drupal.org). For example, a list of all of the hooks that are invoked during each node-related operation, or a list of the hooks you would need to implement to create a field module (in both cases the hook pages would give you more details about each hook).

Tutorials and lengthy examples for how to do things with the Drupal API belong on drupal.org or in the Examples for Developers project. For example, a step-by-step "how to create a field module" tutorial, or a sample field module.

Right?

So in this case, I think we want to distill the essentials of the multiple language system into the group/topic that is in the code, but we don't want to give a full tutorial there, and as noted above, we don't have a good way to include images (and I don't think we would want to include API docs images in the Drupal Core code base).

Of course, the dividing line between "tutorial and lengthy example" vs. "several-paragraph overview" may not be clear... we can definitely tweak it until we figure out what is "just right".

Gábor Hojtsy’s picture

Status:Reviewed & tested by the community» Needs work

I think it would be good to include the essence of the docs in code, and have the extended docs with images on d.o.

plach’s picture

Honestly I don't see the reason to dump the current in-code documentation. IIRC it holds fairly lower-level info than the (definitely good) online docs. What about adding the reference without removing the current docs?

Gábor Hojtsy’s picture

@plach: the goal of the online docs page at http://drupal.org/node/1500308 was that it would have *more* information then the API docs, not that it would have *different* information. Basically Kristen worked on setting that doc page up to have a better intro to what field language means even (it means different things in different cases), and then provide API pointers for different cases.

If the online docs page does not have all the information that we have in the API docs, then that is a mistake. I think the information should be cross-merged (ie. improvements from the doc page vs. what is in API should go to the docs page and vice versa). I've said this above in #38.

plach’s picture

Ok, I'll have a deeper look to differences. However, FWIW, I think keeping online docs and in-code docs should be a non-issue, in the sense that as @jhodgdon was pointing out they should be different, and the intended audience should be too, probably. When I reviewed the new docs I thought they were a higher-level complement to the in-code docs. Having PHP docs living outside code does not really make sense to me.

Again, I find the new docs great but I don't think they should replace the existing ones: maybe we want to merge the last part, which is more technical, in the PHP docs and leave the reference to the online docs for a more high-level description?

However, I'll crosscheck and see if I can make more concrete proposal.

Gábor Hojtsy’s picture

@plach: well, there is nothing stopping from online docs from being deeply technical, the database API docs at http://drupal.org/developing/api/database are pretty darn technical (and there is not much in the code docs about them).

Anyway, our goal here was to have good docs on what multilingual fields are about and how people can use them via the API. If both the API docs and the online docs satisfy that (and/or at least they are cross-linked so people can fill in the details from one to the other), I think we are good. After all it is up to the issue openers and people who did not understand this before to tell if the new docs are good.

@chx? @Dave Reid? @Booba?

Boobaa’s picture

First of all, thanks for turning on the light bulb in my head (whilst it's a bit dim as of now, but definitely there's hope).

Secondly, it seems to me that the debate is not about the documentation itself (which is pretty well-to-the-point and easy-to-understand, congratulations), but the format/style/availability of it. @Dries (and some others, including me) wants documentation that's right in the code (to be able to have a look at it while flying high). Anyway, working with Drupal means always-be-connected (for me), since the (API) docs are only one source of information–and there are times when it's not even the best source. I think this particular debate (about having an image as an (API) docs) belongs to this category: it's easier to understand if it's drawn, but it would be great if it would be available offline.

OTOH, most of the stuff available at http://drupal.org/node/1500308 could definitely be in-code; particularly the flowchart is the only thing that I can't imagine how to reproduce in-code. So, I for one am with @Dries this time to have in-code documentation as much as possible, but with a reference to the only thing that cannot reside there: the flowchart. (This way the documentation could be versioned _with_ the code itself, and not _besides_ the code.)

Additionally, I had a really off-topic and far-reaching idea. If there were some automation that generates offline-browsable format of the online documentation (CHM, bunch of HTML files and stuff, or even PDF), then we could have the values from both the worlds: the images and offline availability. Anyway, I think being online while flying will be available much more sooner than this tool… :S

xjm’s picture

The handbook doc is way better than nothing, but it would also be helpful to explain how to handle the langcodes for field widgets in entity forms, where you are dealing with a FAPI array rather than the entity. (As I recall, field_language() also did not work properly when I was trying to alter certain fields on entity forms.)

Gábor Hojtsy’s picture

@Ryan Weal: are you working on this?

chx’s picture

I am asked to comment but I do not know what to comment. That's all nice and dandy so we now know that the possible language codes are returned by field_available_languages() good, so... if it returns more than one, which one I should use as a contrib author? What's the code snippet to replace uniform LANGUAGE_NONE usage?

Gábor Hojtsy’s picture

@chx: so in other words you are asking the following: "I have a piece of data; I asked the API and this type of data is stored on this entity for French, German and Spanish separately; Now which language should I save my data with?". Did I get your question right? Well, the answer to that is: you should know. It is pretty impossible to tell what language you data is in without you knowing it I think. It can very much depend on your data. Eg. you got that data from a 3rd party service to update the entity, you got that data from a user on a form, etc. You should consider the language information you got with your data (the language information from the 3rd party service, the language of the page the user used to enter the data, etc), and use that language. I don't see what kind of code snippet could we provide for deciding on that language, since it depends on your data.

Ryan Weal’s picture

StatusFileSize
new594 bytes
PASSED: [[SimpleTest]]: [MySQL] 36,280 pass(es).
[ View ]

Interesting developments on this thread. @Dries posted some revised text for the documentation we have in code on April 9. Looks to be mostly grammar cleanup and clarification (commit 47556803a8791f34dea3a0135c2885a70107eee2).

We also have two comments on our documentation node:

(A1). I believe the first comment probably addresses some concerns of @chx: http://drupal.org/node/1500308#comment-5794848 titled "never hardcode LANGUAGE_NONE".

(A2). @plach has also posted a couple links which were really helpful, the thread ends here: http://drupal.org/node/1495648 where we are still waiting on finalization of some helper functions.

It seems to me like the first comment (A1) is a good fix for the time being, while we are nearing a solution in the second comment (A2) that will potentially eliminate the need for the code example and warning not to use LANGUAGE_NONE in (A1).

With all of this in mind, here is a simple patch that just adds the reference to our documentation node while maintaining @Dries revisions to the in-code text. I think it will make sense to revisit the documentation once A2 is accepted. If it makes sense to put either or both of these into the in-line code now just let me know and I'll re-roll.

Ryan Weal’s picture

Status:Needs work» Needs review
Ryan Weal’s picture

StatusFileSize
new518 bytes
PASSED: [[SimpleTest]]: [MySQL] 36,273 pass(es).
[ View ]

Bad, bad, bad @Ryan Weal left a space at the end of a line!

Gábor Hojtsy’s picture

Well, Dries' comment was that he was looking for more of the online docs built into the text as it makes sense, no?

xjm’s picture

Thanks @Ryan Weal. Couple notes regarding the patch:

  • Any @see should go at the end of the docblock. Reference: http://drupal.org/node/1354#see
  • We could also use @link/@endlink to give the link a title and more context.
  • Adding a link is a good idea, but I think we should probably add more actual text from the handbook page to the API documentation as well. (Links to external resources should be used sparingly and should never be relied on as the sole source of information. The codebase should be self-contained without need for the internet.) I'm completely fine with committing a small addition like this first and then keeping this issue open to port the updated docs into the codebase, but a complete patch is easier to review. :)
jhodgdon’s picture

Status:Needs review» Needs work

Setting status per #65

Gábor Hojtsy’s picture

Issue tags:-sprint

Ryan: are you still working on this? I think its would be prudent to remove the sprint tag since there is no ongoing work on this :/ I'd like to use this tag for stuff that is actively worked on and people can help out / review.

plach’s picture

Added a note about the new entity_language() function in http://drupal.org/node/1500308.

Gábor Hojtsy’s picture

Issue tags:+sprint

Should be back on sprint at least for the week.

Ryan Weal’s picture

Hey folks, I'm still working on this. There is a lot of good stuff here, nice work everyone. I would like to demo some code to be sure I'm doing everything right but my D8 install was broken so I tried to run the installer and it did not work! I'll pull a fresh copy today and see if I can get this thing running again. If not, I will just provide example code and others can review it.

Gábor Hojtsy’s picture

Issue tags:-sprint

Ryan: still working on it? It's been a month :/

jvns’s picture

Assigned:Ryan Weal» Unassigned
Status:Needs work» Postponed

The field system is undergoing massive changes, so this needs to be postponed until http://drupal.org/node/1696640 is finished.

chx’s picture

Version:8.x-dev» 7.x-dev
Priority:Major» Critical
Status:Postponed» Needs review

Eh, who cares about D8 at this point, it's vastly different anyways but D7 still suffers.

jhodgdon’s picture

Status:Needs review» Needs work

Resetting status per #65, and besides it is a D8 patch.

David_Rothstein’s picture

Version:7.x-dev» 8.x-dev
Priority:Critical» Major

The D7 and D8 documentation in this section of the code look almost identical right now, so I'm pretty sure this needs to go in D8 first.

Also, "critical" doesn't seem like the right status for a documentation issue... though it would definitely be great to see this improved.

David_Rothstein’s picture

Issue summary:View changes

Updated issue summary.

jhedstrom’s picture

Version:8.0.x-dev» 7.x-dev
Issue summary:View changes

This is back to 7.x I think since #2067079: Remove the Field Language API.

chx’s picture

Status:Needs work» Closed (won't fix)

Originally I asked for rolling the multilingual fields back. Then it wouldn't have been too late. Now it is. Documenting at this point is pointless.

David_Rothstein’s picture

Status:Closed (won't fix)» Needs work
Related issues:+#1260640: Improve field language API DX

I think there is a real need to improve the documentation here, particularly what @xjm mentions in #17 (how to use language codes properly in different scenarios).

It might make sense to postpone that on #1260640: Improve field language API DX, which aims to add missing API functions for some of those scenarios.