The Google Cloud Vision API offers following features- Landmark Detection, Logo Detection and Optical Character Recognition, in addition to the Label Detection feature.
These can be useful to fill the alt text/title text by default.

These features would be used to fill the alt text of the image file based on the choice given by the end user.
The user can check the checkboxes present in the Edit form of the Image File entity, and on its basis, the alt text would be appropriately filled.

Remaining Tasks:
Create a patch which would fill the alt text of the image file entity.

(This is one of the tasks of my GSOC 2016 Project.)

Comments

ajalan065 created an issue. See original summary.

ajalan065’s picture

Status: Active » Needs review
FileSize
1.69 KB

Here is the first patch for the landmark detection, and storage of the result in the Alt Text of the image file.
As the use of services was already committed to the module, the patch is built over that.

ajalan065’s picture

The previous patch had some extra parameters, which have been removed in this patch

eugene.ilyin’s picture

Status: Needs review » Needs work

I think that approach of the label detection is better for alt attribute. Because not all the images have landmarks, but I think that all images have the label.
Using of the Landmark detection can be useful, but I don't think that makes sense to fill alt by landmark by default.
Can we have the feature to fill alt by landmark but make it configurable?

What do you think about it?

ajalan065’s picture

We may have labels filled in the alt attribute. But we already have those data in the taxonomy terms.
IMHO, it would be better if we use the alt attribute to show the data from Landmark/Logo/Text within the images.

eugene.ilyin’s picture

Taxonomy terms as tags and filling the alt attribute are two different features. Please do not confuse it.
For me as an end user would be nice to be able enable checkbox "Fill alt/title by Google Vision API" and select method: Label Detection or Landmark or etc...

ajalan065’s picture

Issue summary: View changes
ajalan065’s picture

Title: Integrate the Landmark Detection feature of the Vision API » Fill the Alt Text of the Image File using Google Vision API
Issue summary: View changes
ajalan065’s picture

Status: Needs work » Needs review
FileSize
4.71 KB

Here is the patch which provides options for the image file to set the alt text for the file by one of the four methods- Label, Landmark, Logo and Optical Texts.

naveenvalecha’s picture

Status: Needs review » Needs work
  1. +++ b/google_vision.module
    @@ -159,3 +191,66 @@ function google_vision_file_entity_add_labels($file, $field, $vid) {
    +          // If we have retrieved labels.
    

    if there will be no response then $output would never set.

  2. +++ b/google_vision.module
    @@ -159,3 +191,66 @@ function google_vision_file_entity_add_labels($file, $field, $vid) {
    +    $entity->set('field_image_alt_text', $output);
    

    Add a check if $output is available then set the value. See the review above in case 'labels'

Rest looks good to me.We need test coverage for this as well.

ajalan065’s picture

Status: Needs work » Needs review
FileSize
13.71 KB
9.71 KB

Here is the patch following the suggestions of #10 along with the tests.

Please review :)

eugene.ilyin’s picture

Could you explain what is "field_image_alt_text" and why is it hardcoded?

ajalan065’s picture

Alt text field remains already present on setting the drupal environment in the simple webtests, with the name "field_image_alt_text".
Hence, I directly used that value.

  • eugene.ilyin committed 0471004 on 8.x-1.x
    Issue #2743815 by ajalan065, eugene.ilyin: Fill the Alt Text of the...
  • eugene.ilyin committed a5b2b37 on 8.x-1.x authored by ajalan065
    Issue #2743815 by ajalan065, eugene.ilyin: Fill the Alt Text of the...
eugene.ilyin’s picture

Status: Needs review » Needs work

Committed.

I have several remarks:
1. Please check my additional commit. If you don't agree with something, please let me know.
2. Why do you use name "features" for this functionality? This name looks pretty abstract.
3. If the user will select any option for "features" he will be not able to disable it. Please add the option "none".
4. I think that better to fill "alt" field only if it's empty. Better to give the user ability to overwrite the alt for certain image manually.

ajalan065’s picture

Hi eugene.ilyin

#1- I will check the commit, and would inform you by tomorrow.
#2- I had derived the name "features" from the functionality of the Cloud Vision API.
#3- I will add the option "none" and post the new patch by tomorrow
#4- The alt field is filled as soon as the file is uploaded. At that moment, the field remains always empty. However, the user may override the value if he feels.

eugene.ilyin’s picture

#2- I had derived the name "features" from the functionality of the Cloud Vision API.

I propose to change it. Features is too abstract name. What about "alt_auto_filling" or something like this?

#4- The alt field is filled as soon as the file is uploaded. At that moment, the field remains always empty. However, the user may override the value if he feels.

No in "google_vision_entity_presave" you always override alt on value from Google Vision. There is no way to set custom alt.

naveenvalecha’s picture

I propose to change it. Features is too abstract name. What about "alt_auto_filling" or something like this?

+1 to this.

ajalan065’s picture

Status: Needs work » Needs review
FileSize
7.38 KB

Here is the patch which allows the edit of alt text after it is filled by the vision api.
Also, the "features" is replaced.

Please review :)

naveenvalecha’s picture

+++ b/google_vision.module
@@ -241,14 +250,25 @@ function google_vision_set_alt_text($entity, $field) {
+        case 'none':
+          // If none is selected, do nothing.
+          break;

Why this case ?
can't we use the default value if there's not any case ?

ajalan065’s picture

Hi naveenvalecha, the case "none" was added so that the user may switch to option "none" if he does not want to fill the alt text.
As, I had added the option "None", so instead of using the default case, I thought better to use that specific option.

naveenvalecha’s picture

Status: Needs review » Reviewed & tested by the community
+++ b/google_vision.module
@@ -241,14 +250,25 @@ function google_vision_set_alt_text($entity, $field) {
+ * Edit the current value of the Alt Text field of the image file.

Add a @TODO here to move this to the service later if there will be more helper functions like this.

Rest looks good to go.Thanks!

ajalan065’s picture

Added the @todo as suggested.

naveenvalecha’s picture

Assigned: ajalan065 » eugene.ilyin

Ignore the #23 patch and interdiff.
i have filed a followup issue for it https://www.drupal.org/node/2776441

Still RTBC for #19
Assigning to eugene for final look into this.

naveenvalecha’s picture

Status: Reviewed & tested by the community » Needs review
naveenvalecha’s picture

Assigned: eugene.ilyin » ajalan065
Status: Needs review » Needs work
Issue tags: +Needs reroll

Patch in #19 no longer applies to head. Please reoll it.

ajalan065’s picture

Assigned: ajalan065 » naveenvalecha
Status: Needs work » Needs review
Issue tags: -Needs reroll
FileSize
7.43 KB

Here is the rerolled patch.

naveenvalecha’s picture

naveenvalecha’s picture

Status: Needs review » Reviewed & tested by the community
+++ b/google_vision.module
@@ -353,7 +366,15 @@ function google_vision_set_alt_text($entity, $field) {
+
+/**
+ * Edit the current value of the Alt Text field of the image file.
+ */
+function google_vision_edit_alt_text($entity, $field, $value) {
+  //Set the new value to the Alt Text field.
+  $entity->set($field->getName(), $value);
+}

Add a followup to move this function to a helper service.

naveenvalecha’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed to 8.x-1.x
Thanks!

ajalan065’s picture

Ya, I am working on moving the functions to the helper service, and would post the patch in the follow up issue.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.