One of the most useful and interesting features offered by the Vision API is Safe Search Detection (or Explicit Content Detection), which identifies and detects any explicit/adult content in an image.

This feature is particularly useful to keep a check on nudity or violence on the sites and avoid any such instances.

The Safe Search detection feature is implemented in such a way that it puts constraints on the image fields. If the user enables the Safe Search option for a particular image field, then the field is validated whenever an image file in uploaded in that field. If the file contains any explicit content which is not fit for display, the constraint message is displayed and the image would not be uploaded.

(This is one of my tasks of GSOC 2016 Project)

CommentFileSizeAuthor
#30 interdiff-29-30.txt7.43 KBajalan065
#30 Service-mocked-2744845-8-30.patch17.63 KBajalan065
#29 test-module-not-working.patch17.66 KBajalan065
#25 interdiff-24-25.txt3.16 KBajalan065
#25 Updated-schema-file-2744845-8-25.patch14.46 KBajalan065
#24 interdiff-23-24.txt5.88 KBajalan065
#24 Webtests-working-2744845-8-24.patch14.15 KBajalan065
#23 interdiff-22-23.txt13.75 KBajalan065
#23 Browser-test-removed-2744845-8-23.patch12.72 KBajalan065
#22 interdiff-18-22.txt14.68 KBajalan065
#22 Tests-2744845-22.patch20.18 KBajalan065
#18 interdiff-15-18.txt2.74 KBajalan065
#18 file-system-service-injected-2744845-8-18.patch6.42 KBajalan065
#15 interdiff-12-14.txt5.11 KBajalan065
#15 Constraints-added-to-fields-2744845-8-14.patch6.05 KBajalan065
#12 interdiff-7-12.txt3.48 KBajalan065
#12 Constructor-injected-2744845-8-12.patch6.24 KBajalan065
#7 interdiff-2744845-3-7.txt6.96 KBajalan065
#7 Contraint-Validator-implemented-2744845-8-7.patch5.49 KBajalan065
#3 safe-search-detection-2744845-8-3.patch1.6 KBajalan065
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

ajalan065 created an issue. See original summary.

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented
ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented
Status: Active » Needs review
FileSize
safe-search-detection-2744845-8-3.patch1.6 KB

Here is the patch for it.

penyaskito’s picture

penyaskito
he/him
Primary language Spanish
Location 🧑🏽‍🌾Seville 💃, Andalusia 🇪🇸, UTC+2 🇪🇺
CreditAttribution: penyaskito at Lingotek commented

Thanks!

I appreciate this, but would love to give some direction. In my mind, this should be a Constraint that we can add to any field, check https://www.drupal.org/node/2015723 for (not a lot of :-( ) documentation

We could then configure each field (third party settings configuration), if we want to perform this check or not. Could you expend some time exploring this possibility?

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented

Sure!! I would work on following your directions :)

eugene.ilyin’s picture

eugene.ilyin CreditAttribution: eugene.ilyin as a volunteer and at DrupalJedi commented

We have two important things as a result of our discussion in hangouts:
1. We need to check type of entity and kind of file before than send this file to Cloud API
2. Would be nice be able enable this feature for specific field

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented
FileSize
Contraint-Validator-implemented-2744845-8-7.patch5.49 KB
interdiff-2744845-3-7.txt6.96 KB

The previous patch implemented safe search as a function in the *.module file, however, this patch implements it as constraints and validators as was suggested in the above comments.
Summary of the work which the patch does:
If any content types have any image field, then for each image field, it adds the option to enable Safe Search Detection in the Edit form of the Image field.
If the Safe Search is off, then any image can be uploaded irrespective of whether it contains any violence or explicit contents.
If the Safe Search is on, it validates whether it contains the explicit content or not. If it contains any such content, constraints are implemented and an error message is shown.

penyaskito’s picture

penyaskito
he/him
Primary language Spanish
Location 🧑🏽‍🌾Seville 💃, Andalusia 🇪🇸, UTC+2 🇪🇺
CreditAttribution: penyaskito at Lingotek commented

Thanks for working on this, looks much better now!
Some remarks:

  1. +++ b/src/Plugin/Validation/Constraint/SafeSearchConstraintValidator.php
@@ -0,0 +1,51 @@
+    foreach(\Drupal::service('entity_field.manager')->getFieldDefinitions('node', $data->getType()) as $field_name => $field_def) {

    The constraint shouldn't know about the entity field manager. Can we add the constraint to fields themselves instead of the entity? I think it's possible, but not 100% sure. This way we wouldn't depend on the field manager here. 

  2. +++ b/src/Plugin/Validation/Constraint/SafeSearchConstraintValidator.php
@@ -0,0 +1,51 @@
+              $result = \Drupal::service('google_vision.api')->safeSearchDetection($filepath);

    This should be injected in the constructor. 

  3. +++ b/src/Plugin/Validation/Constraint/SafeSearchConstraintValidator.php
@@ -0,0 +1,51 @@
+              if(!empty($result['responses'][0]['safeSearchAnnotation'])) {
+                $adult = $result['responses'][0]['safeSearchAnnotation']['adult'];
+                $likelihood = array('LIKELY', 'VERY_LIKELY');
+                // if the image has explicit content.
+                if(in_array($adult, $likelihood)) {

    Can we encapsulate this in the service? Something as \Drupal::service('google_vision.api')->hasExplicitContent($filepath)

penyaskito’s picture

penyaskito
he/him
Primary language Spanish
Location 🧑🏽‍🌾Seville 💃, Andalusia 🇪🇸, UTC+2 🇪🇺
CreditAttribution: penyaskito at Lingotek commented
Status: Needs review » Needs work

Back to needs work.

eugene.ilyin’s picture

eugene.ilyin CreditAttribution: eugene.ilyin as a volunteer and at DrupalJedi commented

Good job but with little remarks.
According to the Drupal Coding Standards you should use gap between "if" and condition and etc... https://www.drupal.org/coding-standards#controlstruct

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented

@penyaskito, the 3rd point in your above suggestions, you suggested moving that piece of code to the services. I had done that initially. But then I was convinced to keep all the functions in the service as they were, and manipulate the responses as and where we apply them. Hence, I coded that bit in the validators, which I felt would look better.
But if you are still convinced that we should move it to services, do let me know and I will implement that way. :)

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented
Status: Needs work » Needs review
FileSize
Constructor-injected-2744845-8-12.patch6.24 KB
interdiff-7-12.txt3.48 KB

The constructor has been implemented in the ConstraintValidator file. Also, the spaces have been added after the if keyword.

However, the application of constraint directly over the fields is throwing "maximum function nested level reached" error.

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented

As far as I have noticed, the implementation of constraints over fields require two things:
1. the field name (which depends on the user who creates the field). To fetch this, we need to get the fields of the node, which would require entity field manager.
2. The file uri (in order to call the api). This uri needs to be present in the ConstraintValidator file, as the call is being made over there. I am not able to come up presently with any other way to get the uri, other than the method I implemented above, and that again needs to use the entity file manager.
Is there any way to fetch the uri other than the method above?

Presently, these two points are appearing on applying the constraint to fields directly.

Please guide me if I am going wrong. :)

eugene.ilyin’s picture

eugene.ilyin CreditAttribution: eugene.ilyin as a volunteer and at DrupalJedi commented

However, the application of constraint directly over the fields is throwing "maximum function nested level reached" error.

Mhh, I've applied your patch and it works for me without fatal error. How can I reproduce it?

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented
FileSize
Constraints-added-to-fields-2744845-8-14.patch6.05 KB
interdiff-12-14.txt5.11 KB

Here is the final patch, which applies the constraints to the fields successfully as was suggested in #8. The services have also been injected into the constructors.
The spaces are also taken care of as was suggested in #10.

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented

Hi eugene.ilyin,
As I was getting the error, hence I did not upload the patch that reproduced the error. I was working on it, and after resolving the issue, I uploaded the fully functional patch. :) It works as expected

naveenvalecha’s picture

naveenvalecha
He/Him
Primary language Hindi
CreditAttribution: naveenvalecha as a volunteer and at Acquia for Acquia commented 
+++ b/src/Plugin/Validation/Constraint/SafeSearchConstraintValidator.php
@@ -0,0 +1,71 @@
+        if ($filepath = \Drupal::service('file_system')->realpath($file_uri)) {

inject the "file_system" service

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented
FileSize
file-system-service-injected-2744845-8-18.patch6.42 KB
interdiff-15-18.txt2.74 KB

file_system service injected.
Here is the patch.

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented
Title: Integrate the Safe Search Detection feature for an image » Inplement the Safe Search Detection feature as a Constraint on images
Issue summary: View changes

Updated the issue summary.

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented
Title: Inplement the Safe Search Detection feature as a Constraint on images » Implement the Safe Search Detection feature of the Vision API as a Constraint on images
naveenvalecha’s picture

naveenvalecha
He/Him
Primary language Hindi
CreditAttribution: naveenvalecha as a volunteer and at Acquia for Acquia commented

looks good. can we add tests here for testing this validation

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented
FileSize
Tests-2744845-22.patch20.18 KB
interdiff-18-22.txt14.68 KB

Here is the patch with two test files attached.
One is web tests which creates a custom content type along with an image field with constraint applied.
The only task remaining is to validate the constraint, where I am currently stuck.

Another is browser web test. The problem persisting here are the assertions. Each and every method,be it assertText() or drupalPostForm() requires custom coding, and can not be used directly.

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented
FileSize
Browser-test-removed-2744845-8-23.patch12.72 KB
interdiff-22-23.txt13.75 KB

The progress of the web tests.
As mentioned in the above comments, the validate part is still to be implemented.

I am unable to get the answer to the question that the validate() method is a part of SafeSearchConstraintValidator.php which is called from the module when a particular bundle field(in this case image) is filled.
So, would this method be accessible inside the web test environment?
How can I make use of the validate() inside my webtest?

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented
FileSize
Webtests-working-2744845-8-24.patch14.15 KB
interdiff-23-24.txt5.88 KB

Here is the patch with web tests working as per expected.
The Browser tests were removed after discussion with naveenvalecha on grounds that most of the assertions and methods required are not presently available in the BrowserTestBase, and writing custom codes for most of the assertions including assertResponse() and DrupalPostForm(), etc would not be feasible for simple browser tests. And the browser tests have been postponed to the future time when the assertions are committed to Core.

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented
FileSize
Updated-schema-file-2744845-8-25.patch14.46 KB
interdiff-24-25.txt3.16 KB

In this patch, I have updated the schema.yml of the module and have also altered names to 'safe_search' at few places.
This patch is fully ready for review from my side :)

penyaskito’s picture

penyaskito
he/him
Primary language Spanish
Location 🧑🏽‍🌾Seville 💃, Andalusia 🇪🇸, UTC+2 🇪🇺
CreditAttribution: penyaskito at Lingotek commented
Status: Needs review » Needs work 
  1. +++ b/src/Tests/SafeSearchConstraintValidationTest.php
@@ -0,0 +1,213 @@
+    \Drupal::configFactory()->getEditable('google_vision.settings')->set('api_key', 'AIzaSyAtuc_LOB70imSYsT0TXFUNnNlMqiDoyP8')->save();
...
+    $this->assertFieldByName('api_key', 'AIzaSyAtuc_LOB70imSYsT0TXFUNnNlMqiDoyP8', 'The key has been saved');

    Is this your key? No way you want to share that.

    We should mock the google service, we don't need to rely on it for running tests. 

  2. +++ b/src/Tests/SafeSearchConstraintValidationTest.php
@@ -0,0 +1,213 @@
+    $image = file_get_contents('http://www.menshealth.com/sites/menshealth.com/files/articles/2015/12/man-snoring.jpg'); // string
+    $file = file_save_data($image, 'public://explicit.jpg', FILE_EXISTS_REPLACE);

    If we mock the service as said, we don't rely on having external images referenced in tests. Also, we don't want to ship with violent or nudity content for testing.

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented

We should mock the google service,

I could not get what you meant by this?

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented

As far as I had observed, mocking is only performed in unit and kernel tests.
I could not find any instance of mocking in webtests.

Please guide me on how to advance on it for webtests.

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented
FileSize
test-module-not-working.patch17.66 KB

The test module does not work and throws "SessionTestTrait not found " error.
Please have a look at it. I am unable to resolve where is the module going wrong?

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented
Status: Needs work » Needs review
FileSize
Service-mocked-2744845-8-30.patch17.63 KB
interdiff-29-30.txt7.43 KB

Here is the patch which mocks the service, hence we neither relies on a google browser key for tests, nor on the external images for reference.
Here is the description of what I have done.
The google_vision.api service is mocked, and I have overridden the safeSearchDetection(). As there is no other way to detect whether the image has any explicit content without the use of API key, and the sole purpose of the test is to check whether the constraint applies correctly or not, I have passed the response which would be sent by the API if the image contains explicit content, taking the simpletest generated image.
And just to maintain the rhythm of the original function, I have just initialized the api_key with a test value, though it does not have any other work, except that the safe search would not be done if the key is not set.

  • ajalan065 authored 2c544a3 on 8.x-1.x 
    Issue #2744845 by ajalan065, penyaskito, eugene.ilyin, naveenvalecha:...
eugene.ilyin’s picture

eugene.ilyin CreditAttribution: eugene.ilyin as a volunteer and at DrupalJedi commented

Committed.
This patch is already pretty big. If something is not in the best way, let's correct it later.

I have a question. Why do we need this?

+function google_vision_entity_bundle_field_info_alter(&$fields, EntityTypeInterface $entity_type, $bundle) {
+  if ($entity_type->id() == 'node') {

Images can be not only in nodes.

naveenvalecha’s picture

naveenvalecha
He/Him
Primary language Hindi
CreditAttribution: naveenvalecha as a volunteer and for Acquia commented
Status: Needs review » Fixed

Filed a followup for this. #2756429: Remove the restriction of Label Detection/safe search detection feature from node entity
Fixed this issue.

ajalan065’s picture

ajalan065 CreditAttribution: ajalan065 at Google Summer of Code commented
Assigned: ajalan065 » Unassigned

I will do the follow up of this issue in the new issue created by naveenvalecha.
Unassigning it.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.