One of the most useful and interesting features offered by the Vision API is Safe Search Detection (or Explicit Content Detection), which identifies and detects any explicit/adult content in an image.

This feature is particularly useful to keep a check on nudity or violence on the sites and avoid any such instances.

The Safe Search detection feature is implemented in such a way that it puts constraints on the image fields. If the user enables the Safe Search option for a particular image field, then the field is validated whenever an image file in uploaded in that field. If the file contains any explicit content which is not fit for display, the constraint message is displayed and the image would not be uploaded.

(This is one of my tasks of GSOC 2016 Project)

Members fund testing for the Drupal project. Drupal Association Learn more

Comments

ajalan065 created an issue. See original summary.

ajalan065’s picture

ajalan065’s picture

Status: Active » Needs review
FileSize
1.6 KB

Here is the patch for it.

penyaskito’s picture

Thanks!

I appreciate this, but would love to give some direction. In my mind, this should be a Constraint that we can add to any field, check https://www.drupal.org/node/2015723 for (not a lot of :-( ) documentation

We could then configure each field (third party settings configuration), if we want to perform this check or not. Could you expend some time exploring this possibility?

ajalan065’s picture

Sure!! I would work on following your directions :)

eugene.ilyin’s picture

We have two important things as a result of our discussion in hangouts:
1. We need to check type of entity and kind of file before than send this file to Cloud API
2. Would be nice be able enable this feature for specific field

ajalan065’s picture

The previous patch implemented safe search as a function in the *.module file, however, this patch implements it as constraints and validators as was suggested in the above comments.
Summary of the work which the patch does:
If any content types have any image field, then for each image field, it adds the option to enable Safe Search Detection in the Edit form of the Image field.
If the Safe Search is off, then any image can be uploaded irrespective of whether it contains any violence or explicit contents.
If the Safe Search is on, it validates whether it contains the explicit content or not. If it contains any such content, constraints are implemented and an error message is shown.

penyaskito’s picture

Thanks for working on this, looks much better now!
Some remarks:

  1. +++ b/src/Plugin/Validation/Constraint/SafeSearchConstraintValidator.php
    @@ -0,0 +1,51 @@
    +    foreach(\Drupal::service('entity_field.manager')->getFieldDefinitions('node', $data->getType()) as $field_name => $field_def) {
    

    The constraint shouldn't know about the entity field manager. Can we add the constraint to fields themselves instead of the entity? I think it's possible, but not 100% sure. This way we wouldn't depend on the field manager here.

  2. +++ b/src/Plugin/Validation/Constraint/SafeSearchConstraintValidator.php
    @@ -0,0 +1,51 @@
    +              $result = \Drupal::service('google_vision.api')->safeSearchDetection($filepath);
    

    This should be injected in the constructor.

  3. +++ b/src/Plugin/Validation/Constraint/SafeSearchConstraintValidator.php
    @@ -0,0 +1,51 @@
    +              if(!empty($result['responses'][0]['safeSearchAnnotation'])) {
    +                $adult = $result['responses'][0]['safeSearchAnnotation']['adult'];
    +                $likelihood = array('LIKELY', 'VERY_LIKELY');
    +                // if the image has explicit content.
    +                if(in_array($adult, $likelihood)) {
    

    Can we encapsulate this in the service? Something as \Drupal::service('google_vision.api')->hasExplicitContent($filepath)

penyaskito’s picture

Status: Needs review » Needs work

Back to needs work.

eugene.ilyin’s picture

Good job but with little remarks.
According to the Drupal Coding Standards you should use gap between "if" and condition and etc... https://www.drupal.org/coding-standards#controlstruct

ajalan065’s picture

@penyaskito, the 3rd point in your above suggestions, you suggested moving that piece of code to the services. I had done that initially. But then I was convinced to keep all the functions in the service as they were, and manipulate the responses as and where we apply them. Hence, I coded that bit in the validators, which I felt would look better.
But if you are still convinced that we should move it to services, do let me know and I will implement that way. :)

ajalan065’s picture

Status: Needs work » Needs review
FileSize
6.24 KB
3.48 KB

The constructor has been implemented in the ConstraintValidator file. Also, the spaces have been added after the if keyword.

However, the application of constraint directly over the fields is throwing "maximum function nested level reached" error.

ajalan065’s picture

As far as I have noticed, the implementation of constraints over fields require two things:
1. the field name (which depends on the user who creates the field). To fetch this, we need to get the fields of the node, which would require entity field manager.
2. The file uri (in order to call the api). This uri needs to be present in the ConstraintValidator file, as the call is being made over there. I am not able to come up presently with any other way to get the uri, other than the method I implemented above, and that again needs to use the entity file manager.
Is there any way to fetch the uri other than the method above?

Presently, these two points are appearing on applying the constraint to fields directly.

Please guide me if I am going wrong. :)

eugene.ilyin’s picture

However, the application of constraint directly over the fields is throwing "maximum function nested level reached" error.

Mhh, I've applied your patch and it works for me without fatal error. How can I reproduce it?

ajalan065’s picture

Here is the final patch, which applies the constraints to the fields successfully as was suggested in #8. The services have also been injected into the constructors.
The spaces are also taken care of as was suggested in #10.

ajalan065’s picture

Hi eugene.ilyin,
As I was getting the error, hence I did not upload the patch that reproduced the error. I was working on it, and after resolving the issue, I uploaded the fully functional patch. :) It works as expected

naveenvalecha’s picture

+++ b/src/Plugin/Validation/Constraint/SafeSearchConstraintValidator.php
@@ -0,0 +1,71 @@
+        if ($filepath = \Drupal::service('file_system')->realpath($file_uri)) {

inject the "file_system" service

ajalan065’s picture

file_system service injected.
Here is the patch.

ajalan065’s picture

Title: Integrate the Safe Search Detection feature for an image » Inplement the Safe Search Detection feature as a Constraint on images
Issue summary: View changes

Updated the issue summary.

ajalan065’s picture

Title: Inplement the Safe Search Detection feature as a Constraint on images » Implement the Safe Search Detection feature of the Vision API as a Constraint on images
naveenvalecha’s picture

looks good. can we add tests here for testing this validation

ajalan065’s picture

FileSize
20.18 KB
14.68 KB

Here is the patch with two test files attached.
One is web tests which creates a custom content type along with an image field with constraint applied.
The only task remaining is to validate the constraint, where I am currently stuck.

Another is browser web test. The problem persisting here are the assertions. Each and every method,be it assertText() or drupalPostForm() requires custom coding, and can not be used directly.

ajalan065’s picture

The progress of the web tests.
As mentioned in the above comments, the validate part is still to be implemented.

I am unable to get the answer to the question that the validate() method is a part of SafeSearchConstraintValidator.php which is called from the module when a particular bundle field(in this case image) is filled.
So, would this method be accessible inside the web test environment?
How can I make use of the validate() inside my webtest?

ajalan065’s picture

Here is the patch with web tests working as per expected.
The Browser tests were removed after discussion with naveenvalecha on grounds that most of the assertions and methods required are not presently available in the BrowserTestBase, and writing custom codes for most of the assertions including assertResponse() and DrupalPostForm(), etc would not be feasible for simple browser tests. And the browser tests have been postponed to the future time when the assertions are committed to Core.

ajalan065’s picture

In this patch, I have updated the schema.yml of the module and have also altered names to 'safe_search' at few places.
This patch is fully ready for review from my side :)

penyaskito’s picture

Status: Needs review » Needs work
  1. +++ b/src/Tests/SafeSearchConstraintValidationTest.php
    @@ -0,0 +1,213 @@
    +    \Drupal::configFactory()->getEditable('google_vision.settings')->set('api_key', 'AIzaSyAtuc_LOB70imSYsT0TXFUNnNlMqiDoyP8')->save();
    ...
    +    $this->assertFieldByName('api_key', 'AIzaSyAtuc_LOB70imSYsT0TXFUNnNlMqiDoyP8', 'The key has been saved');
    

    Is this your key? No way you want to share that.

    We should mock the google service, we don't need to rely on it for running tests.

  2. +++ b/src/Tests/SafeSearchConstraintValidationTest.php
    @@ -0,0 +1,213 @@
    +    $image = file_get_contents('http://www.menshealth.com/sites/menshealth.com/files/articles/2015/12/man-snoring.jpg'); // string
    +    $file = file_save_data($image, 'public://explicit.jpg', FILE_EXISTS_REPLACE);
    

    If we mock the service as said, we don't rely on having external images referenced in tests. Also, we don't want to ship with violent or nudity content for testing.

ajalan065’s picture

We should mock the google service,

I could not get what you meant by this?

ajalan065’s picture

As far as I had observed, mocking is only performed in unit and kernel tests.
I could not find any instance of mocking in webtests.

Please guide me on how to advance on it for webtests.

ajalan065’s picture

The test module does not work and throws "SessionTestTrait not found " error.
Please have a look at it. I am unable to resolve where is the module going wrong?

ajalan065’s picture

Status: Needs work » Needs review
FileSize
17.63 KB
7.43 KB

Here is the patch which mocks the service, hence we neither relies on a google browser key for tests, nor on the external images for reference.
Here is the description of what I have done.
The google_vision.api service is mocked, and I have overridden the safeSearchDetection(). As there is no other way to detect whether the image has any explicit content without the use of API key, and the sole purpose of the test is to check whether the constraint applies correctly or not, I have passed the response which would be sent by the API if the image contains explicit content, taking the simpletest generated image.
And just to maintain the rhythm of the original function, I have just initialized the api_key with a test value, though it does not have any other work, except that the safe search would not be done if the key is not set.

  • ajalan065 authored 2c544a3 on 8.x-1.x
    Issue #2744845 by ajalan065, penyaskito, eugene.ilyin, naveenvalecha:...
eugene.ilyin’s picture

Committed.
This patch is already pretty big. If something is not in the best way, let's correct it later.

I have a question. Why do we need this?

+function google_vision_entity_bundle_field_info_alter(&$fields, EntityTypeInterface $entity_type, $bundle) {
+  if ($entity_type->id() == 'node') {

Images can be not only in nodes.

naveenvalecha’s picture

Status: Needs review » Fixed
ajalan065’s picture

Assigned: ajalan065 » Unassigned

I will do the follow up of this issue in the new issue created by naveenvalecha.
Unassigning it.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.