Problem/Motivation

In Drupal 1.3.0 we have object detection: #3570609: Add support for Object Detection operation

This will give back a list of common object names and how probable they are. The difference compared to vision is that it answer in ms instead of seconds and can be run on webserver hardware or for free via Huggingface.

This means that we could add a valdidation that checks that a common object name appears and either refuses or refuses when its missing.

Proposed resolution

  • Create a new Validation type that works with object detection.
  • Look for keyword(s).
  • Add an AND or OR filter.
  • Have a rule that either dissaproves or approves if it appears.

Remaining tasks

User interface changes

API changes

Data model changes

CommentFileSizeAuthor
#4 3575363-object-detection-validation.png160.65 KBabhisekmazumdar

Issue fork ai_validations-3575363

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

marcus_johansson created an issue. See original summary.

abhisekmazumdar’s picture

abhisekmazumdar
he/him
Primary language English
Location India
commented
Version: 1.0.x-dev » 1.3.x-dev
Assigned: Unassigned » abhisekmazumdar

abhisekmazumdar opened merge request !1

abhisekmazumdar’s picture

abhisekmazumdar
he/him
Primary language English
Location India
commented
Assigned: abhisekmazumdar » Unassigned
Status: Active » Needs review
StatusFileSize
new 3575363-object-detection-validation.png160.65 KB

How to test:

This MR depends on the following issues(few of them are not part of any stable releases, so make sure to pull them)

Setup

  • Content type: Article
  • Field: Image
  • Rule set: Default shortcuts (Field validation rule set)
  • Rule: Image Object check (AI object detection constraint)

Rule configuration

  1. Object Detection Model: Huggingface – detr-resnet-50
  2. Keywords: e.g. person, bagpack, apple (one per line or comma-separated)
  3. Finder: Exact
  4. Minimum Confidence: 0.8
  5. Keyword Filter: Any keyword (OR)
  6. Rule Mode: Disapprove when object(s) found
  7. If model is not available: Skip validation
  8. Message: e.g. This value is not valid (Object found)

Example screenshot

What was tested

  • Rule appears and saves correctly in the Field validation rule set.
  • Validation runs on the Article image field (approve/disapprove and keyword behaviour as configured).
  • Model dropdown lists Huggingface object-detection model(s) when the AI module and provider support object detection.

[ x ] AI Generated Code
This code was mainly generated by an AI with human guidance, and reviewed, tested, and refined by a human.

arianraeesi’s picture

arianraeesi commented
Issue tags: +AI Initiative Sprint, +AI Product Development
marcus_johansson’s picture

marcus_johansson commented
Assigned: Unassigned » marcus_johansson
marcus_johansson’s picture

marcus_johansson commented
Status: Needs review » Needs work

The composer needs to point to AI 1.3.0 and one nit and one fix neded.

marcus_johansson’s picture

marcus_johansson commented
Assigned: marcus_johansson » Unassigned
abhisekmazumdar’s picture

abhisekmazumdar
he/him
Primary language English
Location India
commented
Status: Needs work » Needs review
marcus_johansson’s picture

marcus_johansson commented
Status: Needs review » Reviewed & tested by the community

It works well and looks good now. However there are still some phpstan issue - I'm guessing you have to add AI 1.3.x-dev to the composer.json

abhisekmazumdar’s picture

abhisekmazumdar
he/him
Primary language English
Location India
commented

Oh yes, I missed updating the AI dependency. I just added a new commit for it.

  • abhisekmazumdar committed a8ed2339 on 1.3.x 
    feat: #3575363 Create object detection validation

By: marcus_johansson...
abhisekmazumdar’s picture

abhisekmazumdar
he/him
Primary language English
Location India
commented
Status: Reviewed & tested by the community » Fixed

Thank You for all the work here. This is merged into 1.3.x

Also created: https://www.drupal.org/project/ai_validations/issues/3576079

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

arianraeesi’s picture

arianraeesi commented
Issue tags: -AI Product Development

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.