[Tracker]
Update Summary: [One-line status update for stakeholders]
Check-in Date: MM/DD/YYYY
Additional Collaborators:
Metadata is used by the AI Tracker. Docs and additional fields here.
[/Tracker]
Gemini is one of the only providers that has file embeddings, outside of AWS Titan. This means that you can index and search using an image and image likeness.
Currently the embedding solution only allows for text, we should add at least images here (and think about other multimodal in the AI module)
Add the image input to embeddings.
[ ] AI Assisted Issue
This issue was generated with AI assistance, but was reviewed and refined by the creator.
[ ] AI Assisted Code
This code was mainly generated by a human, with AI autocompleting or parts AI generated, but under full human supervision.
[ ] AI Generated Code
This code was mainly generated by an AI with human guidance, and reviewed, tested, and refined by a human.
[ ] Vibe Coded
This code was generated by an AI and has only been functionally tested.
Show commandsStart within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
changes
Comments
Comment #2
a.dmitriiev commentedComment #4
a.dmitriiev commentedEmbeddings model 001 supports also text + image, but doesn't support image on its own. Embeddings model 2 supports all combinations: text, image and text + image. I have also added the unit tests for this.
Comment #5
jucs7 commentedComment #6
jucs7 commentedI tested the MR locally using the API Explorer module (Branch from #3586422) against both models. With gemini-embedding-2 the embeddings behave correctly across all three combinations (text, image, text+image) — different inputs produce different vectors as expected.
However, with embedding-001, a text-only request and an image+same text request produce identical embedding vectors, which indicates the image input is not affecting the output.
Comment #7
jucs7 commented