Problem/Motivation

Requesting support for text-to-image with Google Vertex AI. I see it mentioned as a to-do on this closed issue https://www.drupal.org/project/ai_provider_google_vertex/issues/3504260 but it wasn't clear if or when it might happen. According to their documentation, Vertex AI does provide text-to-image capabilities https://cloud.google.com/vertex-ai/generative-ai/docs/image/overview and the feature works using the Cloud Console. Within Drupal I don't see a way to add or use it, and text-to-image is not listed as supported for Vertex AI on the matrix here: https://project.pages.drupalcode.org/ai/1.1.x/providers/matris/

Steps to reproduce

Create Google Cloud project & enable Google Vertex AI
Install modules including: ai, ai_provider_google_vertex & ai_media_image
Confirm AI connection (add & test a Chat model, for example)
No way to add an Imagen (or other text-to-image) model on this page /admin/config/ai/providers/google_vertex
No providers available in the 'text to image' drop-down at /admin/config/ai/settings
No providers available in the drop-down at /media/add/image

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Issue fork ai_provider_google_vertex-3537441

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

en-cc-org created an issue. See original summary.

en-cc-org’s picture

en-cc-org commented
Issue summary: View changes

yce made their first commit to this issue’s fork.

yce opened merge request !12

yce’s picture

yce commented
Status: Active » Needs review

Hi,

I've created an MR for the text-to-image support.
I did a quick test with the ai_media_image module as well, that also seems to be working.

en-cc-org’s picture

en-cc-org commented

Thank you for the quick response! Note to others: a side effect is that uploading images via /media/add/image will fail. The workaround is to disable/uninstall ai_media_image until it becomes supported by ai_provider_google_vertex
Thanks again

mrdalesmith’s picture

mrdalesmith commented
Status: Needs review » Needs work

I added the patch to a test site using this module, but I got some issues.

Visit form at /admin/config/ai/providers/text_to_image/google_vertex?destination=/admin/config/ai/providers/google_vertex

Warning: Undefined array key "advanced_settings" in Drupal\ai_provider_google_vertex\Element\TextToImageFormElement->processElement() (line 93 of modules/contrib/ai_provider_google_vertex/src/Element/TextToImageFormElement.php).

The Model ID reports that it is required but description indicates it should be possible to leave it empty.

Using the AI Explorer to generate an image gives the error: [error removed]

Revisiting this using settings suggested at https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/im... and I no longer get the PHP notice error. The Explorer works without issue to generate the images using the imagen-4.0-ultra-generate-001 model. I can't help thinking that since there are specific right values you can enter into the model field, this would be better as an options list so it's clear what you should add here. The link to Google's help pages is useful but has a lot of detail not relevant for this specific use case: the UX of the settings form isn't great without this, but it does work.

mrdalesmith’s picture

mrdalesmith commented
Status: Needs work » Needs review
mxr576’s picture

mxr576
Primary language Hungarian
Location Hungary
commented
Version: 1.1.0 » 2.x-dev
Issue tags: +Needs backport