Problem/Motivation

When trying to change AI assistant provider from the "AI provider" dropdown, it is not possible
IF the current provider is not available.

Steps to reproduce (required for bugs, but not feature requests)

Please provide information like AI modules enabled, which AI provider, browser, etc.

Enable ollama for example and vllm.
create assistant and select your local ollama model there in the AI provider.
Now make the local ollama unavailable for Drupal, (stop the ollama for example)

Go to assistant:
/admin/config/ai/ai-assistant/ai_test_chatbot

Try to change the current provider from the Ai Provider dropdown.
Result: IT does not let you change cos the current (in this case Ollama) is not available for drupal.

it just gives error and the model dropdown for the new model is empty, so you cant choose any.

Failed to get models from Ollama: Server error: `GET http://ai.test.net/api/tags` resulted in a `503 Service Unavailable` response: <html><body><h1>503 Service Unavailable</h1> No server is available to handle this request. </body></html>
1 error has been found:

Proposed resolution

Remaining tasks

Optional: Other details as applicable (e.g., User interface changes, API changes, Data model changes)

Issue fork ai-3541483

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

anaconda777 created an issue. See original summary.

littlepixiez’s picture

Assigned: Unassigned » littlepixiez

littlepixiez’s picture

StatusFileSize
new69.34 KB
new26.33 KB
new34.87 KB
new29.28 KB
new57.05 KB
new40.61 KB

I came across this issue when testing the AI dev recipe which defaults to use "openai" even if you don't have a valid API key, which was causing me issues updating the form.

There seem to be a couple of scenarios you are unable to update the provider:

Scenario 1. The provider stored on the AI assistant is not configured properly
1. Create an AI assistant and choose a provider that is configured correctly
2. Break the provider configuration (ie, change the port to make the models unavailable)
3. Edit the AI assistant
Expected: Able to update AI assistant to a different provider
Actual: Unable to update AI assistant - models is empty and required with "Ollama" remaining selected
Ollama API error
Ollama models missing (required)

Scenario 2. The provider/model stored on the AI assistant has been removed
1. Create an AI assistant and choose a provider that is configured correctly
2. Remove the provider/break the provider by removing the API key
3. Edit the AI assistant
Expected: Able to update AI assistant to a different provider
Actual: Unable to update AI assistant - providers is empty and models won't load
Providers dropdown is empty
Models form cannot load

My proposal would be to add some conditional checks/validation and add some helpful error messages that allow changing provider.

If the assistant is using a provider that is no longer available:
Invalid provider message

If no models are available, the models form does not load and gives an error:
Invalid models error

I need to test some more scenarios, and ensure all areas are covered. This caught me off-guard and prevented me testing the assistant functionality so thanks for raising this, along with my issue with "openai" not being available hopefully this UX fix will work.

My main concern is that the form helper where we are giving the models warning is used in many places, but that may be useful for other parts of this module's ecosystem too.

littlepixiez’s picture

Assigned: littlepixiez » Unassigned
Status: Active » Needs review

marcus_johansson made their first commit to this issue’s fork.

marcus_johansson’s picture

Version: 1.1.2 » 1.2.x-dev
Component: ...to be triaged » AI Assistants API
Issue tags: +needs backport to 1.1.x, +Needs upstream bugfix, +priority

Rebasing and since the MR is on 1.2.x, I'll change to that but set issues tags.

marcus_johansson’s picture

Issue tags: +AI Initiative Sprint
jibran’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

Added some review comments. I think we should also add the tests to check that the fix is working.

anaconda777’s picture

This patch fixed the issue for me:
https://git.drupalcode.org/project/ai/-/merge_requests/871.patch

Thank you

arianraeesi’s picture

Issue tags: +AI Product Development
abhisekmazumdar’s picture

Assigned: Unassigned » abhisekmazumdar

Let's see what it needs.

abhisekmazumdar’s picture

Assigned: abhisekmazumdar » Unassigned
Status: Needs work » Needs review

Thank You for the changes and testing so far.

I added a new commit with some improvements that will:

* Pre-validate provider/model before form generation to prevent exceptions
* Validate models in AJAX callbacks to catch invalid selections immediately
* Add error logging for provider failures (logged to 'ai' channel)
* Show generic user-facing error messages instead of exposing technical details
* Clear invalid values from form state automatically
* Cache provider options to reduce redundant calls
* Fix AJAX callback to use proper class reference
* Add accessibility attributes to error messages

b_sharpe’s picture

Assigned: Unassigned » b_sharpe
b_sharpe’s picture

Assigned: b_sharpe » Unassigned
Status: Needs review » Needs work

I think there's a lot of overkill here. I tried testing a fresh 1.2.x with an invalid provider and it didn't cause any errors, but just reverted back to default. It does have a Try/Catch that it is hitting when trying to create an invalid instance. We can use that to alert the user their provider is not valid.

When using an invalid model on a valid provider, there was no error; however, it just used the default config. This one I believe we could gate by adding modal validating into the same try/catch as it's up to the provider to give this with getAvailableConfiguration($operation_type, $model);. OpenAi for example just gives you defaults if the model is invalid.

I think this entire problem goes away inside of `src/Service/AiProviderFormHelper.php` and with just providing the user a warning on exception.

in `generateAiProvidersForm()`

      try {
        $model = $form_state->getValue($prefix . 'ai_model', 'Undefined');
        $llmInstance = $this->aiProviderPluginManager->createInstance($provider);

        // .... other stuff

      catch (\Exception $e) {
        // Warn provider/model was set back to default or null if no default available.
      }
joshua1234511’s picture

Assigned: Unassigned » joshua1234511
joshua1234511’s picture

agree we can keep this simpler, in the existing try/catch around provider/model instantiation, on exception we can reset to default and set a clear user warning and still render the form so the user can choose another provider/model.
That should fix the “stuck when current provider is unavailable” case without the extra validation/caching.

Just show a user-facing warning when the provider/model is invalid or reverts to default.

I’ll simplify the MR to that approach, if all agree.
Will keep this open for suggestions for this week and implement if no one disagrees.

joshua1234511’s picture

There have been no further updates or objections to the proposed simplified approach, so I’m picking this up and proceeding with the implementation.

joshua1234511’s picture

Status: Needs work » Needs review
joshua1234511’s picture

Assigned: joshua1234511 » Unassigned

a.dmitriiev made their first commit to this issue’s fork.

a.dmitriiev’s picture

Status: Needs review » Reviewed & tested by the community

I have rebased the MR branch and now phpstan is green. There is one warning for php unit next minor, but it is also not related to the change in this MR and it was seen before.

I also agree with the approach from MR 1278, as it is easier and straightforward.

fago’s picture

Status: Reviewed & tested by the community » Needs review

Please make sure to manually verify the change is good as part of the review.

a.dmitriiev’s picture

Status: Needs review » Reviewed & tested by the community

Sorry, I forgot to write that I have manually tested it.

Testing steps:

1. Install OpenAI provider and configure the key.
2. Install Anthropic provider and configure the key.
3. Create an Assistant, select OpenAI provider and the model.
4. Change the key for OpenAI provider here `/admin/config/system/keys` so it is not valid anymore.
5. Clear the caches.
6. Go to assistants page - observe the error message.
7. Now try to change to Anthropic. The provider can be changed without errors.
8. Change back to OpenAI - observe the error message again.
9. Try to save, observe the error message that the model needs to be selected.

  • fago committed bc7fa259 on 1.2.x authored by joshua1234511
    fix: #3541483 Warn when provider/model is invalid in AI form helper
    
    
    By...

  • fago committed 3d689c82 on 1.x
    fix: #3541483 Warn when provider/model is invalid in AI form helper
    
    
    By...

  • fago committed 67cc15ac on 1.3.x
    fix: #3541483 Warn when provider/model is invalid in AI form helper
    
    
    By...

  • fago committed 7d0ecf8b on 1.4.x
    fix: #3541483 Warn when provider/model is invalid in AI form helper
    
    
    By...

  • fago committed c9a82d81 on 2.x
    fix: #3541483 Warn when provider/model is invalid in AI form helper
    
    
    By...

  • fago committed 14944d87 on 2.0.x
    fix: #3541483 Warn when provider/model is invalid in AI form helper
    
    
    By...
fago’s picture

Status: Reviewed & tested by the community » Fixed

thx, merged!

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

  • fago committed 3d689c82 on 3574181-assistant-agent-form
    fix: #3541483 Warn when provider/model is invalid in AI form helper
    
    
    By...

  • fago committed c9a82d81 on 2.0.x
    fix: #3541483 Warn when provider/model is invalid in AI form helper
    
    
    By...

arianraeesi’s picture

Issue tags: -AI Product Development