Much of the heavy lifting is currently done in Search API Backend plugin yet is quite Milvus/Zilliz specific. Move it away into VDB Providers interfaces/base class to allow different VDB Providers sufficient control.
Attempting to do Pinecone #3470417: Provide integration with Pinecone revealed this; however, similar problems would happen for any VDB Provider
Refactor to allow other VDB Providers
MR coming
VDB Providers can provide their own settings for the search api backend
N/A
Interfaces changed within AI Search
Show commandsStart within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
changes
Comments
Comment #3
scott_euser commentedI added comments to the MR just to make it easier to review. Ultimately though its just moving things over from search api backend into the vdb providers to give the individual providers more control.
Comment #4
scott_euser commentedComment #6
ronaldtebrake commentedI see this added the
"probots-io/pinecone-php": "^1.0"requirement.This comes with a php 8.2 requirement that basically stops allowing us to use it on Drupal 10.2 where 8.1 is still supported.
Is this truly a hard requirement?
Comment #7
yautja_cetanu commentedDoes this mean ai search itself requires 8.1 even if you're not using pinecone? I think we need the ai module and ai search to support 8.1. Pinecone needing 8.2 isn't bad as long as its possible to use ai search without it
Comment #8
ronaldtebrake commentedYeah, as it's part of the root composer.json it means whenever you require
drupal/aiyou get all of the requirements and theirs.Can imagine it's difficult with all the providers and their libraries and dependencies and maintenance of those.
Comment #9
scott_euser commentedI have no idea how to make dependencies requirements of specific sub-modules only. Thus far all dependencies per sub-module have been just tossed in the ai core composer.json like this. Open to suggestions.
Maybe some mechanism to have all these (like qdrant, pinecone, openai, anthropic) as a suggested composer dependencies (https://getcomposer.org/doc/04-schema.md#suggest) that block enabling the module if not installed. E.g. via https://api.drupal.org/api/drupal/core%21lib%21Drupal%21Core%21Extension...
It makes it harder for end-users to install sub-modules though, but maybe that's the lesser of two evils?
Can we move this to a separate issue though please rather than reviving unrelated closed issue?
Comment #10
scott_euser commentedCreated here, https://www.drupal.org/project/ai/issues/3478274 - will update with notes