Solr VDB Provider

Uses Apache Solr's dense vectors as a vector database provider for the AI module and AI Search. A site that already runs Solr for keyword search can run its semantic search on the same service, so it doesn't need a second vendor.

Features

• Stores AI Search embeddings in a Solr core and searches them with Solr's kNN query parser.
• Connects to Solr over HTTP through Drupal's own HTTP client. It does not need Search API Solr, and both can use the same Solr server.
• Keeps every collection in one core, so one core can serve any number of AI Search servers.
• Adds its vector fields and field types through Solr's Schema API, so there is no configset to copy or upload.
• Supports cosine, dot product and Euclidean similarity. On Solr 10 it also supports scalar and binary quantization, which fit several times as many vectors in the same memory.
• Turns AI Search's conditions into Solr filter queries that run before the kNN search, so a filtered search still returns the nearest matches. Supported operators: =, <>, IN, NOT IN, <, <=, >, >=, BETWEEN and NOT BETWEEN.
• Converts Lucene's scores back into the similarity or squared distance that Milvus returns, so AI Search's score thresholds mean the same thing on Solr.
• Groups chunks by item when AI Search checks access to each result, so each page of results lists distinct items.
• Supports basic authentication, with the password stored in a Key.

Requirements

• Drupal 10.3 or 11.
• AI 1.2 or later, with AI Search enabled.
• Key.
• Solr 9.6 or later. Quantization needs Solr 10.
• A core for the vectors, made from Solr's _default configset. For example, in Docker: solr-precreate vectors. The provider adds its own fields and turns off the configset's guessing of new fields.

Setting up

1. Enable the module.
2. Go to Configuration » AI » Vector Database Providers » Solr Configuration and enter the scheme, host, port, path and core. For basic authentication, enter a user name and pick a Key that holds the password. The form pings the core before it saves.
3. Choose the quantization for new collections: none, scalar or binary.
4. Create a Search API server with the AI Search backend, choose Apache Solr as its vector database, and name a collection. The collection is created when the server is saved. For a server imported from configuration, it is created on the server's first indexing.

Limitations

• Vectors are indexed but not stored, so the raw embedding can't be read back from Solr.
• Solr has no databases, so the provider ignores AI Search's database name. Collections are told apart by a field on each document.

Project status

This module is experimental. The storage layout may change before a stable release, and a change could mean reindexing.

Similar projects

Other vector database providers for the AI module include Milvus, Pinecone and Postgres. This module is for sites that already run Solr or prefer to.