Compatibility with AI search "RAG/Vector Search"

I spent yesterday looking into this for the first time (RAG tool with AI Chatbot) and in the cases where you want to use a View or different Solr options (parse mode, dismax/edismax, etc) its lost in the tool because its called and executed directly. This was causing me to never get any results.

That caused me to write my own plugin like:

  /**
   * {@inheritdoc}
   */
  public function execute() {
    // Collect the context values.
    $this->searchString = $this->getContextValue('search_string');

    $end_results = [];

    try {
      $view = Views::getView('search');
      $view->setDisplay('sitesearch');
      $view->setExposedInput(['keyword' => $this->searchString]);
      $view->execute();

      $i = 1;

      /** @var \Drupal\views\ResultRow $result */
      foreach ($view->result as $result) {
        /** @var \Drupal\node\NodeInterface $node */
        $node = $result->_entity;
        $url = $node->toUrl()->toString();
        $end_results[] = "#$i: " . $node->getTitle() . " (<a href=\"$url\">visit page</a>)<br>";
        $i++;
        if ($i >= 6) {
          break;
        }
      }
    }
    catch (\Exception $e) {
      $this->setOutput("We're sorry, but I could not search the site. Please give me a few moments and try again.");
      return;
    }
    if (count($end_results)) {
      $output = "I was able to find some relevant content for you! Here are the top results based on what you asked:<br><br>";
      $output .= implode("<br/><br/>", $end_results);
      $output .= "<br/><br/>-----<br/>Didn't find what you were looking for? Try using our more robust <a href='/search'>site search</a>!";
      $this->setOutput($output);
    }
    else {
      $this->setOutput("No results were found when searching in the rag index " . $this->index . " for the following prompt: " . $this->searchString . ".\n");
    }
  }

This produced results. I could also confirm that the event subscriber in this module was also fired, so it performed a RAG search against Solr. These are probably items to report back to the main ai_search module because it can be a little confusing when the RAG tool returns nothing. I see that ai_search module provides its own search backend that creates the content property.

Anyway, if I add that event to the current query event subscriber:

  public function postExtractResults(PostExtractResultsEvent $event): void {
    $results = $event->getSearchApiQuery()->getResults();

    foreach ($results as $result) {
      $result->setExtraData('content', 'Content string here');
    }
  }

I can see the extra data in the result on the FunctionCall plugin I made. Perhaps the shortest solution here is adding on the processor setting to ask the user "which" field should be returned as 'content' in this context.

Something like this perhaps?

A more complete example may look something like this:

  /**
   * {@inheritdoc}
   */
  public function postExtractResults(PostExtractResultsEvent $event): void {
    $query = $event->getSearchApiQuery();

    if ($query->getIndex()->isValidProcessor('solr_densevector')) {
      try {
        $processor = $query->getIndex()->getProcessor('solr_densevector');
        $settings = $processor->getConfiguration();

        if (!empty($settings['content_field'])) {
          $results = $event->getSearchApiQuery()->getResults();

          foreach ($results as $result) {
            $field = $result->getField($settings['content_field']);
            $field_values = [];
            
            $values = $field->getValues();

            foreach ($values as $value) {
              $text = ($value instanceof TextValue) ? $value->getText() : $value;
              $text = strip_tags($text);
              $field_values[] = $text;
            }

            $result->setExtraData('content', implode(' ', $field_values));
          }
        }
      }
      catch (\Exception $e) {
         // log here
      }
    }
  }

I tested this using the node title field only.

I've added some additional touches.