This is still happening, and I can reproduce it reliably. I'm reopening with a clean reproduction and a patch.

The problem

On an index with a File datasource and a Content datasource, attachment fields on nodes are never filled in.
Searching for text inside an attached PDF finds nothing. The same attachments search fine on an index without a File datasource.

Steps to reproduce

  1. Create an index with only the Content datasource. Add a "Search api attachments" field for a file field, e.g. saa_field_documents. Index. The field is filled in and the documents are searchable.
  2. Add the File datasource to the same index. Reindex.
  3. saa_field_documents is now empty on every node.

I confirmed this on our site with two indexes side by side, with the same field and the same processor settings. The index without a File datasource returns results. The index with one returns nothing.

The cause

In FilesExtractor::addFieldValues():

// If the parent entity is not a file, no need to parse the
// saa static::SAA_FILE_ENTITY item.
 if (!$is_entity_type_file && $field_name == static::SAA_FILE_ENTITY) {
   break;
 }

When the index has a File datasource, getFileFieldsAndFileEntityItems() adds saa_file_entity to the list before the other datasources' file fields. So for any item that isn't a file, the loop hits saa_file_entity first and break exits before any real file field is processed.

Why this is a bug and not intended behaviour

  • The comment says there's no need to parse that one item. That describes continue, not break.
  • The result depends on datasource order. If the File datasource comes after Content, the node fields get processed. An intentional rule wouldn't change with array order.
  • It fails silently. The field is on the index and configured, but it stays empty with no warning.
  • It doesn't serve an access purpose. Text indexed on a node is subject to the node's own access. Private files are handled by the separate "exclude private files" setting.

I suspect this is why it was closed as "cannot reproduce": with the File datasource after Content, the bug doesn't show.

The fix
Change break to continue. It's a one-line change, and I've attached a patch. It only skips the file-entity item for items that aren't files, which is what the comment describes. Items that are files behave exactly as before, so the risk of a regression is low. We've run it with both datasources on one index and everything indexed as expected.

Comments

garethhallnz created an issue. See original summary.

izus’s picture

thanks for this

  • izus committed 8bf04fa5 on 10.0.x
    fix: #3626968 FilesExtractor stops processing file fields when the index...
izus’s picture

Status: Active » Fixed

this is now merged

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

garethhallnz’s picture

Status: Fixed » Needs review
garethhallnz’s picture

Status: Needs review » Closed (outdated)

Resolved in release 10.0.12

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.