Problem/Motivation

Currently it is not possible to show fields from the backend that were inserted outside of the scope of Drupal. When working with backends such as elastic search and solr there are use cases where there are fields available in the documents that are not part of the Drupal indexing logic. Or in the case of integrations with existing Solr/Elastic Search indexes we want to show lists of that data without touching that data. It could also be that you want to create facets for those fields.
In the case of multisite searching, we want to show data from another drupal site in our Drupal 8 site, adding this would allow that.

Proposed resolution

Add a section in the search api field listing page that shows the "external" fields,filled up with information straight from the backend. If they are available in the field listing, facet api and views can easily use those as well to add support for them. They should not be removable and do not have any action associated with them. They should be collapsed by default as this is a rather advanced use case.

Remaining tasks

Everything

User interface changes

drunken_monkey is refactoring the search api field listing and how fields are being added. This impacts that work or vice versa. It's recommended to wait with this issue until that refactoring is done.

API changes

Some new functions to get external fields should be added to the backend interfaces

Data model changes

Not that I know

Estimated Value and Story Points

This issue was identified as a Beta Blocker for Drupal 8. We sat down and figured out the value proposition and amount of work (story points) for this issue.

Value and Story points are in the scale of fibonacci. Our minimum is 1, our maximum is 21. The higher, the more value or work a certain issue has.

Value : 13
Story Points: 13

Comments

Nick_vh created an issue. See original summary.

drunken monkey’s picture

Status: Active » Postponed
Related issues: +#2574969: Add a Views-like UI for adding fields

Hm, yes, probably makes sense. While it's easy to implement something like this in the module itself (via a datasource and some custom code), I guess it would save a lot of code and also potentially make things clearer for users if we implement this right in the Search API itself.
As you said, this is postponed for the moment, but then we should definitely look at it again.

Btw, the issue for refactoring the "Fields" tab/workflow is #2574969: Add a Views-like UI for adding fields.

mkalkbrenner’s picture

This is a real use-case, especially for the multisite feature that came from apachesolr 7.x.
See #2682347: Expose solr-only fields to views.

drunken monkey’s picture

Status: Postponed » Active
drunken monkey’s picture

Status: Active » Needs review
Issue tags: +DevDaysMilan
StatusFileSize
new10.87 KB

This is a first attempt at implementing this, but most likely needs some refining. Best would be if someone from the Solr module could take a look and maybe try to implement the method, and then we'll see whether this makes sense or where it has weaknesses.
Some things to (additionally) consider:

  • To avoid clashes between user-defined and backend-defined fields, I've now defined all field IDs starting with "search_api_" as reserved and ask backends to use that as a prefix for their fields.
  • I'm not completely consistent in calling them "backend-defined fields". Do you find that this harms understanding/clarity? Should we be consistent and, if so, would "server-defined" be a better term?
  • Should we maybe add our three to four "magic" fields, defined in the query interface, too, for Index::getFields(TRUE)?
  • Is the Views code handling this sensible like this? I assume server-defined properties will usually not have a property path set – and otherwise, they should also have a data definition and be available as fields. Might not make sense, though.

Status: Needs review » Needs work

The last submitted patch, 5: 2625154-5--backend_defined_fields.patch, failed testing.

The last submitted patch, 5: 2625154-5--backend_defined_fields.patch, failed testing.

drunken monkey’s picture

Status: Needs work » Needs review
StatusFileSize
new1.36 KB
new12.23 KB

This should fix the test fail, but the questions in #5 of course still remain.

Status: Needs review » Needs work

The last submitted patch, 8: 2625154-8--backend_defined_fields.patch, failed testing.

drunken monkey’s picture

Status: Needs work » Needs review

Most likely just the branch fails.

borisson_’s picture

Status: Needs review » Reviewed & tested by the community

Looks great, good work @drunken monkey

mkalkbrenner’s picture

Having that support implemented within Search API will be great!
Even if I still have to figure out how the building blocks to implement multi-site search using this new API, I think the current proposed starting point should be committed quickly.

  • drunken monkey committed 8bbd514 on 8.x-1.x
    Issue #2625154 by drunken monkey: Added support for backend-defined...
drunken monkey’s picture

Status: Reviewed & tested by the community » Fixed

OK, if you say so. Thanks for reviewing!
Committed.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.