HS Taxonomy filter with relation - Received an invalid response from the server

I want to filter on a taxonomy with hierarchical select selection type, but this taxonomy is not directly mapped to the node type but is accessible via a node relation.

Using the default filter selection types (list, autosuggest) the filter works fine, but when I switch to the HS selection type I receive:
Received an invalid response from the server