Follow-up to #2623012: Implement interfaces and base classes for URL-based sources

getRequestHeaders() and setRequestHeaders() are currently part of the DataFetchPluginInterface, but are really only applicable to an HTTP fetcher. Let's get them out of the general interface. I see adding a "fetcher_options" configuration key for the fetcher plugin, where headers could be specified.

CommentFileSizeAuthor
#4 make_custom_headers_an-2640512-3.patch5.24 KBmikeryan
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

mikeryan created an issue. See original summary.

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Murphysboro, IL, USA
CreditAttribution: mikeryan at Virtuoso Performance commented
Component: Source plugins » Plugins
mikeryan’s picture

mikeryan
he/him
Primary language English
Location Murphysboro, IL, USA
CreditAttribution: mikeryan as a volunteer and at Virtuoso Performance commented
Version: 8.x-1.x-dev » 8.x-2.x-dev
Issue tags: -Needs tests

So, DataFetcherPluginInterface currently has setRequestHeaders(), getRequestHeaders(), getResponseContent(), and getResponse() APIs. Only getResponseContent() is relevant in general for fetchers, and the name sort of assumes HTTP - it should be renamed getContent() (I'll leave a BC stub implementation for it).

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Murphysboro, IL, USA
CreditAttribution: mikeryan at Virtuoso Performance commented
Status: Active » Needs review
FileSize
make_custom_headers_an-2640512-3.patch5.24 KB
Grayside’s picture

Grayside CreditAttribution: Grayside at Phase2 for Norwegian Cruise Line commented

In my work on #2640516: Support paging through multiple requests I've been thinking of HTTP headers as "metadata", with the idea that other fetch methods could implement other sorts of metadata extraction mechanisms than reading headers. (e.g., id3 tags on music files). Overall makes sense to pull it from the interface, but something along the metadata notion might be valuable when more parser plugins use the data fetchers.

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Murphysboro, IL, USA
CreditAttribution: mikeryan at Virtuoso Performance commented
Status: Needs review » Needs work

The headers in this case are the *request* headers, being sent out. Extracting response headers (or other metadata) would be a subject for another issue entirely...

But, that does remind me of the "fetcher_options" thought I had in the issue summary, I should apply that here...