Plan for Feeds 7.x-2.0-beta4 release [#2826403]

For Feeds 7.x-2.0-beta4 somewhat bigger changes are planned in order to fix major bugs.

Feeds 7.x-2.0-beta4 is planned to be released on September 24, 2017.

Cron issues
There are a lot of reports about imports that do not work during cron. We hope to fix at least some of these by running the imports during cron as either user 1 or the user who created the feed node.
#2541944: Switch to feed author or user 1 during imports (taxonomy mapping does not work with cron)
#1995728: [META] Cron import not working on 7.x-2.0-alpha8 and later
Validation
Feeds will be stricter on validating imports to minimize SQL errors on imports. Furthermore, importers itself will be validated too. At first only for if the configured bundle and language on the processor exist. Feeds will also no longer allow attaching two importers to the same content type. Some settings will get locked when there is already imported data.
#2379631: field_attach_validate() must be called before programmatic entity saves
#2474589: Changing processor wipes mappings with no warning
#2640108: Display an error message when attaching an importer to a content type that is already in use by an other importer
#2320781: Validate feed importer configuration: check for invalid bundle and invalid language
HTTP Fetcher issues
There are two big issues with the HTTP fetcher. They are both related to imports that need multiple cron runs to complete. One is that a source is entirely stored in the database. Second is that a source to import from is stored as a temporary file, which is removed after 6 hours. If the import hasn't been completed by then, the import will be aborted and the importer will be stuck in a locked state.
#2032333: [Meta] Store remote files in the file system instead of in the database
#1029102: Importing Large CSV document (downloaded and processed in the background)

Comments

Comment #1

10 November 2016 at 17:18

MegaChriz created an issue. See original summary.

Comment #2

m.stenta

he/him

English

commented 23 August 2017 at 13:53

Bumping this issue - any chance we could put out a beta4 and move the remaining issues mentioned in this to a new one for beta5? There are a lot of bug fixes in the 7.x-2.x dev branch right now that would be nice to include in a recommended release.

Comment #3

megachriz

Dutch

commented 23 August 2017 at 14:07

@m.stenta
I was hoping to finish off #1029102: Importing Large CSV document (downloaded and processed in the background) first, but maybe you are right and I should postpone that one to the next release. I have not enough time right now to verify if there was something else left to be done before I could create a new release. As there are some changes in dev that could in theory break contrib (a lot of methods are added to the FeedsFetcherResult class that could in theory already be defined by classes that extend that class in contrib), I would need to write instructions on how to downgrade to Feeds 7.x-2.0-beta3.

Comment #4

m.stenta

he/him

English

commented 23 August 2017 at 14:50

Mm yea, breaking changes make things tricky. I think this one also has potential to break things: #2379631: field_attach_validate() must be called before programmatic entity saves

Documentation is a good idea, at the very least. Another option might be to start a 3.x branch for the breaking changes. Hard to go back on the things that are already committed to 2.x though... :-/

Comment #5

megachriz

Dutch

commented 23 August 2017 at 15:00

7.x-3.x is an option indeed, though I think that the chance of breaking things is small. And if I would open 7.x-3.x now, Feeds 7.x-2.0-beta3 will likely be the last release of the 7.x-2.x series. Some of the changes that have been made since 7.x-2.0-beta3 were made to fix major bugs. So in my opinion, a 7.x-3.x release or a 7.x-2.0-beta4 makes little difference.

Comment #6

m.stenta

he/him

English

commented 23 August 2017 at 15:24

I suppose one disadvantage of using 7.x-3.x is that it may signify some relation to the 8.x-3.x branch. But they will be vastly different (I assume?), so that might be misleading. The changes between 7.x-2.0-beta3 and the current 7.x-2.x-dev are not very significant (not major refactoring), so a 3.x branch might not be necessary anyway.

Are you aware of any contrib modules that extend FeedsFetcherResult? If the risk is small, then I think it would be OK to describe the potentially breaking changes in the release notes.

I don't think there are any implementations of hook_update_N() since the 7.x-1.0-beta3 release, so there aren't any database changes. If someone experiences an issue with beta4, the downgrade back to beta3 would be simple: replace the module with beta3 (https://ftp.drupal.org/files/projects/feeds-7.x-2.0-beta3.tar.gz) and clear the cache.

I think I would vote for "7.x-2.0-beta4", given those thoughts.

Comment #7

megachriz

Dutch

commented 23 August 2017 at 15:31

Yes, almost every module that provides a Feeds fetcher plugin does extend the FeedsFetcherResult class. What I did though while I was adding these extra methods was checking every module providing a Feeds fetcher that I could find on drupal.org (basically from this list: https://www.drupal.org/node/856644) and at least by looking at the code it didn't seem I would break these.

There is in fact a database update since 7.x-2.0-beta3 and one that would cause errors when going back to 7.x-2.0-beta3. See #2863209: Error: Class 'FeedsHTTPCache' not found in _cache_get_object().. The instructions for downgrading to 7.x-2.0-beta3 are basically in comment #5 of that issue.

Comment #8

m.stenta

he/him

English

commented 23 August 2017 at 15:34

Ah great! Thanks for the links and clarifications! Good info!

Comment #9

megachriz

Dutch

commented 21 September 2017 at 11:44

Change records for the beta4 release have been created. I now have to write the release notes, fix one UX issue that I found and update the changelog. Hopefully I can make the release in a few days.