# Summary

Import or aggregate data as nodes, users, taxonomy terms or simple database records.

# Project URL

https://www.drupal.org/project/feeds

# Where is the code?

https://www.drupal.org/node/2022803

# Estimated completion date

Unknown

# Dependencies

N/A

# Who's doing the port?

Unknown

# What help do they need?

# D8 roadmap

#1960800: [meta] Feeds 8.x roadmap

# Background and reference information

#1960800: [meta] Feeds 8.x roadmap

Comments

Kristen Pol created an issue. See original summary.

mgifford’s picture

Issue summary: View changes
MegaChriz’s picture

Issue summary: View changes

My personal plan for working towards a working D8 release:

  1. First, get a stable Feeds 7.x-2.0 out.
    This will make my mind free to focus solely on the D8 version. If I would work on the D8 version right now, I would feel I have to switch back a forth constantly between both versions and I think that doesn't help to make good progress.

    To me, a stable Feeds 7.x-2.x release means when at least the following issues are fixed:

  2. Rework the UI: #2443471: Implement configurable parsers..
    It would be good to tackle the weak spots of the UI that are in the 7.x-2.x version. A great UX improvement would be the possibility to provide the file at the beginning of the process, so Feeds can detect the source format and columns. What blocks for moving this forward is the fact that the workflow for providing a file at the beginning doesn't fit all use cases.
  3. Port features of the 7.x-2.x version.
    In the past few years a number of new features were added to Feeds that are not yet available in the D8 version, for example:

    I think it would be wise to not start porting these until the base features and the UI of Feeds are more stable.
    As these features are backed up with tests it will hopefully be easier to port them by just starting to port the tests first.

  4. Testing and bugfixing.
  5. Stable D8 release.

People could help me with starting on the D8 version sooner by help fixing the listed 7.x-2.x issues from above with:

  • Reviewing them thoroughly (not "It works for me" messages).
  • Writing automated tests
james.bcn’s picture

In order to test the D8 feeds module as it currently stands, once you have set up a feed type, how does one trigger an import? I'm impressed with how complete it seems at this stage, but I'm unable to do an actual import.

MegaChriz’s picture

@james.bcn
Go to Administration > Content > Feeds (http://ww.example.com/admin/content/feed) and add a Feed (http://ww.example.com/feed/add).

james.bcn’s picture

Excellent thank you @MegaChriz

MegaChriz’s picture

Issue summary: View changes

I've set "Who's doing the port?" to "Unknown", as I've deciced to not work on the D8 port of Feeds in the nearby future. However, if someone steps up and is committed to work on the D8 version for a long time or if funding for the project is possible, I may change my mind. But for now I say no.

TomSherlock’s picture

Hi, MegaChriz.

Just saw your comment (#7) about no longer supporting port of Feeds to D8.

Who can I direct my question to? I essentially want to get a sense of current state of the module and its current UI.
Please see: https://www.drupal.org/node/1960800#comment-11125863

Thanks & Regards

thomasmurphy’s picture

Hi, I'm playing Drupal 8 catchup and the lack of feeds seems like a biggie. Forgive my ignorance, but does the new architecture of Drupal 8 mean that feeds is less necessary, or is it just something that the community hasn't got around to yet? The migrate module is in core, but has no UI, as far as I can tell, so isn't really a feeds substitute. In the mean time I'll try out this and see if it breaks https://www.drupal.org/project/feeds/releases/8.x-3.x-dev

tim_dj’s picture

How much funding are we talking about? Having feeds module in Drupal 8 seems essential to me!

I am sure we could start some sort of crowd funding for Feeds if needed.

thomasmurphy’s picture

We've just used feeds on a D8 site and it turned out to be a much better approach than using the migrate module approach, despite the current documentation. The only downside is we needed the xpath parser module, which doesn't exist for 8 yet, so we had to write a feeds submodule for the specific transform required. This was a big topic at our latest Drupal meetup, and we'll be discussing the feeds module family at Drupal South in a few weeks, hopefully building some momentum.

johnatasJMO’s picture

Hi #thomasmurphy, can you share how was xpath parser module created? I need to use feeds for simple article import from an RSS feeds. Is the development too extensive? Otherwise, I will have to consider to go with D7.

How can i help?

MegaChriz’s picture

@chocosanto
For more information about the port of the XPath parser, see #2465535: [meta] Feeds extensible parsers 8.x roadmap.

thomasmurphy’s picture

I had a good chat with Lee Rowlands at Drupal South and he mentioned that the aggregator module in D8 core has been rebuilt based on some mature Symfony tools do a lot of what people used feeds for in D7, but the documentation is lacking. I'm going to look into this and possible contribute some documentation if I make any progress.

daboo’s picture

thomasmurphy,

I tried aggregator before feeds. While it does the same stuff as feeds, it has the following limitations: feed items are not stored as nodes, thus are not indexed for inclusion in local searches. This limits SEO for these items as well. It is possible to add on more advanced search functionality to include in local searches, however, I didn't like how it performed. I also didn't like the complexity for future additions and changes.
Feeds works great on Drupal 8. It stores data in nodes so I can create custom meta tags. The pages can be laid out exactly as I want them without limitations. I also tag the feeds so that they can be easily grouped.
The only piece that does not work at the moment is the scheduler, which means that i monitor 200+ feeds each day and manually kick off the update for new feeds once per day which is not a huge deal.
Http://hauntersweb.com is an example. Tags are displayed on the right under "spider sites" and summaries at the bottom of the page for the latest feed entries.

wturrell’s picture

Hi. What, if anything, would be suitable for selectively exporting/importing nodes between sites in D8 at present? (Sites would have same config & UUID, if that helps, and not necessary for it to be fully automated.)

thomasmurphy’s picture

@daboo I meant to adapt the Symfony tools that are already available to Drupal 8 to achieve a feeds "node creation" end result, rather than just using the aggregator module directly. That's what Lee Rowlands suggested, at least.

thomasmurphy’s picture

webchick’s picture

Just noting here that Feeds not being D8-ready yet is coming up repeatedly in an informal survey on what's blocking people from adopting Drupal 8.

So it seems like for the health of the ecosystem, we need some sort of solution here. Here's my best understanding of the current situation:

  • There is currently an 8.x-3.x dev release of Feeds, though this is not listed on the project page (maybe one simple thing to do, to help raise awareness that it's in progress?).
  • It sounds like @MegaChriz is mostly a volunteer contributor, and is actively asking for either other folks to take the development lead here, or funding to help him prioritize this work. He's also moving house, which is going to adversely affect his availability for Feeds until at least November (understandably!).
  • There was also a survey done, to try and hone in on an "MVP" for Feeds 8.x-3.0 functionality, which appears to be: "a working CSV parser, a working XML parser and a port of Feeds Tamper. Nice to have features like unpublishing/deleting nodes that no longer appear in the feed or language support as in the 7.x-2.x version may not make it in the 8.x-3.x version (or maybe in a very basic form, if possible)."
  • There have also been tentative plans discussed for an 8.x-4.x branch which utilizes the core Migrate system, but this is currently not started. (And, honestly, makes sense to stabilize what's there now first.)
  • In terms of alternatives:

Anything I missed?

MegaChriz’s picture

@webchick
That's a nice summary, thank you :). One thing I like to add what also plays a role is that I feel I also need to take care for the D7 version of Feeds, which has a quite large user base. I notice myself wanting to keep getting back to that version and fixing (major) bugs in it. Maybe because I feel I better control it, maybe because I still see the D8 version as one big mountain to climb. I'm also considering if I should ignore more support requests, in order to have more time for development. But then I feel like I leave these users in the cold. Anyway, I find it too hard to ignore them for long.

I'm indeed trying get the development of the D8 version ongoing, but so far I'm failing to keep pushing through. But I will keep trying :). And I'm certain if there would be more developers dedicated to Feeds I would get more motivation to go on with it. Feeling I'm standing alone on the development side kinda reduces my motivation. I do have received support in the past in terms of UI sketches and plans for steps to take for using Migrate components in Feeds. I also received help with creating the survey. And I was pleased with the amount of responses on the survey: 288!

thomasmurphy’s picture

So do we have the final feeds user survey results? There are people in the New Zealand Drupal community happy to contribute resources. Do you still want a UX definition provided?

MegaChriz’s picture

@thomasmurphy
That's great news! I hope I can find time to write a plan for moving Feeds forwards. I've already some UI designs to start with.

The survey results can be found in #2847682-5: Drupal 8 survey Feeds/Migrate usage and functionality.

Well, roughly this are the next steps:

  1. Finish the CSV parser UI: #2669974: Implement the CSV parser UI
  2. Create an UI for the XML parser in https://www.drupal.org/sandbox/megachriz/2465523
  3. Port Feeds Tamper to D8.
  4. Release 8.x-3.0-alpha1
  5. Investigate which components of Migrate can be used for Feeds.

What I plan to do:

  1. Open issues to coordinate the work and write down substeps.
  2. Close down duplicates issues (now information about the development is scattered over a few issues).
  3. Update the project page to reflect what the next steps are.
  4. Release 7.x-2.0-beta4 (and document its changes in change records).

Thomas, if you would like to continue with #2669974: Implement the CSV parser UI and help to make it work without errors, that would be great.

thomasmurphy’s picture

I'll see if I can get it into the Drupal South code sprint, or otherwise hustle :)

thomasmurphy’s picture

Feeds 8 port is now top of the list for the Drupal South code sprint (November 15th Auckland) so throw your issues at me and I will attempt to throw devs at them.

memoday’s picture

@thomasmurphy Thanks Thomas! This is great news!

We need the same functionality as the one in D7 version and Feeds Tamper module is actually complementing the Feeds module and it's very important. CSV and TSV import are so important and they should work without errors.

pcranston’s picture

@thomasmurphy - awesome! Things I'm really dependent on are:

- XML parser
- unpublishing nodes that have dropped out of the feed

thanks!!

MegaChriz’s picture

Alright, I've updated the short term and long term goals for Feeds. I also explained in detail the first next steps that need to be taken to accomplish the short term goals. I hope this gives direction to people who want to help with the development.

See #2917928: Plan for 8.x-3.0-alpha1 release for the short term goals.
See #1960800: [meta] Feeds 8.x roadmap for the long term goals.

What I plan to do next:

This means that at least for now I'll leave implementing the UI's for the CSV parser and the XML parser to others. I do plan to review code that is posted though.

Oh, and I love some feedback on the written steps that I have written for #2669974: Implement the CSV parser UI and #2917924: Implement the XML Xpath parser UI. Does it give you developers enough direction to get started on them?