In house we have a mature library of functions and regex expressions that we use for our scrapers. After learning about Feeds recently, I realize this may be a better way to work. However, given all the possibilities and and combinations of recipes, I'm a bit confused.
Here is what we do, and any advice on how to get started recreating this functionality using feeds would be appreciated.
Basically we have a target HTML page, for example. One of our functions simply finds the title of the page, strips the tags, and formats it for inclusion into a custom node_save function. Wash rinse repeat for body and other fields, and in the end we have an array of data that makes up the new node to be saved.
Since we already have all these regular expressions to scape content for fields in our node, would REGEX be the way to go here? Simply plug in our expressions and then let FEEDs and Drupal handle all the formatting, etc?
Or, because we do formatting manually, should we simply create a series of custom mappings? I guess what I am asking is we have all these great functions to get and handle data just like we want; it would be great to just add these to feeds somehow so that when the title was requested, the function would run, and then we would have the result we wanted ready to be mapped and saved into a new node.
I am still trying to understand how all this works, so please let me know if that is not clear.
Thanks!
Comments
Comment #1
Gastonia commentedAnyone out there?
Comment #2
twistor commented