Mapper to joined tables supporting multiple values per field [#554060]

This preliminary patch allows the feedapi_data mapper to map to other data tables that join to the base feedapi_data table. For instance, you can store an array of tids provided by a parser along with each feed item.

The mapping side works quite well, currently querying all data tables and finding those that join, and then inserting an arbitrary number of them linked by said joined field from the newly inserted feedapi_data base record. For greater reliability, this module creates a unique id in the feedapi_data base table default schema that is the ONLY thing it will find joins against. This is 1) to prevent something like blank ids messing up/orphaning the insertions if weird things are mapped to each other and 2) to make the deletion scripts somewhat more modularized and intuitive...

The main remaining issue relates to the deletion and maintenance of records on updating feedapi_data items and purging them. Presently it will assume ownership of any records in any data table that joins to the feedapi_data table on the new default 'id' serial field. This means it will remove and recreate all such records on update, and will delete all such records on purges. This was the main reason we didn't allow arbitrary joins to be mapped, because you might, perhaps, parse a node nid and store it in your default feedapi_data base table and join it to some nid map with some other info, in which case deleting the data row probably wouldn't be expected to also delete all records joined and mapped to it.

Currently the id-based deletion might still be too unpredictable to some. To me it is not the end of the world in the default iteration because this particular id is specific to the feedapi_data base table, so once that record is removed the id would ~always~ reference a non-existent piece of information in any other tables in which it appears. It is conceivable the user would want to keep some of those rows even if the mappings are orphaned--if they want to retain a glossary of terms, for instance, and don't care where they came from anymore--but that isn't supported :-P

Ideally there would be a specific setting to tell whether certain rows were "dependents" of other rows in the data module, and it would then support inserting, deleting, etc of such records within its own API. This is an extensive change for which time is not currently alloted, however, so in the meantime this may be a viable simple working solution.

Comment	File	Size	Author
#6	554060-4_join_map_and_cleanup.patch	13.73 KB	David Goode
#5	554060-3_join_map_and_cleanup.patch	13.64 KB	David Goode
#3	554060-2_join_map_and_cleanup.patch	13.5 KB	David Goode
#1	554060-1_map_to_joined.patch	5.99 KB	alex_b
	feedapi_data.joinmapping.patch	5.38 KB	David Goode

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

alex_b CreditAttribution: alex_b commented 20 August 2009 at 01:29

File	Size
554060-1_map_to_joined.patch	5.99 KB

Nice work. Cleaned up some comments.

I wonder whether we could clean up the code in feedapi_data_after_refresh(). I'm thinking of something like:

if ($data = feedapi_mapper_map($feed_node, 'feedapi_data', $item)) {
  $table_data = _feedapi_data_unpack($data, $table);
  foreach ($table_data as $t => $data) {
    $t->save($data);
  }
}

Encapsulating the rather complex unpacking routine would make the code more legible.

jt1, thisdatatable : use right_table and this_table instead.

There may be a smarter return value from feedapi_data_get_joined_tables() that would be more handy for what you are trying to use the result for.

Maybe:


  array(
    'table_name_1' =>  'field_name', 
    'table_name_2' => 'field_name',
    //...
  )

This could greatly simplify the code that you need for unpacking the result of feedapi_data_get_joined_tables() in the both places where you invoke it.

Comment #2

alex_b CreditAttribution: alex_b commented 20 August 2009 at 16:27

4) We _could_ use $sub_fields for storing field information for depending tables.

Comment #3

David Goode CreditAttribution: David Goode commented 20 August 2009 at 21:30

File	Size
554060-2_join_map_and_cleanup.patch	13.5 KB

Here is a monster patch that 1) cleans up the code as per alex's request as far as reasonably possible 2) Makes feedapi_data use save() and update() instead of after_refresh and 3) implements the expire hook as asked for in #552420: Implement item expiry.