My use case is: Client has a stock list which covers their entire range of product, many which are not to be sold through the website, however they would like to use this master list to update the stock for only products on the website and not to create new products.

I guess this may well be resolved if Feeds is fixed to care about Required Fields?

CommentFileSizeAuthor
#57 interdiff-1286298-55-57.txt3.14 KBMegaChriz
#57 feeds-skip-new-items-1286298-57.patch5.32 KBMegaChriz
PASSED: [[SimpleTest]]: [MySQL] 7,182 pass(es). View
#55 don_t_create_new_items-1286298-55.patch2.18 KBcharginghawk
PASSED: [[SimpleTest]]: [MySQL] 6,975 pass(es). View
#47 don_t_create_new_items-1286298-47.patch2.69 KBfietserwin
PASSED: [[SimpleTest]]: [MySQL] 6,996 pass(es). View
#45 don_t_create_new_items-1286298-45.patch2.8 KBfietserwin
FAILED: [[SimpleTest]]: [MySQL] 6,906 pass(es), 16 fail(s), and 18 exception(s). View
#28 feeds-d6-only-update-existing-items-1286298-28.patch5.68 KBosopolar
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch feeds-d6-only-update-existing-items-1286298-28.patch. Unable to apply patch. See the log in the details link for more information. View
#25 1286298-25.patch2.2 KBfietserwin
PASSED: [[SimpleTest]]: [MySQL] 5,105 pass(es). View
#10 feeds-nocreate-1286298-10.patch2.43 KBTriskelion
PASSED: [[SimpleTest]]: [MySQL] 4,503 pass(es). View
#7 feeds-nocreate-1286298-7.patch2.28 KBTriskelion
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch feeds-nocreate-1286298-7.patch. Unable to apply patch. See the log in the details link for more information. View
#6 feeds-nocreate-1286298-6.patch2.4 KBTriskelion
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch feeds-nocreate-1286298-6.patch. Unable to apply patch. See the log in the details link for more information. View
Members fund testing for the Drupal project. Drupal Association Learn more

Comments

johnbarclay’s picture

Category: support » feature

So for the node processor, an option could be added to the "Update Existing nodes" field:

  • Do not update existing nodes
  • Replace existing nodes
  • Update existing nodes (slower than replacing them)
  • Do not add new nodes
killtheliterate’s picture

Love to hear if there's any method to achieve this

checker’s picture

I have the same problem with product stock: update but not create. Do you have found any solution?

checker’s picture

As a work around you can write a little plugin for feeds tamper module. that plugin can check if an entity is available or not. if not you can unset the entry and feeds importer won't import this (and won't create a new entry)

joshualoman’s picture

Anyone found a simple solution for this? I need this functionality too!! Checker's suggestion i don't really understand..

Triskelion’s picture

Status: Active » Patch (to be ported)
FileSize
2.4 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch feeds-nocreate-1286298-6.patch. Unable to apply patch. See the log in the details link for more information. View

I have attached a quick patch to the latest dev version which appears to implement this. I am still in the process of testing.

Update: Typos in code. Corrected patch next

Triskelion’s picture

FileSize
2.28 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch feeds-nocreate-1286298-7.patch. Unable to apply patch. See the log in the details link for more information. View

I should not do these things late at night. Typos corrected.

agileadam’s picture

Patch #7 works great for me! I was confused when I was getting a success message of "There are no new commerce products." but it was because the entities I was importing were unchanged from the existing entities. Checking "Skip hash check" forced the records to update. I tested a csv that had one new product, and one existing product, and it updated only the existing product (did not create the new product). Excellent!

Thanks @triskelion!

twistor’s picture

Status: Patch (to be ported) » Needs work

Very nice! This will need tests though. It should also depend on #1835106: Make entity loading generic. which is going in tonight.

Triskelion’s picture

FileSize
2.43 KB
PASSED: [[SimpleTest]]: [MySQL] 4,503 pass(es). View

Noticed that there have been some changes in the way this module works.

I rerolled the patch with the new dev version. Hope I got it right.

twistor’s picture

That's close, I don't think we need the changes to FeedsNodeProcessor at all.

Triskelion’s picture

It's just a couple of support functions in FeedsNodeProcessor. I made the changes to sync processing with FEEDS_UPDATE_EXISTING as they are logically equivalent, except FEEDS_SKIP_NEW does not create new nodes. Am I misunderstanding the logic?

My main concern is in the timing of the loop escape in FeedsProcessor. As before, I placed it just before the 'try' statement, but the new handling of the entity load threw me off a bit.

Thanks for the prompt acknowledgement.

a.ross’s picture

Until this patch is in, you can get the "update only" behavior by using Rules. Example configuration for products:

  1. Create your importer, for example "Product Stock"
  2. Create a rule that triggers on the event "Before saving an item imported via Product Stock."
  3. Add a condition "entity exists by property", choose "commerce product" entity and choose the SKU field (or any other unique field) and select the SKU of the imported item.
  4. Negate that condition
  5. Add an action "Skip import of Feeds item" and simply choose the imported item.
rotty_dean’s picture

a.ross this rule looks like what i am looking for but i can't seem to get the mapping from the feed. So at the moment everything is skipped. I have tried:

NOT Entity exists by property
Parameter: Entity type: Commerce Product, Property: SKU, Value: [xpathparser:1]

I need this because i have a list of products that need unpublishing, and in that list about 90% was never added to the database so they need to be skipped.

a.ross’s picture

I haven't done this with an XML parser, only a CSV parser, but that shouldn't make a difference.
You should be able to select the field you need (SKU of the imported item) as you normally would in Rules. Click "Switch to data selection" below where you (incorrectly) entered "xpathparser:1". Then you'll be able to select the SKU of the imported "commerce-product".

marktheshark’s picture

Have you installed the unique field module in order to be able to update products with their SKU as a unique key?

johnv’s picture

Status: Needs work » Needs review

Triggering testbot.

marktheshark’s picture

Does the patch concern only nodes, or can it also be used for entities (e.g. Drupal Commerce products)?

a.ross’s picture

The Rule I described works with vanilla Feeds. We have Feeds tamper installed but we're not using it for the stock updates, as a rule makes more sense.

In case you missed it, click "switch to data selection" in Rules to select the SKU of the imported item.

marktheshark’s picture

Thanks, if I wanted to implement the FEEDS_SKIP_NEW functionality, as per the patch, would I need to do it in a different file from FeedsNodeProcessor.inc?

I'm asking because FeedsNodeProcessor.inc probably pertains only to entity types = node.

a.ross’s picture

Don't know, I'm not the author of the patch and I'm happy with our Rule.

marktheshark’s picture

You mean that any changes should apply to FeedsProcessor.inc only so that all others inherit it, right?

marktheshark’s picture

Applied the recommended changes to FeedProcessor.inc and FeedsCommerceProductMultiProcessor.inc (for importing multiple product types at once.

Creation of new entities was successfully skipped, however the updated entities are a bit messed up: all images of the product are now not displaying any more...

fietserwin’s picture

Issue summary: View changes
Status: Needs review » Needs work
  1. +++ feeds.a/plugins/FeedsNodeProcessor.inc	2013-01-18 18:45:07.078496638 -0500
    @@ -54,7 +54,8 @@ class FeedsNodeProcessor extends FeedsPr
    -    if ($this->config['update_existing'] != FEEDS_UPDATE_EXISTING) {
    +    if (($this->config['update_existing'] != FEEDS_UPDATE_EXISTING) &&
    +        ($this->config['update_existing'] != FEEDS_SKIP_NEW)){
           $node->uid = $this->config['author'];
    

    The danger of mixing insert and update behavior is that child classes that derive from the base class might need to be changed as well. This node processor is a good example. But who tells us that there aren't any other child classes in projects outside the feeds project, that would need the same change? (e.g. the commerce product processor)

    To solve this dilemma, we should go for a separate config value that defines how to handle existing entities. Thus defining 2 new constants:

    define('FEEDS_INSERT_NEW, 0);
    define('FEEDS_SKIP_NEW, 1);
    

    and storing that in a config value:
    $this->config['insert_new']

    Whether we then write hook_update_n () or use !empty($this->config['insert_new']) to skip inserting new records is a choice. The latter will give us a shorter and easier patch, the former will be better in the long run.

  2. +++ feeds.a/plugins/FeedsProcessor.inc	2013-01-18 18:48:56.078480637 -0500
    @@ -571,7 +578,8 @@ abstract class FeedsProcessor extends Fe
    +        FEEDS_SKIP_NEW => t('Update existing nodes only (Do not create new nodes)', $tokens),
           ),
    

    Change nodes to @entities (and rewrite to phrase the action only once, but this is also a symptom of mixing insert and update behavior)

I will try to post a patch shortly (without a hook_update_n()).

fietserwin’s picture

Status: Needs work » Needs review
FileSize
2.2 KB
PASSED: [[SimpleTest]]: [MySQL] 5,105 pass(es). View

As the constructor of a feeds class calls defaultConfig(), the whole thing about needing a hook_update_n or not was not needed at all.

Attached a patch that works locally for me. Not sure though, if some tests may need to be adapted.

matsjacobsson’s picture

I also got this to work nicely with a.ross suggestion with rules! Thanks!!

For the condition value I used: commerce-product:sku

And the actions dataselctor: commerce-product

drupalninja99’s picture

Patch here works well with this one: https://drupal.org/node/1539224

osopolar’s picture

FileSize
5.68 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch feeds-d6-only-update-existing-items-1286298-28.patch. Unable to apply patch. See the log in the details link for more information. View

Backport for D6 attached.

In D7 FEEDS_SKIP_NEW is defined as 1 and FEEDS_INSERT_NEW as 0. This could be misleading in case of this code: $this->config['insert_new'] == FEEDS_SKIP_NEW;. Somebody could think when $this->config['insert_new'] is 1 (TRUE?) it means insert new node, but it actually stands for skip new node.

Therefore I defined the constants in D6 conversely with FEEDS_INSERT_NEW as 1 and FEEDS_SKIP_NEW as 0. Maybe this should be fixed in D7 version.

Status: Needs review » Needs work

The last submitted patch, 28: feeds-d6-only-update-existing-items-1286298-28.patch, failed testing.

fietserwin’s picture

If you use constants to get some kind of enum type, you should not bother about their actual value. In languages that provide an enum type you don't even (have to) declare their values.

Attach patches for other main versions than the issues version with a do-not-test or with a different extension.

osopolar’s picture

Status: Needs work » Needs review

@fietserwin: Thanks for the do-not-test hint.

memcinto’s picture

Just going to mention here that on a D7 site where I had the feeds-nocreate-1286298-7.patch installed, when I used a Feeds Importer to update existing nodes, using the "update only don't create new" option, it deleted the images in a image field attached to those nodes. (The image field was not involved in the import in any way.) It didn't just break the link between the node and the file, it deleted the image files off the server. Using the importer with the setting changed to "Update existing nodes" did not delete any files.

Drupali’s picture

Patch in #25 works great for me. Good job.

Darren Oh’s picture

Status: Needs review » Reviewed & tested by the community
Drupali’s picture

Status: Reviewed & tested by the community » Needs work

I am getting the Notices while using this patch with entities. See enclosed image to see the errors.

Notice: Undefined index: insert_new in FeedsProcessor->configForm() (line 654 of /Applications/MAMP/htdocs/FGM/sites/all/modules/contrib/feeds/plugins/FeedsProcessor.inc).
Notice: Undefined index: insert_new in FeedsProcessor->configForm() (line 654 of /Applications/MAMP/htdocs/FGM/sites/all/modules/contrib/feeds/plugins/FeedsProcessor.inc).

tengoku’s picture

patch #25 works for me too! thanks!

even with the commerce feeds product import

caspervoogt’s picture

patch #25 didn't work for me with Commerce Feeds - but the rules approach from #13 works for me.

sedam’s picture

Path #25 didn't work for me. I need to update a product entity without create new product that it exits (only update). Any suggestions? what am i doing wrong? FYI, I'm using lastest Feeds version on D7. thank you so much!

sedam’s picture

Definitely I was doing wrong mapping fields. I marked a SKU as an unique field and after applying the patch #25, produts are updating.

brandonc503’s picture

*deleted as i cant remove account

brandonc503’s picture

*deleted as i cant remove account

IckZ’s picture

subscribe

IckZ’s picture

after some testing I can say, that patch #25 is working for me :) thx a lot!

Aambro’s picture

#13 with rules worked for me. And no need to patch.

fietserwin’s picture

Status: Needs work » Needs review
FileSize
2.8 KB
FAILED: [[SimpleTest]]: [MySQL] 6,906 pass(es), 16 fail(s), and 18 exception(s). View

Reroll of #25.

This patch answers the concerns that were raised:
- #28: switched the definitions to be more in line with the values for the constants regarding update behavior (especially FEEDS_SKIP_EXISTING).
- #32: this refers to the patch in #10, so can be ignored.
- #35: This warnings suggests that there is a child class that does not call the parent implementation of configDefaults(). That turns out to be FeedsEntityProcessor, so I also changed the configDefaults() method for that class. Thanks for reporting: solved in this patch.
- #37: Commerce feeds is another module and I guess that if that does not work, it also overrides methods without calling the parent implementation (like FeedsEntityProcessor did). if Commerce feeds depends on Feeds, it thus might have to adapt code if this patch gets in.
- #38: solved as of #39, so not a problem of this patch.

Status: Needs review » Needs work

The last submitted patch, 45: don_t_create_new_items-1286298-45.patch, failed testing.

fietserwin’s picture

Status: Needs work » Needs review
FileSize
2.69 KB
PASSED: [[SimpleTest]]: [MySQL] 6,996 pass(es). View

Calling parent::configDefaults() in FeedsEntityProcessor lead to an infinite recursion, oops. This patch ran the tests fine locally.

MegaChriz’s picture

Issue tags: +Needs tests

#47 looks promising. This needs tests.

ben van den broeck’s picture

I was trying #47 but it didn't stop creating products in my drupal commerce on importing a csv.

fietserwin’s picture

#49: what processor are you using? I wouldn't be surprise if that is the one of the commerce feeds module and that was already reported in #37.

ben van den broeck’s picture

It is indeed with the commerce processor, I thought this was working with the commerce processor before, but probably i'm wrong. Now i use the rules solution offered in #13.

brandonc503’s picture

*deleted as i cant remove account

charginghawk’s picture

FYI, Feed's entity processor has been moved to its own module: https://www.drupal.org/project/feeds_entity_processor

So you may need to apply this patch there.

MegaChriz’s picture

Status: Needs review » Needs work
Issue tags: +Needs reroll

The generic entity processor indeed moved to an other module, but you no longer have to patch that in this issue. The entity processor was only patched here because it didn't automatically inherit the "insert_new" setting.

The patch needs a reroll and the changes to FeedsEntityProcessor need to be left out in this patch.

The patch also needs an automated test.

charginghawk’s picture

FileSize
2.18 KB
PASSED: [[SimpleTest]]: [MySQL] 6,975 pass(es). View

Rerolling with FeedsEntityProcessor.inc changes stripped out.

charginghawk’s picture

Status: Needs work » Needs review
MegaChriz’s picture

Issue tags: -Needs tests, -Needs reroll
FileSize
5.32 KB
PASSED: [[SimpleTest]]: [MySQL] 7,182 pass(es). View
3.14 KB

And here is a patch with an automated test.

The following is tested with a source file containing 10 items:

  • Skip new nodes, skip existing nodes, no nodes on the system: no nodes inserted nor updated.
  • Skip new nodes, skip existing nodes, two nodes on the system: no nodes inserted nor updated.
  • Skip new nodes, update existing nodes, two nodes on the system: 2 nodes updated.
  • Insert new nodes, update existing nodes, two nodes on the system: 8 nodes inserted.

The probably only weird thing is that Feeds reports "There are no new nodes" even when you don't intent to import new nodes. Maybe this message should be "There are no nodes to update" when the setting "Insert new nodes" is set to "Do not insert new nodes" and when the setting "Update existing nodes" is set to "Update existing nodes". Thoughts?

joelpittet’s picture

The messaging is minor point but I on board with a +1 for "There are no nodes to update."

brandonc503’s picture

i ran #57 patch against the
7.x-2.x-dev 2015-Jun-18
and
Checking patch plugins/FeedsProcessor.inc...
Checking patch tests/feeds_processor_node.test...
Applied patch plugins/FeedsProcessor.inc cleanly.
Applied patch tests/feeds_processor_node.test cleanly.

fyi

this was my first patch. so this was my first updating module for security and patching again for first time

fietserwin’s picture

I also updated a site to the dev version, installed the module with the now separate entity processor and applied the latest patch: applies and works.

Regarding the message: the condition is actually that nothing happened: no updates, no changes, no unpublishing, no deletions and no failures. so the message should be somethinkg like "No action was necessary on processing this feed" or "No action taken on processing this feed" or "...". Can some native English speaker come up with a nice sentence?

joelpittet’s picture

No action taken on processing this feed

I believe this is slightly better:

No actions taken while processing this feed

Hope that helps, Cheers

maxplus’s picture

Hi,
just tested it with commerce feeds because I did not want to create new products during a stock update.
Worked great at first glance

Thanks!

a.milkovsky’s picture

Status: Needs review » Reviewed & tested by the community

#57 works for me as well. Thank you! I think it is time for RTBC

  • twistor committed 3383fd1 on 7.x-2.x authored by fietserwin
    Issue #1286298 by fietserwin, Triskelion, MegaChriz, charginghawk,...
twistor’s picture

Status: Reviewed & tested by the community » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

madelyncruz’s picture

I've tested #13 and it works well. Thank you a.ross

kevster’s picture

Having this problem and cant apply against latest dev as getting hunk issues - cant apply 2 out of 5 - I guess as its moved on now. Also have issue re #23 @marktheshark re my commerce prods having the image path wiped out despite the field not being in my importer.

Any chance of an update on the patch please?

MegaChriz’s picture

@kevster
The patch in this issue has already been committed. There is no need to apply it to the latest dev. The issue of images being wiped out is a different issue. Maybe there is an existing issue for images being wiped out (I couldn't find one so far)? If you like to have this issue addressed, search for an existing issue first. If you can't find one, then try to reproduce the issue on a clean install. Finally, open a new issue with the steps to reproduce the problem.