We've been gradually moving from Drupal 6 and Ubercart to Drupal 7 and Commerce. The site I'm currently in charge of is an osCommerce site we're converting to Drupal 7 and Commerce. The client has manually recreated each of the 50 or so discounts they have in osCommerce in the Commerce Discount module and this is were our problem became apparent.

We now have about 60 or so price calculation rules being run in succession for any given price and it takes nearly a second to generate the price of a single product. A page with 100 products takes over a minute to load. The fact that Commerce Discount generates 1 rule per discount instead of 1 global rule is an issue on its own and a big part of this issue, but I will post that in the correct channel.

If we enable pre-calculation it seems the only way to generate prices is to manually run the batch job. Which obviously will not work well with timed discounts etc.

The hack I'm currently exploring is to enable pre-calculation and then hack commerce_product_calculate_sell_price() so if it does not find a pre-calculated price it stores the result of the current calculation returned from rules in the pre-calculation table. Then I will make sure pre-calculated prices are updated whenever a product is updated. So it will function like a cache_get/set wrapped around the price calculation.

We just have to get this one fixed in a hurry because this site is close to launching, but we would like to hear your thoughts on this.

The discounts on this site are only term/sku discounts so it should work fine. But we have a lot of clients using Ubercart who have user/role specific discounts as well. A more flexible solution would be needed if we ever were to convert those to Drupal 7 and Commerce.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

TwiiK’s picture

I do not understand how the pre-calculation is supposed to work either. I tested it on a clean Kickstart installation and it created 1 row per rule in the database table, and why does it ignore conditions? How is it supposed to work with discounts if it ignores conditions? In my testing on the Kickstart installation I wasn't able to get it to work at all.

And it seems impossible to use pre-calculated prices unless you manually ask for them because there is no way of calling commerce_product_calculate_sell_price() with $precalc = TRUE unless you manually do so or hack the function itself.

This is what I ended up doing. First the hack:

function commerce_product_calculate_sell_price($product, $precalc = FALSE) {
  // Hack to call my own function instead.
  return MYMODULE_commerce_product_calculate_sell_price($product);
}

Then my implementation of the pre-calculation:

function MYMODULE_commerce_product_calculate_sell_price($product) {
  // First create a pseudo product line item that we will pass to Rules.
  $line_item = commerce_product_line_item_new($product);

  // Allow modules to prepare this as necessary.
  drupal_alter('commerce_product_calculate_sell_price_line_item', $line_item);

  // Attempt to fetch a database stored price.
  $module_key = 'MYMODULE';

  $result = db_select('commerce_calculated_price')
    ->fields('commerce_calculated_price', array('amount', 'currency_code', 'data'))
    ->condition('module', 'commerce_product_pricing')
    ->condition('module_key', $module_key)
    ->condition('entity_type', 'commerce_product')
    ->condition('entity_id', $product->product_id)
    ->condition('field_name', 'commerce_price')
    ->execute()
    ->fetchObject();

  // If a pre-calculated price was found.
  if (!empty($result)) {
    // Wrap the line item, swap in the price, and return it.
    $wrapper = entity_metadata_wrapper('commerce_line_item', $line_item);

    $wrapper->commerce_unit_price->amount = $result->amount;
    $wrapper->commerce_unit_price->currency_code = $result->currency_code;

    // Unserialize the saved prices data array and initialize to an empty
    // array if the column was empty.
    $result->data = unserialize($result->data);
    $wrapper->commerce_unit_price->data = !empty($result->data) ? $result->data : array();

    return $wrapper->commerce_unit_price->value();
  }

  // Pass the line item to Rules.
  rules_invoke_event('commerce_product_calculate_sell_price', $line_item);

  // If we found no pre-calculated price save the newly calculated price in the database.
  if (empty($result)) {
    MYMODULE_save_product_price($line_item, $module_key);
  }

  return entity_metadata_wrapper('commerce_line_item', $line_item)->commerce_unit_price->value();
}

function MYMODULE_save_product_price($line_item, $module_key) {
  $product_id = $line_item->commerce_product[LANGUAGE_NONE][0]['product_id'];

  // Build the record of the pre-calculated price and write it.
  $wrapper = entity_metadata_wrapper('commerce_line_item', $line_item);

  if (!is_null($wrapper->commerce_unit_price->value())) {
    $record = array(
      'module' => 'commerce_product_pricing',
      'module_key' => $module_key,
      'entity_type' => 'commerce_product',
      'entity_id' => $product_id,
      'field_name' => 'commerce_price',
      'language' => !empty($product->language) ? $product->language : LANGUAGE_NONE,
      'delta' => 0,
      'amount' => round($wrapper->commerce_unit_price->amount->value()),
      'currency_code' => $wrapper->commerce_unit_price->currency_code->value(),
      'data' => $wrapper->commerce_unit_price->data->value(),
      'created' => time(),
    );

    // Save the price.
    drupal_write_record('commerce_calculated_price', $record);
  }
}

It's the exact same function except it always looks for a pre-calculated price and if it doesn't find one it saves the newly calculated price in the pre-calculation table. The save part of the code is just ripped from the batch operation.

So as you can see it's very much like wrapping the price calculation in cache_get() and cache_set(). It seems to work very well. The pages load quickly and the correct prices are shown.

Edit: I haven't added the node update part yet so it may need to be tweaked some. Also, the comment formatting seems to remove the empty lines within the php tags so the code looks messy. Not my fault. :p

rszrama’s picture

I'm skeptical of the performance cost even with 50 or 60 Rules on the event, given that Eurocentres.com had hundreds and experiences no such lengthy delays in price calculation. Any insight into why those are taking so long? Are they running a bunch of extra queries, perhaps? May be more related to the loading of all those discount entities, for example, in which case switching from loading them one at a time to loading them all at once would significantly reduce the calculation time.

rszrama’s picture

Oh, but I do agree that pre-calculation isn't really what you need. I might even say the same thing about Commerce Discount. We just recently tried to use it on a project and found it insufficient. dpolant is currently working on improving the module.

rszrama’s picture

Re: pre-calculation and its interaction with Commerce Discount, I don't think the two will work together since Commerce Discount essentially creates a parallel price calculation step instead of interacting with the existing event to handle product price manipulations (specifically in the event of order % discounts).

TwiiK’s picture

I will investigate further, but that will take some time as I'm quite busy. :)

The root of the problem, as you say, is Commerce Discount in either case. But we're commited to that module at the moment. We've hacked it a lot to make it fit our needs and we will evaluate it further down the line to see if we are to replace it or attempt to patch it.

TwiiK’s picture

Things move slow so close to christmas, but I've tested a bit more. :p

We have a few custom rules conditions with node loads etc. for evaluating fields on the node the product belongs to. I thought these could be the culprit so I tried commenting out these, but saw no speed increase at all.

A page with 10 products takes 6-7 seconds to load. If I comment out the rules part of the commerce_product_calculate_sell_price() function and just return the price the same pages takes 600 milliseconds to load.

A kind colleague hooked me up with XHProf and I was able to pinpoint the main source of the problem.

It's the first line in this condition in the commerce_discount_date module:

/**
 * Rules condition: Check discount can be applied.
 */
function commerce_discount_date_condition($discount_name) {
  $wrapper = entity_metadata_wrapper('commerce_discount', $discount_name);
  $time = time();
  return $time >= $wrapper->commerce_discount_date->value->value() && $time <= $wrapper->commerce_discount_date->value2->value();
}

So I guess I'll take it up with the maintainers of that module as all the issues here are caused by the discount module. :)

Edit: I just changed the function that called this one to pass along the wrapper it had already loaded instead of the discount name so commerce_discount_date_condition() wouldn't have to load it again and shaved off almost the entire 6-7 seconds from the page load time. So that one line of code was solely to blame.

bojanz’s picture

Title: Price calculation is too slow and pre-calculation is too rigid » commerce_discount_date performance issues
Project: Commerce Core » Commerce Discount
Version: 7.x-1.8 » 7.x-1.x-dev
Component: Product pricing » Code
Category: Support request » Bug report
Priority: Normal » Major

Let's move this to Commerce Discount then.

agoradesign’s picture

I might even say the same thing about Commerce Discount. We just recently tried to use it on a project and found it insufficient.

@rszrama: what alternative would you recommend in order to get maintainable discounts - imho requiring to create rules manually is not a maintainable solution, unless you're a Drupal developer. But customers normally don't are developers. It should be easy as creating/editing nodes, etc

rszrama’s picture

I wasn't saying Commerce Discount shouldn't be used, just that it wasn't necessarily right for the original poster. I'd recommend it as the best opportunity to simplify the user interface for merchants - we're continually improving it and just need more feedback / use to make it even better. : )

bpirkle’s picture

Has anyone made any progress on this? We have a client whose site has accumulated over 600 commerce discount rules. We're seeing slow load times on its main catalog page.

Specifically, removing the price field from the relevant display drops the main catalog page load times from ~6 seconds down to ~2 seconds.

Now, the site may not really need that many rules, so my first line of investigation is whether we can trim the rule count and give the site owner guidance about how to avoid this in the future. Nevertheless, some sites with many products may legitimately need many rules. And if it easy for a nontechnical administrator to cause rule explosion and slow down their site, it is a real problem.

If I come up with anything that seems useful to the module maintainers or the general community, I'll report it back here.

bpirkle’s picture

Followup: further investigation revealed that of the 600+ commerce discount pricing rules, only ~150 corresponded to active products. Some of the "extra" rules corresponded to disabled products, other to deleted products. Additionally, there were a few commerce discounts floating around for deleted products.

I created a speculative module to clean all this up. After executing the module and zapping the unnecessary rules, site performance improved as expected.

The module is kinda rough, but I'm sharing it in case it is helpful to anyone. Use at your own risk.
https://bitbucket.org/bpirkle/commerce-discount-rule-cleanup

Jim.M’s picture

Version: 7.x-1.x-dev » 7.x-1.0-alpha1
FileSize
731 bytes

We have encountered almost the same issue, but from the expiring perspective - we have a hundred of discounts and most of them are short-term (about just a few days). But once the discount is expired it still remains active as a rule and rule invoker check over and over again if the discount is applicable even if it has been expired for weeks. To address this issue we created a patch to the commerce_discount_date component to disable such rules at the rules rebuilding process (usually by cron).

ramotowski’s picture

Issue summary: View changes

Thank You TwiiK.
Code from the first comment #1 reduced page load time from 7 seconds to less than 1 second.
I manage Commerce site with ~5000 products, 400 discounts (percent, per product category) and calculating discounted prices without cache was a bottleneck.
Now there is only need to manually clean the calculated price table after any discount has changed.

ramotowski’s picture

Issue summary: View changes
joelpittet’s picture

Version: 7.x-1.0-alpha1 » 7.x-1.x-dev
Status: Active » Needs review

Moving this back to -dev branch because that's where the magic happens;) And needs review because there is a patch in #13

Status: Needs review » Needs work

The last submitted patch, 12: commerce_discount-rule_disabling-2134899.patch, failed testing.

The last submitted patch, 12: commerce_discount-rule_disabling-2134899.patch, failed testing.

rszrama’s picture

Priority: Major » Normal
Status: Needs work » Needs review
FileSize
1.45 KB

Simple enough patch. I cleaned it up with the use of REQUEST_TIME instead of time() and added comments. I also deferred wrapping the discount until after the check to see if the field was empty - no need to wrap just for that.

Status: Needs review » Needs work

The last submitted patch, 18: 2134899.disable_expired_discounts.patch, failed testing.

joelpittet’s picture

Thanks @rszrama, I've committed and pushed this to 1.x-dev.

  • joelpittet committed e5c2fb8 on 7.x-1.x authored by rszrama
    Issue #2134899 by rszrama, Jim.M: commerce_discount_date performance...
joelpittet’s picture

Note those failing tests are not caused by the patch. I've been trying to resolve them since DrupalCon LA

Fatal error: Call to undefined function entity_load_single() 

It's as if the entity module is not downloading.

rszrama’s picture

Status: Needs work » Fixed

Ahh, interesting. Ok, was worried for a minute I broke something awful. ; )

joelpittet’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

catchsaid’s picture

Having experienced the same performance issues with the commerce_discount_date module, I have isolated the issue to the 'Check discount dates' rule condition. This condition is wasteful as it's getting the value of the date field on every run. When a date value is retrieved, it's parsed and a date object is created.

Instead when building the rule we should use data comparison core rules condition. Doing this mean we only need to parse the date once (at rule build) instead of whenever the price is calculated.

I have attached a patch which does this.

catchsaid’s picture

joelpittet’s picture

This issue is closed, could you open up a new issue with your patch?