I've got a site with user profiles provided by profile2. Whenever I save a user profile, it takes a really long time (~20 seconds). I've traced this through to apachesolr_attachments_entity_update() being called. This calls apachesolr_attachments_clean_index_table(), which then runs a delete query against apachesolr_index_entities_file for every single entity type in the system, which is what's actually taking the time.

I've managed to improve the performance by adding an index on parent_entity_type in my DB, but it seems really odd to me that it needs to run all these queries when saving an unrelated entity anyway. (Hence I'd add a patch for the index, but my gut feeling is more needs to be done here that that).

Comments

OliverColeman’s picture

Version: 7.x-1.x-dev » 7.x-1.3
Priority: Normal » Major
Issue summary: View changes
Status: Needs work » Active

"Me, too". Our apachesolr_index_entities_file table has 182488 records, it takes about 6 seconds to run each delete query, and we have a lot of entity types, so creating/updating and deleting nodes takes a long time. Performing these table cleaning operations does seem unnecessary upon CRUD operations on entities. Can they be moved to a cron job?

I've marked this bug as Active with Priority Major as it makes the module unusable, at least for large numbers of attachments. Apologies if this is poor form, feel free to revert it back to normal priority if so.

Reuben Unruh’s picture

There's a patch to move the table clean to a cron job in this issue.

mdupont’s picture

Given apachesolr_attachments_clean_index_table() doesn't do any operation that is specific to a given entity, it doesn't make sense to run it every time an entity is created, updated or deleted. This type of maintenance operation should rather be done at cron time.

Attached a patch that moves calls to apachesolr_attachments_clean_index_table() to a hook_cron(), and performs only 1 delete query instead of multiple ones. As an additional benefit, only 1 instance of the query will be run, eliminating the risk of deadlocks.

mdupont’s picture

Status: Active » Needs review
mdupont’s picture

Version: 7.x-1.3 » 7.x-1.x-dev
milesw’s picture

The slowness is due to apachesolr_index_entities_file missing the appropriate indexes.

Whether or not apachesolr_attachments_clean_index_table() can be moved to cron is unclear, but the maintainers are already discussing that in the other issue.

Here is a patch to add the indexes and resolve the performance issue.

claudiu.cristea’s picture

StatusFileSize
new1.41 KB
new811 bytes

@milesw, thank you for the patch. I was looking for it. Anyway, because some sysadmins have already added indexes manually to quick save their servers it's good to add a check.

claudiu.cristea’s picture

Title: Module slows down entity saves for entities it has nothing to do with. » Performance! Add missed indexes to {apachesolr_index_entities_file} table
Issue tags: +Performance
nironan’s picture

Status: Needs review » Reviewed & tested by the community

Patch does the job!

andras_szilagyi’s picture

#8 works as intended

  • 76bb426 committed on 7.x-1.x
    Revert #2017705
    
  • claudiu.cristea authored b59aa8f on 7.x-1.x
    Issue #2017705 by claudiu.cristea, mdupont, milesw: Performance! Add...

  • 6a450ed committed on 7.x-1.x
    Revert #2017705
    
janusman’s picture

Noting I tried adding some of these but then started getting some warnings upon creating some new nodes with attached files. Not sure exactly what it was, so for now I reverted the changes. Still needs some work/testing IMO.

janusman’s picture

Status: Reviewed & tested by the community » Needs work