hi, I am currently in the process of optimizing my site for speed and memmory usage. Because of this I am experimenting with APC but the size of the {cache} table made me wonder what uses so much space. Part of the Answer was Feeds.
I want to push {cache} to APC but as the feeds caches are only needed for cron runs they should not be in APC. The most obvious way would be to put the feeds caches to a seperate cache bin and exclude this bin from APC.
For this I had to modify the feeds module to use that cache bin.
The attached patch is against the current head and adds a own cache bin for feeds.
I would be interested what you think about this.

thanks
Clemens

CommentFileSizeAuthor
#7 feeds-cache-table-1449464-7.patch3.42 KBtwistor
#5 feeds-cache-table-1449464-5.patch4 KBklausi
feeds_own_cache_bin.diff2.95 KBcbergmann
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

5n00py’s picture

5n00py CreditAttribution: 5n00py commented

I also have problem with cache table and feeds.
I think this issue is related to next one:
#1360910: Don't cache_set full feed result in http_request.inc (or make optional)
One of the possible solutions is - store only headers in feeds_http_download_ instead of full data.

vinoth.3v’s picture

vinoth.3v CreditAttribution: vinoth.3v commented
Status: Needs review » Active

APC always gets 100% fragmentation because of this Feeds.
It will be good if feeds have the own cache bin.

vinoth.3v’s picture

vinoth.3v CreditAttribution: vinoth.3v commented
Priority: Normal » Major
twistor’s picture

twistor CreditAttribution: twistor commented
Assigned: Unassigned » twistor
Priority: Major » Normal

This is 2 parts. One, Feeds should not be caching downloads in the db. There's an issue for this somewhere. Two, Feeds should have a custom cache table since there could be a large number of entries.

klausi’s picture

klausi
he/him||they/them
Primary language German
Location 🇦🇹 Vienna
CreditAttribution: klausi commented
Assigned: twistor » Unassigned
Issue summary: View changes
Status: Active » Needs review
FileSize
feeds-cache-table-1449464-5.patch4 KB

The other issue is #2032333: [Meta] Store remote files in the file system instead of in the database.

We can still start with a separate bin here, which is already an improvement, so that we can take Feeds out of Memcache for example.

twistor’s picture

twistor CreditAttribution: twistor as a volunteer commented

Awesome, this has been on my mind again lately.

+++ b/feeds.module
@@ -892,7 +900,7 @@ function feeds_get_importer_id($content_type) {
 function _feeds_importer_digest() {
   $importers = &drupal_static(__FUNCTION__);
   if ($importers === NULL) {
-    if ($cache = cache_get(__FUNCTION__)) {
+    if ($cache = cache_get(__FUNCTION__, 'cache_feeds')) {

Does it make sense to move the importer digest? It's probably not a big deal either way, but this does get called on hook_node_update.

twistor’s picture

twistor CreditAttribution: twistor as a volunteer commented
FileSize
feeds-cache-table-1449464-7.patch3.42 KB

Let's go ahead and make this a dedicated HTTP cache. While we're at it, change the hashing algorithm to make things easier for people who are concerned about that.

  • twistor committed 0f9ba16 on 7.x-2.x authored by klausi 
    Issue #1449464 by klausi, cbergmann: Own cache Bin for feeds
twistor’s picture

twistor CreditAttribution: twistor as a volunteer commented
Status: Needs review » Fixed

Changed on commit:
Used sha256 instead of sha512 to be a bit nicer to our indexes.
Fixed comment on feeds_update_7212().

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.