Failed to rebuild registry on PDO Exception: duplicate entry [#1324000]

This is an issue imported from feeds XPath parser #1265564: PDOException due to class auto-loading registry when running a cache clear, but since the registry experts are here I'm hoping you have some insight.

Any attempt at clearing the cache with the above module fails due to a PDO Exception : Duplicate entry 'FeedsXPathParserHTML-class' ... into {registry}

I attempted to use the registry rebuild drush command to rebuild the registry, and it raised the same error:

Doing registry_rebuild() in DRUSH_BOOTSTRAP_DRUPAL_DATABASE
Bootstrapping to DRUPAL_BOOTSTRAP_FULL
Doing registry_rebuild() in DRUPAL_BOOTSTRAP_FULL
WD registry: PDOException: SQLSTATE[23000]: Integrity constraint     [error]
violation: 1062 Duplicate entry 'FeedsXPathParserHTML-class' for key
'PRIMARY': INSERT INTO {registry} (name, type, filename, module,
...
in _registry_parse_file() (line 179 of
/var/aegir/platforms/dbn-drupal-7.8/includes/registry.inc).

Also of note is that this error appears thrice when running the registry rebuild, twice in english and once in french (this is a localized site, french only).

Comments

Comment #1

rfay

English

Palisade, CO, USA

CreditAttribution: rfay commented 27 October 2011 at 19:25

I'm going to suspect that you actually have the class FeedsXPathParserHTML declared in two places. Can you grep for that?

grep -rl FeedsXpathParserHTML [siteroot]

will show you all files that mention that class.

Comment #2

mathieuhelie CreditAttribution: mathieuhelie commented 27 October 2011 at 19:36

Doesn't seem like it is

grep -rl FeedsXPathParserHTML .
./modules/feeds_xpathparser/feeds_xpathparser.info
./modules/feeds_xpathparser/.git/index
./modules/feeds_xpathparser/feeds_xpathparser.module
./modules/feeds_xpathparser/FeedsXPathParserHTML.inc
./modules/feeds_xpathparser/FeedsXPathParserBase.inc
./modules/feeds_xpathparser/tests/feeds_xpathparser_parser_html.test


grep -r FeedsXPathParserHTML .
./modules/feeds_xpathparser/feeds_xpathparser.info:files[] = FeedsXPathParserHTML.inc
Binary file ./modules/feeds_xpathparser/.git/index matches
./modules/feeds_xpathparser/feeds_xpathparser.module:    'FeedsXPathParserHTML' => array(
./modules/feeds_xpathparser/feeds_xpathparser.module:        'class'  => 'FeedsXPathParserHTML',
./modules/feeds_xpathparser/feeds_xpathparser.module:        'file'   => 'FeedsXPathParserHTML.inc',
./modules/feeds_xpathparser/FeedsXPathParserHTML.inc: * Provides the FeedsXPathParserHTML class.
./modules/feeds_xpathparser/FeedsXPathParserHTML.inc:class FeedsXPathParserHTML extends FeedsXPathParserBase {
./modules/feeds_xpathparser/FeedsXPathParserBase.inc: * Provides the base class for FeedsXPathParserHTML and FeedsXPathParserXML.
./modules/feeds_xpathparser/tests/feeds_xpathparser_parser_html.test:    $this->setPlugin('xpath', 'FeedsXPathParserHTML');
./modules/feeds_xpathparser/tests/feeds_xpathparser_parser_html.test:    $importer_url = $this->feeds_base . '/xpath/settings/FeedsXPathParserHTML';
./modules/feeds_xpathparser/tests/feeds_xpathparser_parser_html.test:      'feeds[FeedsXPathParserHTML][xpath][exp][debug][xpathparser:0]' => TRUE,
./modules/feeds_xpathparser/tests/feeds_xpathparser_parser_html.test:      'feeds[FeedsXPathParserHTML][xpath][exp][debug][xpathparser:0]' => FALSE,
./modules/feeds_xpathparser/tests/feeds_xpathparser_parser_html.test:      'feeds[FeedsXPathParserHTML][xpath][context]' => '/foo',
./modules/feeds_xpathparser/tests/feeds_xpathparser_parser_html.test:      'feeds[FeedsXPathParserHTML][xpath][sources][xpathparser:0]' => 'bar',
./modules/feeds_xpathparser/tests/feeds_xpathparser_parser_html.test:      'feeds[FeedsXPathParserHTML][xpath][sources][xpathparser:1]' => 'baz',
./modules/feeds_xpathparser/tests/feeds_xpathparser_parser_html.test:      'feeds[FeedsXPathParserHTML][xpath][context]' => 'sdf asf',
./modules/feeds_xpathparser/tests/feeds_xpathparser_parser_html.test:      'feeds[FeedsXPathParserHTML][xpath][sources][xpathparser:0]' => 'asdf[sadfas asdf]',
./modules/feeds_xpathparser/tests/feeds_xpathparser_parser_html.test:      'feeds[FeedsXPathParserHTML][xpath][context]' => '//tr[starts-with(@class, "odd ") or starts-with(@class, "even ")]',
./modules/feeds_xpathparser/tests/feeds_xpathparser_parser_html.test:      'feeds[FeedsXPathParserHTML][xpath][sources][xpathparser:0]' => 'td[1]/a',
./modules/feeds_xpathparser/tests/feeds_xpathparser_parser_html.test:      'feeds[FeedsXPathParserHTML][xpath][sources][xpathparser:1]' => 'td[1]/a/@href',
./modules/feeds_xpathparser/tests/feeds_xpathparser_parser_html.test:    $this->assertFieldByName('feeds[FeedsXPathParserHTML][xpath][context]', '//tr');
./modules/feeds_xpathparser/tests/feeds_xpathparser_parser_html.test:    $this->assertFieldByName('feeds[FeedsXPathParserHTML][xpath][sources][xpathparser:0]', 'booya');
./modules/feeds_xpathparser/tests/feeds_xpathparser_parser_html.test:    $this->assertFieldByName('feeds[FeedsXPathParserHTML][xpath][sources][xpathparser:1]', 'boyz');

I initially had the module in both sites/all/modules and sites/dev.example.com/modules, but I removed it from sites/all/modules before running the registry rebuild.

Comment #3

mathieuhelie CreditAttribution: mathieuhelie commented 27 October 2011 at 19:49

I found a new clue.

This is a dev site I aegir-cloned from a staging site (the cloning task reported a failure due to this registry rebuild error, but kept the site up). When I clear the cache on the staging site, all goes well. When I clear the cache on the dev site, it reports the registry error on the path of the staging site.

    [:db_insert_placeholder_0] => FeedsXPathParserHTML
    [:db_insert_placeholder_1] => class
    [:db_insert_placeholder_2] => sites/example.com/modules/feeds_xpathparser/FeedsXPathParserHTML.inc
    [:db_insert_placeholder_3] => feeds_xpathparser
    [:db_insert_placeholder_4] => 0

When it should read sites/dev.example.com/modules/feeds_xpathparser/FeedsXPathParserHTML.inc.

Comment #4

rfay

English

Palisade, CO, USA

CreditAttribution: rfay commented 27 October 2011 at 20:04

You can edit the registry table of course, but it's odd that this is not being solved by registry_rebuild here. I suspect we may be missing a step (like perhaps emptying the entire table before continuing).

Comment #5

mathieuhelie CreditAttribution: mathieuhelie commented 27 October 2011 at 20:12

It's odd that this problem is arising specifically for the XPathParserHTML class.

Comment #6

rfay

English

Palisade, CO, USA

CreditAttribution: rfay commented 27 October 2011 at 20:20

If it's possible for you to provide me with a database (and probably a codebase) I'll try to study this problem. You can dropbox it to me at randy at randyfay dot com.

Comment #7

mathieuhelie CreditAttribution: mathieuhelie commented 27 October 2011 at 20:49

I forced a resolution by connecting into mysql and searching through all tables for references to my staging site path. The cache table was full of them, for all sorts of modules, not only XPath Parser. I truncated the cache table, then drush cache-clear went through just fine.

This is likely a problem with aegir site cloning.

Comment #8

q0rban CreditAttribution: q0rban commented 28 October 2011 at 17:16

We should be able to simulate this by simply setting up a multisite that has modules with classes in sites/example.com/modules and then changing the path to sites/dev.example.com/modules.

Comment #9

juampynr CreditAttribution: juampynr commented 14 December 2011 at 22:07

The patch for Feeds module in the above issue fixes the error.

Comment #10

rfay

English

Palisade, CO, USA

CreditAttribution: rfay commented 14 December 2011 at 22:09

@juampy THANKS!

Were you able to recreate the problem described in this issue, and therefore able to test and mark this fixed?

Comment #11

juampynr CreditAttribution: juampynr commented 14 December 2011 at 22:36

rfay, yes, it happened to me consistently when I cloned a whole site (code and db) and attempted to clean the cache. The patch has been committed by the maintainer in Feeds-3.x-dev.

However, I have to admit that I still do not know exactly what is going on with the registry and I have found similar issues in other projects that seem to happen for a different reason. Check out the following:

#1024560: Enabling throws DB errors
#1224498: PDOException
#1283208: PDOException: SQLSTATE[23000]

Comment #12

rfay

English

Palisade, CO, USA

CreditAttribution: rfay commented 14 December 2011 at 22:54

Status:

Active

» Fixed

I'm going to call this fixed then, unless somebody points out otherwise.

Comment #13

juampynr CreditAttribution: juampynr commented 15 December 2011 at 11:49

I investigated deeper in Ctools plugin management and found the real issue. Please see http://drupal.org/node/1371700#comment-5367382 as it needs a review.

Comment #14

rfay

English

Palisade, CO, USA

CreditAttribution: rfay commented 15 December 2011 at 15:49

Status:

Fixed

» Active

Reopening then. Thanks for your work on this, @juampy

Comment #15

rfay

English

Palisade, CO, USA

CreditAttribution: rfay commented 15 December 2011 at 15:50

@juampy I would really be interested to know if #1307864: Backport to Drupal 6 (or make generic drush command) solves the problem, if you're able to recreate it now.

Comment #16

juampynr CreditAttribution: juampynr commented 15 December 2011 at 16:40

@rfay, I have read the patch given at http://drupal.org/node/1307864#comment-5290598 and I think it refers to a different issue. What it is exactly what you want me to test?

The error that is described at this issue (not the one referenced but the one described in the present page) is related to a bug in CTools recovering a cached entry for the "registry" table that is wrong. Once the patch in CTools is applied, the cache can be cleared without errors.

Comment #17

juampynr CreditAttribution: juampynr commented 15 December 2011 at 16:41

Actually, it would be great if @mathieuhelie tries the patch from http://drupal.org/node/1371700#comment-5367382 and verifies that it solves the issue.

Comment #18

rfay

English

Palisade, CO, USA

CreditAttribution: rfay commented 15 December 2011 at 16:44

@juampy the current dev actually invokes hook_init(), which causes a few failures with registry rebuild. That issue removes that. I know it improves a number of scenarios. So if you have a database that demonstrates the problem it would be cool to know if that gives us a workaround.

Comment #19

juampynr CreditAttribution: juampynr commented 15 December 2011 at 16:49

@rfay, you can easily recreate that database. Here is how (taken from http://drupal.org/node/1347894#comment-5366800):

Download Feeds module within a named site at sites directory (ie. sites/drupal7.localhost/modules/contrib/feeds).
Enable Feeds module.
Create a feed.
Clone the whole site (code and db) to another box.
Clean the cache in the new site > The error occurs.

If you want, send me your contact details through my Contact form and I can send you through Dropbox a full drupal installation and db dump that you just need to configure to recreate it yourself.

Comment #20

rfay

English

Palisade, CO, USA

CreditAttribution: rfay commented 15 December 2011 at 16:52

Cool - If you'd dropbox it to randy@randyfay.com that would be great. Greater still if you could try it out with that patch :-) Thanks so much.

Comment #21

rfay

English

Palisade, CO, USA

CreditAttribution: rfay commented 23 December 2011 at 01:01

@juampy thank you very much for the setup; it helps a lot. I can easily recreate the problem.

I think I'll add an non-API cache clear (truncate {cache}) into drush rr, that will at least help the situation.

Oh - your SETUP.txt (thanks) could be improved by mentioning that this is explicitly multisite. I have no idea why that matters, but you should mention that you have it set up that way.

Comment #22

rfay

English

Palisade, CO, USA

CreditAttribution: rfay commented 23 December 2011 at 01:15

Status:

Active

» Fixed

I believe #1307864: Backport to Drupal 6 (or make generic drush command) fixes this; Would appreciate any feedback. The core issues remain, of course.

Comment #23

juampynr CreditAttribution: juampynr commented 23 December 2011 at 12:54

Hi @rfay, you are welcome. I have updated the SETUP.txt with your suggestions.

The multi-site configuration matters because paths of registered clases and functions are saved into registry table. Therefore, the integrity error arises when the same class is registered for one of the sites (done by Drupal core) and then from another site (because it was in cache and CTools grabs it from there on hook_registry_files_alter).

In environments where everything is at sites/all, these errors wont happen.