Failure to handle DOI and BibTex pre-populate action

Hmm, I have no idea where that output of the field values could be coming from. I tried both the bibTex you posted and I also entered the DOI from the image and they both worked. See attached image displaying the entry from the DOI import, note the first item in the "New Publications" block on the left is the bibTex Cut and Paste entry.

I suppose you could try installing the -dev version (which is what I am using and tested your file on) and see if that makes any difference.

Are you running your web server on Vista?

Ron.

You may be onto something with those CCK fields, I've never tested these attached but I'll do that and see if it makes any difference.

Ron.

I'm stumped too and the only way to get to the bottom of this is to start adding some "debugging" code to see what's happening.

If you could edit the biblio_form_validate() function at line 1427 in the biblio.module file and print out the value of $op that would be a start, so after the $op = ... line add a print command as show below. Once this is done, you should try to create a node using a DOI lookup again and it should print out 'Populate using DOI' and stop. If so, that's a good sign, and you will then need to remove the first modification add another line down near 1435 to see what if any data is coming back from the DOI lookup (see second code section below)

function biblio_form_validate($form, & $form_state) {
  $op = $form['#post']['op'];
  print $op; die;               // <---------------------Debug Line

 ....

        $doi = substr($form_state['values']['doi_data'], $doi_start);
        $node_data = biblio_crossref_xml_import($doi);
        print_r($node_data);die;               // <---------------------Debug Line
        if (isset($node_data['error'])) {
         
        .....

Ok, that tells us that the DOI lookup was a success (you can see all the information is there). Now for the $64,000.00 question, why isn't it getting back to the form... A little more detective work is required... (By the way, I never asked you what version of PHP you are running, although with Ubuntu I suspect it will be fairly recent)

Could you add the following print statement at the begining of the biblio_form() function at around line 1024...

function biblio_form($node, $form_state) {
  global $user;
  print_r($form_state);         // <------------------Debug Line (no "die" statement)
  if (isset($form_state['post']['biblio_type'])) {
  ....

Now the first time you go to create a biblio node you should see something like this at the top of the form...

Array ( [storage] => [submitted] => [post] => Array ( ) ) 

after you enter the DOI and click the populate button, you should see something like this at the top of the form...

Array ( [storage] => [submitted] => [post] => Array ( [doi_data] => 10.1002/stc.227 [op] => Populate using DOI [paste_data] => [biblio_type] => -1 [changed] => [form_build_id] => form-ca717082d2bf3cde81828817842b1247 [form_token] => f716c4cedcd233ce5bf3b317feff848c [form_id] => biblio_node_form ) ) Array ( [storage] => Array ( [biblio_doi] => 1 [biblio_paste] => 1 [biblio_type] => 102 ) [submitted] => 1 [item_count] => Array ( [field_file] => 0 ) [values] => Array ( [nid] => [vid] => [uid] => 1 [created] => 1246323662 [type] => biblio [language] => en [changed] => [doi_data] => 10.1002/stc.227 [op] => Populate using DOI [doi_submit] => Populate using DOI [paste_data] => [paste_submit] => Populate using BibTex [biblio_type] => 102 [title] => A review of loading conditions for railway track structures due to train and track vertical interaction [body] => [format] => 0 [revision] => 0 [log] => [name] => [date] => [status] => 1 [promote] => 1 [sticky] => [submit] => Save [preview] => Preview [form_build_id] => form-b20e261a8af39d6cd8ba6d47cfffdfd6 [form_token] => f716c4cedcd233ce5bf3b317feff848c [form_id] => biblio_node_form [book] => Array ( [menu_name] => [mlid] => 0 [nid] => new [router_path] => node/% [has_children] => 0 [options] => Array ( ) [module] => book [original_bid] => 0 [parent_depth_limit] => 8 [plid] => -1 [weight] => 0 [bid] => 0 [pick-book] => Change book (update list of parents) ) [comment] => 0 [menu] => Array ( [mlid] => 0 [module] => menu [hidden] => 0 [has_children] => 0 [customized] => 0 [options] => Array ( ) [expanded] => 0 [parent_depth_limit] => 8 [link_title] => [parent] => navigation:0 [weight] => 0 ) [upload] => [attach] => Attach [field_test] => Array ( [0] => Array ( [value] => [_error_element] => field_test][0][value ) ) [field_file] => Array ( [0] => Array ( [fid] => 0 [list] => 1 [data] => Array ( [description] => ) [filepath] => [filename] => [filemime] => [filesize] => 0 [_weight] => 0 [filefield_upload] => Upload [filefield_remove] => Remove [upload] => ) [field_file_add_more] => Add another item ) [field_test2] => Array ( [0] => Array ( [value] => [_error_element] => field_test2][0][value ) ) [iid] => [image] => [image_title] => [biblio_secondary_title] => Structural Control and Health Monitoring [biblio_short_title] => Struct. Control Health Monit. [issn] => [biblio_issn] => 15452263 [doi] => 10.1002/stc.227 [month] => 03 [year] => 2008 [biblio_volume] => 15 [biblio_issue] => 2 [biblio_date] => 03/2008 [biblio_contributors] => Array ( [1] => Array ( [0] => Array ( [firstname] => Alex M. [lastname] => Remennikov [auth_type] => 1 [name] => Remennikov, Alex M. ) [1] => Array ( [firstname] => Sakdirat [lastname] => Kaewunruen [auth_type] => 1 [name] => Kaewunruen, Sakdirat ) ) ) [biblio_first_page] => 207 [biblio_last_page] => 234 [biblio_pages] => 207 - 234 [biblio_year] => 2008 [biblio_doi] => 10.1002/stc.227 [moderate] => ) [clicked_button] => Array ( [#type] => submit [#disabled] => [#value] => Populate using DOI [#post] => Array ( [doi_data] => 10.1002/stc.227 [op] => Populate using DOI [paste_data] => [biblio_type] => -1 [changed] => [form_build_id] => form-ca717082d2bf3cde81828817842b1247 [form_token] => f716c4cedcd233ce5bf3b317feff848c [form_id] => biblio_node_form ) [#programmed] => [#tree] => [#parents] => Array ( [0] => doi_submit ) [#array_parents] => Array ( [0] => doi_lookup [1] => doi_submit ) [#weight] => 0.001 [#processed] => [#description] => [#attributes] => Array ( ) [#required] => [#input] => 1 [#name] => op [#button_type] => submit [#executes_submit_callback] => 1 [#process] => Array ( [0] => form_expand_ahah ) [#id] => edit-doi-submit ) [rebuild] => 1 )

What other modules do you have active? Assuming that was a print of the $form_state variable, it looks like some other module is mucking about with the $form_state array.

HOLLY SMOKES!!!! Well that's an impressive list indeed, and all I can tell you is it appears that one of them is changing $form_state from an array to a stdClass Object now the question is which one... My guess is the one which added the first bit with "[want form] => 1"

If you have access to the Unix machine hosting the installation, you could try this from the Drupal installation directory...

grep -R "want form" sites/all/modules/*

replacing "sites/all/modules/*" with where ever your modules are installed of course.

Hey, good to hear!

I haven't looked at the ctools code in any detail yet, but from a cursory inspection, it would appear that to utilize it correctly, one would have to write a handler for the biblio node type. I have a great deal of respect for programming abilities of MerlinofChaos, and I suspect it's doing what is intended, but I would have to install it and use it a bit in order to make any informed comments further than that.

I guess my question to you is... Do you really need it? It would seem to be still in the beta stages so it's probably not prudent to implement it on a production site.

WRT to contract work, contact me via my contact form and we can discuss that off line.

Cheers,

Ron.

I have pretty much the same problem, however, deactivating ctools hasn't been a success for me. I run the debugs suggested in #6 and got the same results as JB - Populate using DOI and:

Array ( [0] => Array ( [biblio_type] => 102 [biblio_secondary_title] => Expert Systems with Applications [biblio_short_title] => Expert Systems with Applications [issn] => [biblio_issn] => 09574174 [month] => 05 [year] => 2005 [biblio_volume] => 28 [biblio_issue] => 4 [biblio_date] => 05/2005 [doi] => 10.1016/j.eswa.2004.12.039 [title] => A fuzzy logic-based system for assessing the level of business-to-consumer (B2C) trust in electronic commerce [biblio_contributors] => Array ( [1] => Array ( [0] => Array ( [firstname] => F [lastname] => AKHTER [auth_type] => 1 [name] => AKHTER, F ) [1] => Array ( [firstname] => D [lastname] => HOBBS [auth_type] => 1 [name] => HOBBS, D ) [2] => Array ( [firstname] => Z [lastname] => MAAMAR [auth_type] => 1 [name] => MAAMAR, Z ) ) ) [biblio_first_page] => 623 [biblio_last_page] => 628 [biblio_pages] => 623 - 628 [biblio_year] => 2005 [biblio_doi] => 10.1016/j.eswa.2004.12.039 [type] => biblio [comment] => 0 [promote] => [moderate] => [sticky] => [format] => 0 [status] => 1 ) ) 

did then #8 and got:

Array ( [storage] => [submitted] => [post] => Array ( ) ) 

when it's empty but this when I try populating it with the doi:

Array
(
    [storage] => 
    [submitted] => 
    [post] => Array
        (
            [paste_data] => 
            [doi_data] => 10.1016/j.eswa.2004.12.039
            [op] => Populate using DOI
            [biblio_type] => -1
            [changed] => 
            [form_build_id] => form-490c6982c3b933f6c7ba08eb0ea134cc
            [form_token] => e7b306288ca323b6ab82bcb658757197
            [form_id] => biblio_node_form
        )

)

It worked at some point but I then installed a couple of more modules and updated a few others (including biblio and cck).

I run Drupal 6.14 and the following modules

Administration menu 6.x-3.0-alpha3
Ajax 6.x-1.14
Bibliography Module 6.x-1.7
Calendar 6.x-2.2
CCK Facets 6.x-1.x-dev (2009-May-02)
Content Construction Kit (CCK) 6.x-2.5
Date 6.x-2.4
Faceted Search 6.x-1.0-beta2
Footnotes 6.x-2.1
Freemind 6.x-1.x-dev (2009-Mar-18)
Link 6.x-2.6
Node Hierarchy 6.x-1.2
Outline Designer 6.x-1.0-rc3
Prepopulate 6.x-1.1
Storm 6.x-1.24
Taxonomy Manager 6.x-2.1
Term Display 6.x-1.1
Thickbox 6.x-1.2
Views 6.x-2.6

Any help would be much appreciated. I'll try rebuilt it on another server and see whether I can identify which module might cause the problem but got to go in a moment.

I've tracked down the problem in my case to CCK fields (number & text) that I added to the Biblio type.

Also, I've come across two issues/bugs that were introduced in Drupal 6.14 due to some changes to the way forms are handled, http://drupal.org/node/302240 and http://drupal.org/node/591696 - it seems as if those will be dealt with in Drupal 6.15 and so for the moment I can live with not having the CCK fields and will revisit the issue when 6.15 is released.

I have a comparable problem with DOI lookup - not working at all: I just get a blank Biblio form again after submitting the DOI. So tried the debugging options starting at #6 and it seems that I'm not getting any lookup data back : (

For info: I have CTools enabled, but no custom CCK fields; the only delta from this thread is that I can post Biblio nodes into Organic Groups (removing Biblio from OG posting had no effect). I do however, have a Frankenstein monster of a Drupal install, so ran the check at #502562-12: Failure to handle DOI and BibTex pre-populate action and only CTools came up. Disabling CTools had no effect at all.

So I'm now not sure if I've set the module/CrossRef info correct, and what should be really easy is not working at all. So I suppose my question is: Is there anymore guidance on configuring the module correctly and linking in to CrossRef? If not, any other ideas as to why the lookup completely fails?

I completed the OpenURL request form at http://www.crossref.org/requestaccount/ and a few days later received an email with a user name and password - although it wasn't clear during registration that I had to wait a few days for such an email. So I've set the correct user name in the biblio settings of my user account... is there anything else? Is there anything that needs to be set in the admin/settings/biblio to complete the DOI lookup? Or is there something else I need to set up with CrossRef? (For example, the CrossRef membership email states 'If your library implements a SFX or other OpenURL-enabled link resolver, please send us your resolver's base URL.' - is this significant?).

In general library systems, OpenURL etc are a new world for me, so please excuse me if I've been a little ignorant, but there didn't seem to be anything relevant in the module documentation.

Grateful for any suggestions - not only do I have to develop the site, I also have to add in 300+ publications as soon as - Faculty are a pretty unforgiving/unhelpful bunch.

It sounds like you have done a lot of the leg work already, so I'm not sure why it's not working. You should only have to enter your "CrossRef OpenURL Account ID: " which in my case was my email address and then it should work.

What version of Drupal are you using? There seems to be some incompatibilities with 6.14, so if you using that version it could be the problem.

Ron.

Thanks - looks like I'll have to sit tight until 6.15 is released and try again.

Alternatively, you could back down to 6.13 and it will probably work.

Ron.

I've recently upgraded to Drupal 6.15 and Biblio 6.x-1.9 and still can't get DOI lookup to work.

Nothing appears in watchdog... any thoughts at all?

I ran into a similar issue with another user and the PubMed look up service, the issue there was that "allow_url_fopen" was not set in the php.ini settings file.

See http://www.php.net/manual/en/simplexml.requirements.php

Ron.

Thanks for the suggestion Ron... PubMed is looking up fine and allow_url_fopen was set to 'On' in all installs I've been playing with.

So if PubMed is using the same sort of process to pre-populate, then my DOI problem must be related to how my Drupal site is accessing my CrossRef account...? I've manually logged in to http://doi.crossref.org with my account and password and looked up a DOI: all fine. Trying to pre-pop a Biblio node using the same DOI fails. Only thing that seems to be missing in User > Biblio settings is adding in the CrossRef password - is that important?

There seem to be two types of accounts with CrossRef, one is for CrossRef members,and has a username/password combination, the other is the "complimentary OpenURL accounts" which just has a username which is usually your email address. (see http://www.crossref.org/help/Content/05_Interfacing_with_the_CrossRef_sy...) could you try the following URL in your browser (replacing "USER_NAME" with your complimentry OpenURL account name)

www.crossref.org/openurl/?pid=USER_NAME&noredirect=true&format=unixref&i...

If that works then biblio should also work, becuase that is the exact URL that is used (varying the DOI number of course).

Ron.

*Click*

I think the problem has arisen because when I initially applied for a CrossRef user account, the email verification link email never appeared (must have been a server issue their end at the time), so I browsed the CrossRef site and also ended up applying for a full account (without realising there was a difference)...

Looking at the CrossRef documentation, I've now found out that in User account > Biblio settings > CrossRef OpenURL Account ID you can either put the email address [of your complimentary CrossRef account] or the account name:password combination (separated by a colon) [if you are a CrossRef member] and it works just fine too.

Maybe adding replacing the 'Enter your CrossRef OpenURL account ID which you can obtain here OpenURL Account Request Form' text with:

- Either enter your complimentary CrossRef OpenURL account ID (which you can obtain here)
- Or enter your full CrossRef account:password combination (separated by a colon)

Thanks again Ron.

Good idea, I've committed the change... http://drupal.org/cvs?commit=314648

added doc changes as requested.
Marking fixed.
bekasu

Why is this marked fixed? Removing chaos tools is not a fix.

Could elaborate on the issue(s) you are having?

Ron.

I like the other people with ctools enabled can't use DOI import or BibTex import. I get the all blank fields symptom.

Which version of the module are you using?

Hi,

I'm also having no luck with the DOI import and mixed luck with the BibTex import. I do NOT have CTools enabled.

So far I've had 100% success with BibTex data from the ACM (acm.org) site and 100% failure with data from the IEEE site (computer.org), here is a sample of IEEE data that just gives me blank fields:

@article{10.1109/ARES.2009.154,
author = {L. Fuchs and C. Broser and G. Pernul},
title = {Different Approaches to In-House Identity Management - Justification of an Assumption},
journal ={Availability, Reliability and Security, International Conference on},
volume = {0},
isbn = {978-0-7695-3564-7},
year = {2009},
pages = {122-129},
doi = {http://doi.ieeecomputersociety.org/10.1109/ARES.2009.154},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}

I noticed that the IEEE data is terminated with a CR (\r) and a BibTex export from the module is terminated with LF (\n) so I tried switching them but no change in behaviour. I also replaced the - in the 'pages' data with the string {\textendash} that is in the exported data, again to no avail.

Some version information:
Drupal 6.16
Bibliography 6.x-1.10
Apache
Fedora 11

cheers

Edit: Some more BibTex examples.

(1) A variation of the provided example that also fails:

@incollection{epub6963,
       booktitle = {Proc of the 4th International Conference on Availability, Reliability and Security (ARES 2009)},
           month = {March},
           title = {Different Approaches to in-house Identity Management},
          author = {G\"unther Pernul and Christian Broser and Ludwig Fuchs},
         address = {Fukuoka, Japan},
       publisher = {IEEE Computer Society},
            year = {2009},
             url = {http://epub.uni-regensburg.de/6963/}
}

(2) An ACM sourced entry that works:

@inproceedings{1035437,
 author = {Koch, Manuel and Parisi-Presicce, Francesco},
 title = {Formal access control analysis in the software development process},
 booktitle = {FMSE '03: Proceedings of the 2003 ACM workshop on Formal methods in security engineering},
 year = {2003},
 isbn = {1-58113-781-8},
 pages = {67--76},
 location = {Washington, D.C.},
 doi = {http://doi.acm.org/10.1145/1035429.1035437},
 publisher = {ACM},
 address = {New York, NY, USA},
 }

I am using the latest stable. I have tried the latest dev versions but the biblio settings dont show up when users edit their profile making DOI lookup even more impossible.

Both the DOI and PubMed lookups depend upon the PHP installation to be able to use fopen on a URL and thus the php.ini setting "allow-url-fopen" must be set to true. Could you check this setting on your (rmh3093 & lostchord) to see if this is the case.

Ron.

@lostchord: comment 33:

You were right in surmising that the problem was the '\r' line endings. I downloaded a Bibtex file from IEEE and had the same issue, however when I converted it to UNIX line endings '\n' it worked fine for me.

There is a PHP setting which can compensate for this issue (auto-detect-line-endings) if you want to give that a try.

Ron

re: allow-url-fopen ..... Yes its enabled

allow_url_fopen is On.

I changed auto-detect-line-endings setting it to On (after uncommenting it), it had no effect.

Changing \r to \n also doesn't work for me. If there is something bad about the entry should I expect some sort of error message?

The BibTex entry I'm using comes from here:

http://www.computer.org/portal/web/csdl/doi/10.1109/ARES.2009.154

The server configuration if Fedora 11 with Drupal 6.16, the browser is IE8. I'll try a different browser just in case....

Edit 1:

Just tried on a different installation (WinXP) and had no problems. One difference, which might be significant, is FCKEditor. It controls the paste field in the failing case but not in the one that succeeds. I will dig some more.

Edit 2:

FCKeditor is the culprit. Reverting to a plain text editor (which is enabled by a link under the FCKeditor text box) solves the problem without needing to change the \r to \n.

Ah, yes that would do it, since it would likely be converting it to HTML and the line breaks would become <br>

I believe you could configure FCK to ignore that particular text area.

Now, if I could just solve this CTools issue....

Ron.

Edit:

BTW, I presume you already know this but just in case... the "auto-detect-line-endings" setting would only affect entries that are imported from a file (as opposed to cutting and pasting), and it would only take affect after restarting your webserver.

Hmmmm...so the conversion to BR would only happen for CR and not for LF in order to explain the ACM -v- IEEE behaviour. Anyway, that problem solved :-)

Your edit probably explains why turning on auto-detect-line-endings made not a scrap of difference after the restart.

Disabling the FCK module does not have any effect on the DOI import, it still seems to do absolutely nothing. Would I get error messages if the interaction with the remote site messes up in any way? At present it's not giving me a single thing to work on.

Apart from this problem (which is not a show stopper for me) I'm really impressed with this module so far. Really nice.

cheers

For the DOI import to work, you have to have registered with CrossRef for a complementary account,and then have entered that into the appropriate section on your user profile page. Assuming you have done this, you could test it with the following URL...

www.crossref.org/openurl/?pid=[your account name]&noredirect=true&format=unixref&id=doi%3A10.1145/1035429.1035437

Obviously replacing "[your account name]" with the appropriate string (usually your email address).

It should return a bunch of XML data to your browser.

OK, making some progress, the crossref site is not recognising my email address. I've been through the verification loop several times now with it saying everything is OK and I can use the service but it still doesn't want to know my email address.

I've sent a support email. I'll post details of the response here since this might be impacting other users having trouble with DOI import as well.

cheers

No response yet from crossref but access now works and DOI import now works. Someone poked around my unpublicised web site shortly after I sent the support request. They seemed particularly interested in the bibliography section. I'm assuming something in the biblio module would have indicated the originating 'client' via referer headers (???).

Anyway, working now.

If I get a response I'll publish it here. It seems some moderating of access requests might be going on that Biblio users might need to be aware of.

cheers

Yes, I hope that inclusion of this feature in the module will not result in CrossRef removing this capability from their site, although I can't imagine that it's biblio that would be overwhelming them since you still have to enter the DOI's manually one at a time, it's not like you can say "Get me ALL the DOI's in this range".

any update on ctools suport in this module? i really want to be able to use panels.

Not really, other than to say that I can't reproduce the problem on my machines (presumably because, although ctools is enabled, I'm not actually using it anywhere).

If you could point me to some code or other module which use ctools, then maybe I could come up with a solution.

Ron.

See #1201696: Importing into node-add-form conflicts with Panels Page Manager node-edit template for steps to reproduce.

This version is no longer maintained. If this issue is still relevant to the Drupal 7 version, please re-open and provide details.