I have used the approach described in the video Forena and Feeds to import nodes from an external data source, using the settings described here. When run through the UI it works just as it should and provides a good way of keeping a Drupal content type in sync. with external data. Very good !
However when I try to set the Periodic Import to a value, no import takes place. I am using an external Cron service and this is running as confirmed by the Feeds Log. However no new or updated nodes are imported. Are there issues with Periodic import using Forena ?
The dblog shows this error message which might provide a clue as to what is going wrong:
Notice: Undefined variable: fields in FeedsForenaXMLParser->parse() (line 10 of /home/sites/zzzzzzzzz.org/public_html/mmclone1/sites/all/modules/forena/feeds/FeedsForenaXMLParser.inc).
There are a lot of posts in the Drupal forum relating to problems with periodic feed imports, one of which is to set the frequency to "as often as possible". However if I do this it breaks the site. One other approach I followed was to try and run the feed import by module code ( see this article ( #21 ) but I can't get this code to fire..
<?php
function f1mport1_cron() {
$name = 'church_events_f3';
$source = feeds_source($name);
$source->import();
watchdog('f1import1',"Message %number: %message",
array("number"=>10, "message"=>"Feeds Cron Fired"));
}
?>
Can you suggest a set up which will allow the technique described in the video to work at a scheduled frequency. Thanks.
| Comment | File | Size | Author |
|---|---|---|---|
| #4 | Drupal_Forena_2305a.png | 66.99 KB | peterk900 |
Comments
Comment #2
peterk900 commentedI have done some more work here.
1. UI import appears to work faultlessly but logs the undefined variable, so maybe this isn't the cause of the cron issue. The snippet below seems to be the point in the code where the error is occurring, but no changes here seem to have any effect, for example setting it to an empty array.
2. I have got the code to work which fires cron outside the Feeds scheduler - see below. This code will import a .csv when the Feed Periodic Import is set to 'Off'. As noted above, other people have found this is the right way to go to use cron with Feeds.
3. Using the Common Syndication Parser does not work.
I'm not sure what else to try. Something is causing the import of a Forena block to fail when run via cron - whereas it runs just fine via the UI.
It really would be good to solve this because, to my knowledge, there is no other way to automate the import of an external MySQL db into Drupal ( in my tests Feeds SQL breaks the site ), something that I think is becoming increasingly important.
Additional Point
If you run the feed import via code outside of cron then the import works - but the error message now appears in the front end if you ran the code via a form or linked to another trigger such as custom post creation. So this falls between UI - works with no visible error and Cron - fails with visible error. I hope this is helpful.
Comment #3
metzlerd commentedI'm suspecting a permissions issue. Could you try doing it using "run cron now" button so that you're logged in when the cron event fires?
Comment #4
peterk900 commentedThanks for coming back on this.
I have tried running Cron via Configuration -> Cron -> Run Cron but no import takes place. Works like a dream though via the UI. Cron was run as system administrator. I attach a picture of the message and feed logs.
Comment #5
peterk900 commentedI don't know if it is related to the problems encountered with Feeds import but I was unable to connect to the external database via PDO in Forena. However this Drupal site already has a connection to an external db in settings.php ( used for Views Database connector - VDC ) and this gets picked up by Forena when you select Drupal PDO.
The connection string in settings.php works and is the same as used in the Forena uri - which is puzzling. If you have any thoughts on why this string will not connect in Forena but works fine with VDC that would be good as I will probably want Forena to work with a site where VDC is not installed. But getting me on the right track with the scheduled import is the most important !
Comment #6
metzlerd commentedI am extremely busy this week but will try and get to setting up a test environment as soon as I can. Please be patient.
Dave
Comment #7
peterk900 commentedThanks Dave. The imports work manually and I have plenty of things to try with Forena outside importing, so no panic. It would be good to solve this though as once scheduled importing works robustly Forena provides a very elegant way to keep external db's synced with Drupal.
Some final test information in case it helps. I set up a job in Job Scheduler, changed the Feed Import frequency to None and called this code from the scheduler on an hourly basis. Cron is run every 15 minutes outside of Drupal via Easycron. The attached screen shot shows the Feeds Log, the Error log, the Error message referring to the undefined variable in the Forena xml parser and the results when the import is run via the UI, confirming that there was a node to import. The watchdog entries show the code to run the import is firing. The code works fine with a non-Forena import e.g a .csv so it looks to me as though it is this parser error that may well be causing the issue.
Here is the link to the screen prints referred to above.
Peter
Comment #8
peterk900 commentedI believe I have solved the problem(s) causing periodic imports to fail.
1. (As you suspected) Get the Cron job to run with administrative privileges rather than Anonymous(not verified) - see https://www.drupal.org/node/2884333#comment-12118961
2. Get rid of the line 10 error messge by moving the line:
to the line below where $fields is defined in FeedsForenaXMLParser.inc
With these changes cron seems to be importing data as it did when the importer was run from the UI. Fantastic ! This is a lot easier than doing it via automated .csv creation.
Comment #9
metzlerd commentedthat is great news. I will look at the section of code and upload na appropriate patch.
Comment #10
peterk900 commentedYes it is ! With Elysia Cron running code that fires a Forena import every minute you have very tight synchronisation between MySQL data and a Drupal Content Type. With the data in a content type then you have the full flexibility of Views to display it and link it as if it had been created in Drupal - more scope than using Views Data Connector. With this out of the way I am now playing with the reporting power of Forena !
When you look at the code I'd be grateful if you could check out the PDO connection part of the script. As posted earlier I cannot get this to connect to any db other that the Drupal default database. I am currently connecting to an external db via connection data appearing in settings.php.
Thanks.
Comment #11
metzlerd commentedFYI: I have several databases using PDO connections and forena. In fact the sample database that comes with forena is a PDO sqllite database. Usually the reason is confusion on the connection string. Is there a specific error message that I can help you troubleshoot?
Comment #12
peterk900 commentedYou are quite right - it does work when you set up a connection that is not included in settings.php ( which is what I want ). You get an "Unable to connect to database " ( or similar ) message if you try and set up a Forena data connection where one already exists. You get the same message if you try and set up a PDO connection to the Drupal database, rather than selecting Drupal from the driver drop-down. Thanks for making me think again on this !
Comment #14
metzlerd commented