I have a Content Type with a Geofield. I have a feeds importer generating nodes from an XML feed. All the fields are importing fine, except for the Geofield. The importer worked fine before we updated to the new dev version of feeds: 7.x-2.0-alpha8+102-dev. I'm not really able to downgrade, though, due to other required functionality.

I thought this issue was related to https://www.drupal.org/node/2192421, but now I think it's similar but not the same. After a few hours of digging, I've tracked down why it's not saving, but I don't know the implications. Essentially, when feeds runs any processor, it runs module_invoke(feeds_presave). The only function that's calling for me is field_feeds_presave() in feeds/mappers/field.inc. That function calls _field_filter_items() from /modules/field/field.module, which then in turn calls geofield_field_is_empty($item,$field) in geofield.module.

The $item being passed in is a simple 2 item array:
(Array, 2 elements)
lat (String, 7 characters ) 33.6434
lon (String, 8 characters ) -80.1966

So, from my reading of geofield_field_is_empty(), it essentially doesn't see an $item['input_format'] defined so that code doesn't run, and it also doesn't see any $item['geom'], so it returns true and the geocoordinate values are removed.

So, I guess my question is, at which point in this process is it going wrong? If I manually run node_save() with the same exact field array values, it works. I'm guessing the standard node_save() doesn't have this intermediary check. So, should there be another step in geofield_set_target_simple() that does some processing and adds in that ['geom'] value? I've attached photos of my setup in case I'm just configuring the import incorrectly.

Thanks!

Comments

SeeWatson’s picture

Issue summary: View changes
megachriz’s picture

Feeds has recently become stricter in handling empty values and calls indeed _field_filter_items() in order to avoid saving field data which the module considers to be empty. See for example these Feeds issues:
#2344827: Importing empty email field: email field is anyway created in the DB
#2499251: Link module: empty fields are rendered
#1107522-195: Framework for expected behavior when importing empty/blank values + text field fix

To fix this issue, Geofield should either adjust the empty check in geofield_field_is_empty() or make sure that geofield_set_target_simple() always sets $item['geom'].

sistro’s picture

I've the same issue.

I solved using alpha9 version of module.

SeeWatson’s picture

StatusFileSize
new492 bytes

Something like this? It corrected the issue for me, but I don't know if this is the right approach.

Yorgg’s picture

The patch worked for me. Thanks.

mitterer10’s picture

I tried to apply this patch but it isn't working as I get errors like "patch unexpectedly ends in middle of line".

I'm using geofield 7.x-2.3 and feeds 7.x-2.0-beta1.
Might this be the problem?

Thanks!

tondeuse’s picture

Works with 7.x-2.3+0-dev, applies fine, thanks,put an end to hours of questions.

SeeWatson’s picture

Hi mitterer10, I don't think the versions would be an issue, as the current dev and stable Geocoder are identical. Maybe just try one of the other approaches here: https://www.drupal.org/patch/apply. I used the p1, I think.

FreeAndEasy’s picture

patch works! please commit!

FreeAndEasy’s picture

Status: Active » Reviewed & tested by the community
joelpittet’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/geofield.feeds.inc
@@ -143,6 +143,10 @@ function geofield_set_target_simple($source, $entity, $target, $values) {
+  if(!empty($entity->{$field_name}['und'][0]['lat']) && !empty($entity->{$field_name}['und'][0]['lon'])){
+    $entity->{$field_name}['und'][0] = geofield_compute_values($entity->{$field_name}['und'][0]);
+  }

This doesn't account for multi value fields. Probably should loop through the fields instead of hard coding a 0

LANGUAGE_NONE should be used instead of 'und'.

And nit on coding standards: spaces need before and after parentisies in the if statement. aka if () { not if(){

joelpittet’s picture

Version: 7.x-2.3 » 7.x-2.x-dev
Status: Needs work » Needs review
StatusFileSize
new531 bytes

Ok maybe this would be a bit more robust?

joelpittet’s picture

And just to be make note, geofield_field_is_empty() is not getting input_format on the $item array so it just checks if 'geom' is empty and returns TRUE because it is not there.

So an alternative solution would just be maybe making that smarter? But I think this solution probably is the best so thank you @SeeWatson

SeeWatson’s picture

Thanks for the review, @joelpittit. I went with und because that was what the rest of the module was using and I didn't want to break the convention. Are incremental formatting improvements the norm for things like this even if they break the existing module convention?

joelpittet’s picture

I was tempted to fix the other uses of 'und' in that function but I held restraint, but we should all be using the language constant in all modules.

The formatting is from drupal's coding standards.

Control statements should have one space between the control keyword and opening parenthesis, to distinguish them from function calls.

https://www.drupal.org/coding-standards#controlstruct

daemonchrist’s picture

Patch in #12 fixed this problem for me. Thanks!

pryrios’s picture

Status: Needs review » Needs work
StatusFileSize
new907 bytes

Patch in #12 only solves the issue for latitude/longitude widgets. I'm facing the same problem using bounding boxes so I have added some more code to that patch but I think we may be missing many cases here. Someone more savvy should really take a look at this mess and find the correct solution.

andrey_dver’s picture

Hello.
I had the same problem.
My modules:
Geofield 7.x-2.3
Feeds 7.x-2.0-beta1
Feeds import had not saving lat and lon.
Patch #17 fixed this problem.
Thanks!

hypertext200’s picture

hypertext200’s picture

Status: Needs work » Reviewed & tested by the community

Patch #17 fixed the problem.

pingevt’s picture

I'm only using the long and lat fields at the moment but Patch #17 worked for me.

mouhammed’s picture

Patch #17 works.

madar’s picture

With long and lat fields Patch #17 worked for me.

dillix’s picture

#17 works fine.
+1 for commit this.

zolee’s picture

Patch #17 fixed the problem.

jeffschuler’s picture

#17 worked for me for simple lat/lon import. Thanks!

madmanmax’s picture

#17 worked for me as well. Now the lat/lon import is working.

redeight’s picture

#17 fixed the issue for me as well.

geru’s picture

#17 worked to fix lat / long not getting imported

denix’s picture

#17 works great! Thanks

nattyweb’s picture

#17 worked for me. Thank you.

kupide’s picture

Patch #17 worked for me too, thanks you.
Please commit !

darvanen’s picture

+1 for commit =)

znerol’s picture

Feeds 7.x-2.0-beta2 is now ready for entity translation, hence the correct language code should be used instead of LANGUAGE_NONE. See also the change record.

megachriz’s picture

the correct language code should be used instead of LANGUAGE_NONE

See also #2672788: Feeds integration: support mapping option "language".

m.stenta’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/geofield.feeds.inc
@@ -142,6 +142,21 @@ function geofield_set_target_simple($source, $entity, $target, $values) {
+      $field[LANGUAGE_NONE][$delta]['geom'] = $field[LANGUAGE_NONE][$delta];

When using bounding box values, why is $field[LANGUAGE_NONE][$delta]['geom'] being set to $field[LANGUAGE_NONE][$delta]?

m.stenta’s picture

This issue is somewhat related because it provides a patch that alters the geofield_field_is_empty() function, specifically checking to see if ['geom'] is not set, and declaring the entire geofield empty if geom is empty.

#2672108: Don't save empty geometry WKT

EDIT: To be clear, they are two separate issues, and both need their own patch. I just wanted to point out that they deal with similar code. :-)

m.stenta’s picture

Status: Needs work » Needs review
StatusFileSize
new796 bytes

Attached is a new patch that uses geofield_compute_values() for both lat/lon and bounding box imports - ensuring that all the geofield data is filled in before saving.

I have tested this with lat/lon import, but not with bounding box import. @Pryrios: would you mind testing this patch, since you are importing bounding boxes?

mikejuic3’s picture

@m.stenta

I have tested your patch and while I am not getting any errors it still doesn't seem to import.

I am building an importer for INSPIRE metadata so am quite busy on this at the moment. I will report back if I figure out what is going wrong.

m.stenta’s picture

@mikejuic3 - What parts are you trying to import, specifically? Latitude + longitude? Bounding box? WKT?

My patch is tested with Lat/lon, but I haven't tested with bounding box (although it should work the same).

mikejuic3’s picture

@m.stenta The xml I am importing only contains bounding box coordinates. So I am importing bounding boxes. The feeds importer debugger shows that it is pulling in the BB coordinates but from there the import goes no further. I have tried both #38 and #17.

Cheers

mikejuic3’s picture

A little more info, I think I have it more narrowed down:

$field[LANGUAGE_NONE][$delta] = geofield_compute_values($value, GEOFIELD_INPUT_BOUNDS);

Seems to return an empty array. $value contains my bounding box but GEOFIELD_INPUT_BOUNDS is not taking the bait. Watch this space.

mikejuic3’s picture

I have it importing from bounds after some modifications. geofield_set_target_simple now looks like this:


function geofield_set_target_simple($source, $entity, $target, $values) {
  list($field_name, $sub_field) = explode(':', $target, 2);
  if (!is_array($values)) {
    $values = array($values);
  }

  $field = isset($entity->$field_name) ? $entity->$field_name : array('und' => array());
  $delta = 0;
  foreach ($values as $value) {
    $field['und'][$delta][$sub_field] = $value;
    $delta++;
  }
// Compute all the geofield values for each field value.
    foreach ($field[LANGUAGE_NONE] as $delta => $value) {
      if (!empty($value['lat']) && !empty($value['lon'])) {
        $field[LANGUAGE_NONE][$delta] = geofield_compute_values($value, GEOFIELD_INPUT_LAT_LON);
      }
      elseif (!empty($value['top']) && !empty($value['right']) && !empty($value['bottom']) && !empty($value['left'])) {
        $wkt_bounds_format = 'POLYGON((left bottom,right bottom,right top,left top,left bottom))';
        $wkt = strtr($wkt_bounds_format, array('top' => floatval($value['top']),
          'right' => floatval($value['right']),
          'bottom' => floatval($value['bottom']),
          'left' => floatval($value['left'])));
        $geometry = geoPHP::load($wkt);
        $field[LANGUAGE_NONE][$delta] = geofield_compute_values($geometry);
     }
    }

  $entity->$field_name = $field;
}

There were a couple of issues that needed to be sorted out and I am sure it can be done more cleanly than this.

Issues encountered:

  1. geofield_compute_values uses geofield_geometry_from_values to get the geometry. The geometry from values function expects the array to be structured as: $raw_data['geom']['top']. I ripped the code, altered it and stuck it in the funtion above, although it should probably be sorted out in geofield_geometry_from_values
  2. I was getting errors on floats that had minus values ie:-22.1234 so I wrapped them in floatval() which fixed that.
  3. Finally I pass $geometry to geofield_compute_values() with no extra arguments and it processes the WKT into all the fields required for the import to work.

I can roll this into a patch but would like some input as to doing it a bit more efficiently.

Cheers.

1mundus’s picture

Did anyone make Geofield CSV import work with Feeds 7.x-2.0-beta2? I have tried all of the patches listed here and none work. My configuration:

Feeds 7.x-2.0-beta2
Geofield 7.x-2.3

Mapping:

Lat -> Coordinates Latitude (field_coordinates:lat)
Lon -> Coordinates Longitude (field_coordinates:lon)

CSV file example:

Title|Language|Lat|Lon
Some title|en|20.00000|20.00000

All of the other fields get imported, but Geofield coordinates don't. I have also tried to make it work without the language set either in CSV or in the feed settings, but the result was the same.

darvanen’s picture

Yes @1mundus #17 worked fine for me with exactly those requirements.

m.stenta’s picture

@mikejuic3 - Your logic makes sense, but I agree we should see if it is the most efficient. Your approach is essentially bypassing the bounding box entirely, by creating a polygon from the points, and then letting geofield_compute_values() fill in the rest.

So in essence, you're throwing out the second parameter of geofield_compute_values(), and therefore geofield_geometry_from_values() (the second parameter being $input_format in both). It works the same, but it feels to me like that piece of information (the input format) should be preserved.

So it sounds like the real issue here is: why is geofield_compute_values($value, GEOFIELD_INPUT_BOUNDS); returning an empty array as you said in comment #42?

Is it because $value needs to be formatted differently? Does it need to be a nested array within $value['geom']?

In other words, what if you use my patch from #38, but add the following line right above line 152:

$value = array('geom' => $value);

Does that make sense? Attached is a new patch to demonstrate what I mean. Could you test to see if this works?

mikejuic3’s picture

@m.stenta

Apologies for taking so long to get back to you. I will test this out today or tomorrow and let you know. I can give a bit more detail then too.

ccjjmartin’s picture

I only tested patch #46 and I can verify it works. By it works, I mean that originally I was able to import nodes but not geofields. And now with the patch, I can import nodes and geofields.

Great work! Thanks for your contribution. [@m.stenta]

roadsideok’s picture

#46 worked for me. Thank you all for the work on these patches!

EDIT: It worked for LAT/LONG geofields on newly imported terms, but does not seem to update existing terms.

kumkum29’s picture

Hello,

#17 works for me with new nodes & existing nodes. I don't use the #46 because this patch seems to create a problem with the existing contents (#49).

Have you planned to commit this patch in the next version of Geofield? (important bug).

Thanks.

kansaj’s picture

#17 I'm not quite sure that #17 updates the nodes, I was not able to update long and lat.

m.stenta’s picture

kansaj: My patch in #46 above can handle lat/lon. Please test that one and let us know if it works for you.

kansaj’s picture

#52 I confirm that it handles correctly new nodes, but old ones are not updated.

gaëlg’s picture

Status: Needs review » Reviewed & tested by the community

@kansaj #46 worked for me, even for existing nodes, as soon as I temporarily checked "skip hash check" in the importer settings, for the first import after the patch had been applied.

m.stenta’s picture

Thanks @GaëlG, good point about the "skip hash check".

@kansaj, @kumkum29: can you please make sure that you have "skip hash check" unchecked when you are testing?

Feeds performs a hash on the data you are importing, and if it finds the same hash when you try to import the same data again, it will assume that nothing needs to be changed, so it won't update the nodes. For testing purposes you should have that hash check turned off, otherwise it will never make it to the code that we want to test.

kansaj’s picture

Hi,
I could confirm the "skip hash check" worked as expected and updated the nodes.

m.stenta’s picture

Awesome, thanks for confirming @kansaj!

I think this is patch ready to be committed! Thanks everyone for helping to review!

makbuk’s picture

#17 works for me.
Thanks.

m.stenta’s picture

@makbuk Thanks for the input - but the patch that we are currently reviewing is #46. Can you confirm that that one works for you as well?

e5sego’s picture

Patch #46 works for me

Note: During import of latitude/longitude data, the correspondending field widget has to be "Latitude / Longitude".

cimo75’s picture

Patch #46 works for me importing lat and lon one by one.

marcopbazz’s picture

Patch #46 worked for me too. I used it for a leaflet widget field

ShedPhotons’s picture

Patch #46 worked for me - imports to POINT.

das-peter’s picture

Another confirmation that the patch works.

ugintl’s picture

#46 WORKED FOR ME

sealionking’s picture

Version: 7.x-2.x-dev » 8.x-1.x-dev
Assigned: Unassigned »
Category: Bug report » Feature request

Need feeds support for D8.5
Could anyone familiar with these two module make a patch/ or patches?
I am constructing a site with info of a big bulk of locations and need to import those lat/lon data to their nodes and terms.
Sincerely wish you guys will offer support to solve this since I am not coder but only a builder.
Many thanks.

m.stenta’s picture

Version: 8.x-1.x-dev » 7.x-2.x-dev
Assigned: » Unassigned
Category: Feature request » Bug report

@sealionking Please open a new feature request for the 8.x-1.x branch. This one is specifically about a bug that exists in the current 7.x-2.x codebase, which has a proposed patch ready for review/commit.

soyjaz’s picture

#46 worked for me aswell - if that's anyhow relevant to point out. Thanks.

frogdog_tech’s picture

#46 bingo

ConradFlashback’s picture

#46 works

  • itamair committed 1f580b3 on 7.x-2.x authored by m.stenta
    Issue #2534822 by m.stenta, SeeWatson, joelpittet, Pryrios: Feeds Import...
itamair’s picture

Status: Reviewed & tested by the community » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

scott.whittaker’s picture

Issue is closed but patch still required. Dev branch only?