Geofield Mapquest Geocoder

Sumarizing the issue:

Geocoding from Google works fine. From MapQuest it does not get data.

Same problem here using Addressfield module.

Moving to the Geocoder issue queue. @Mac_Weber, in reference to re-encoding data, take a look at the included drush function geocoder-backfill. You can find the definition and how to use it in geocoder.drush.inc

Actually moving to Geocoder issue queue... :-/

@Brandonian I am not re-encoding, I'm just creating new nodes on the UI.
The coordinates data is empty if not using google.

You could try using the drush command "drush geocoder-backfill" which will geocode all data that doesn't have corresponding geofield data.

The original poster is correct in his assertion that Mapquest doesn't always return results when other Geocoder services might. Take for instance:

'208 West Market Street,Georgetown,DE,US'

Returned from geofield widget parse on an Addressfield.

The request string (after drupal_http_build_query()):

GET /nominatim/v1/search?q=208%2BWest%2BMarket%2BStreet%2CGeorgetown%2CDE%2CUS&format=json&addressdetails=0&limit=1&osm_type=N HTTP/1.0
Host: open.mapquestapi.com
User-Agent: Drupal (+http://drupal.org/)

I get a response, and it is returned. With the coordinates, I map it to Leaflet. But the pin is nowhere near where it should be, it's off by 15 miles.

Digging in, and making a change like this:

function geocoder_mapquest_nominatim($address, $options = array()) {
  $geocoder_settings = variable_get("geocoder_settings", array());
  $api_url = "http://open.mapquestapi.com/nominatim/v1/search";
  $query = str_replace(' ', '+', $address);
  $params = array(
    #'q' => str_replace(' ', '+', $address),
    'format' => 'json',
    'addressdetails' => 0,
    'limit' => 1,
    'osm_type' => 'N',
  );
  $request = drupal_http_request($api_url . '?q=' . $query . '&' . drupal_http_build_query($params));
  $data = json_decode($request->data);
  return _geocoder_mapquest_nominatim_geometry($data);
}

The new request string (note that the q= address matches Mapquest format here):

GET /nominatim/v1/search?q=208+West+Market+Street,Georgetown,DE,US&format=json&addressdetails=0&limit=1&osm_type=N HTTP/1.0
Host: open.mapquestapi.com
User-Agent: Drupal (+http://drupal.org/)

This returns coordinates too, but the correct ones. The pin is perfectly placed on my Leaflet map. I am not exactly sure why the handler gets a different (correct) result when geocoder_mapquest_nominatim() is modified like that.

Another example, this time, raw code:

  $address = urlencode('208 West Market Street,Georgetown,DE,US');
  $url = 'http://open.mapquestapi.com/nominatim/v1/search?q=' . $address . '&format=json&addressdetails=0&limit=1';
  $response = drupal_http_request($url);
  $data = json_decode($response->data);

  $features[] = array(
    'type' => 'point',
    'lat' => $data[0]->lat,
    'lon' => $data[0]->lon,
    'icon' => array(
      'iconUrl' => '/sites/all/libraries/leaflet/dist/images/marker-icon.png'
    ),
  );
  $map = leaflet_map_get_info('OSM Mapnik');
  return leaflet_render_map($map, $features, '700px');

The above also returns the correct data for coordinates and places it on the map.

Another thing that was affecting my ability to get coordinates was the order in which the address field was being parsed.

Take this address for instance: 1 Mississippi Avenue, Milton, DE, 19968. (Address 2 field is Broadkill Beach). The request was being constructed as:

Broadkill Beach, 1 Mississippi Avenue,DE,US

No results returned, nothing.

Using the raw code above and this page http://open.mapquestapi.com/nominatim/v1/search.php, with this format, I was able to get coordinates quickly:

1 Mississippi Avenue, Broadkill Beach, DE, US

To me, there seems like there is a bug somewhere within either the handler or the addressfield parser in regards to the premise and how the request is constructed for Mapquest.

Has anyone been able to get the Mapquest Nominatim geocoding to work yet? Consistently?

We did some hacking as suggested by kevin (above).

We also went to http://open.mapquestapi.com/nominatim/ and did some sample "Basic Search" requests and observed that sometimes the latitude and longitude seemed possibly too long (as I think was observed by others before).

We then played with return new Point($data[0]->lon, $data[0]->lat); and tried to use the PHP "round" function to get our lat and lon short enough. We had a little success with this, but seemed like it wasn't consistent (and may not have been the right place to have been doing what we were doing). Sometimes it still refused to work at all. After using the PHP "round" function, when it didn't work, it wouldn't give us any error messages, it just wouldn't save any values.

With Yahoo discontinuing their service and Google charging a minimum of $10,000/yr to use theirs (for over 2500 requests per day), it seems like more people would be interested in using Nominatim...right? At least for higher volumes.

I couldn't use it because MQ search didn't seem reliable.

I am about to attempt to import over half a million nodes which will be geocoded. This has to be reliable.

@kevinquillen: Even with your code changes in #6 you feel that the MQ service is unreliable?

Tested without changes in #6: First try it was off by 15 miles
Tested with changes in #6: Got nothing back...zip

Using geocoder 1.2

Mapquest Geocoder is worse than worthless specially since it gives you wrong results without any warning.

markusa,

For that I would suggest using google and getting an enterprise account.

Thats so expensive

looks like the countrycodes need to be submitted as a separate name/value pair:
http://open.mapquestapi.com/nominatim/

I also removed zip codes and just used city and state to get better results

I get returned results in the right city but not even close to the actual building. It returned results for half of my addresses

Regarding greenwe's comment, "It [MapQuest API] returned results for half of my addresses."

We have noticed a lot of issues with the MapQuest API that appeared roughly around January 2013. We drilled down and discovered that including the following fields in the address you send to MapQuest produced null results.

* P.O. Boxes
* Zip+4

Using the same street address works if you you strip out the P.O. Box line and the "+4" portion of a U.S. zip code.

Also, this caused issues.

* Zip codes that also match postal codes in non-U.S. countries

Compare the results from MapQuest and Google for a single zip code.

MapQuest responded with the coordinates for Lüdersfeld, Germany.
http://open.mapquestapi.com/nominatim/v1/search?q=31702&format=json&addr...

Google returned the coordinates for Albany, GA.
http://maps.googleapis.com/maps/api/geocode/json?address=31702&sensor=false

I can get my desired U.S. response from MapQuest if I include a "countrycode" parameter.

http://open.mapquestapi.com/nominatim/v1/search?countrycodes=us&q=31702&...

It feels like we can obtain correct results from MapQuest if we learn the intricacies of its API, but Google seems to "figure out" the desired answer based on whatever we enter into the address field.

I think Geofield module needs more work to provide a stable MapQuest interface. (I'm inclined to switch to a different solution, rather than understanding the MQ interface.)

Problem is still there in the last version. I'll try to work on it this week.

Here's what I got playing with the API (I only indicate the q parameter, the first 3 ones are with the current API, the others with the #6 modification (included in the patch)) :

• 36%2Brue%2Bbabinet%2CToulouse%2CFR: wrong result
• 36%2Brue%2Bbabinet%2CToulouse%2CFR%2C31100: wrong (but different) result
• 36%2Brue%2Bbabinet%2CToulouse%2C31100: wrong (same than precedent) result
• 36+rue+babinet,Toulouse,FR,31100: no result
• 36+rue+babinet,Toulouse,FR: correct result
• 36+rue+babinet,Toulouse,31100: correct result
• 36+rue+babinet,Toulouse: correct result
• 36+rue+babinet,Toulouse,31100 (&countrycodes=FR): correct result
• 36+rue+babinet,Toulouse (&countrycodes=FR): correct result

So, #6 helps by either returning the correct location, or returning nothing, and I think the country code should be sent in a different parameter.

Here is a patch for #6 (all credit goes to kevinquillen, this patch is part of the #1day1patch initiative), that should fix it for some people, but that does not solve everything (for example for France when you send the complete address). I put the issue to "Needs review" for the maintainer to see what he can do, but I think the issue still needs a bit work for the country code.

(it's on the current -dev version, actually)

The precedent patch was wrong because the parameter is "q" and not "query".
In the attached patch, I added a little "hack" that works for me with address field by sending countrycode in a separate parameter. I don't think it's the proper architecture fix, but it works for me for now, so...

Tested patch #19 with a couple Brazilian addresses.
It worked as described on #17.

The patch I add fixes it by removing the zip code from the address being passed to MapQuest:

+++ b/plugins/geocoder_handler/mapquest_nominatim.incundefined
@@ -41,8 +42,16 @@ function geocoder_mapquest_nominatim_field($field, $field_item) {
+    if (!empty($field_item['postal_code'])) {
+      unset($field_item['postal_code']);

There is still a problem on the function geocoder_mapquest_nomination(). If there is data already saved, and later an invalid address is given, then this function will return FALSE. However, it will not update the geocoordinates saved on the database when the last address is invalid.

This function must be changed in order to update the saved values to NULL.
(Does this problem also happen with other plugins?)
Edit: update is working for me after this patch:#1840920: Cannot safely change widget type

Tip: MapQuest sometimes does not geocode address missing small things, such as not including "Avenue" in the given address.

Updating the field with an invalid address is fixed affter applying the patch #1840920: Cannot safely change widget type

Then, the path on #20 seems good to be tested without additional work.

I've been having the problem. ill check out the patch.

+++ b/plugins/geocoder_handler/mapquest_nominatim.incundefined
@@ -24,14 +25,14 @@ $plugin = array(
+  $query = str_replace(' ', '+', $address);

This line must be changed to

+++ b/plugins/geocoder_handler/mapquest_nominatim.incundefined
@@ -24,14 +25,14 @@ $plugin = array(
+  $query = urlencode($address);

Now MapQuest is requiring an API key to deliver results. It has to be added to code.
I'm not sure I will do it myself, as I'm using google for now.

The license is only needed if you make more than a certain number of requests par day, I think.

API Key is required from all users as per
http://developer.mapquest.com/web/products/open/forums/-/message_boards/...

#23 through #26 is really a separate issue. I've rolled a patch against dev without #20 to add the API key to the query.

Edit: oops, left a dpm call in there. I'll post an updated patch momentarily

Fixed version of #26.

Here's #20 rolled against #26

As an aside, I'm not sure I understand why the query was broken out of the params array and is now being concatenated manually. Shouldn't the end effect be the same?

One last time. #28, updated to build the url using url() instead of string concatenation. I'm successfully running queries again now with this one.

@zxaos, #23 is useful on your patch, too.

@Mac_Weber

In #29 I'm calling drupal's url() which doesn't want urlencoded parameters.
However, when I tested simply passing the address in directly and hoping url() would encode it, I got no results, so it seems like it needs some pre-processing. I'm just concerned that we might run into an issue where parameters get incorrectly double encoded if we call urlencode() on the query proper.

I haven't tested it, but you can try a dpm() to see the URL being sent

i'm sorry but i've done all till #32 changing all, so my code is this:

<?php
// $Id$
/**
 * @file
 * Plugin to provide a MapQuest Nominatim geocoder.
 */

/**
 * Plugins are described by creating a $plugin array which will be used
 * by the system that includes this file.
 */
$plugin = array(
  'title' => t("MapQuest Nominatim"),
  'description' => t('Geocodes via MapQuest Nominatim'),
  'callback' => 'geocoder_mapquest_nominatim',
  'field_types' => array('text', 'text_long', 'addressfield', 'location', 'text_with_summary', 'computed', 'taxonomy_term_reference'),
  'field_callback' => 'geocoder_mapquest_nominatim_field',
  'terms_of_service' => 'http://developer.mapquest.com/web/info/terms-of-use',
);

/**
 * Process Markup
 */
function geocoder_mapquest_nominatim($address, $options = array()) {
  $geocoder_settings = variable_get("geocoder_settings", array());
  if (!empty($geocoder_settings['geocoder_apikey_mapquest'])){
    $api_key = $geocoder_settings['geocoder_apikey_mapquest'];
  } else {
    drupal_set_message(t('Unable to perform geocoding'), 'error');
    watchdog('geocoder', 'A MapQuest API key must be set before it can be used', array(), WATCHDOG_ERROR);
    return;
  }
  $api_url = "http://open.mapquestapi.com/nominatim/v1/search";
  $params = array(
    'key' => $api_key,
    'q' => str_replace(' ', '+', $address),
    'format' => 'json',
    'addressdetails' => 0,
    'limit' => 1,
    'osm_type' => 'N',
  ) + $options;
  $request = drupal_http_request(url($api_url, array('query' => $params)));
  $data = json_decode($request->data);
  return _geocoder_mapquest_nominatim_geometry($data);
}

function geocoder_mapquest_nominatim_field($field, $field_item) {
  if ($field['type'] == 'text' || $field['type'] == 'text_long' || $field['type'] == 'text_with_summary' || $field['type'] == 'computed') {
    return geocoder_mapquest_nominatim($field_item['value']);
  }
  if ($field['type'] == 'addressfield') {
     $options = array();
    if (!empty($field_item['country'])) {
      $options['countrycodes'] = $field_item['country'];
      unset($field_item['country']);
    }
    if (!empty($field_item['postal_code'])) {
      unset($field_item['postal_code']);
    }
    $address = geocoder_widget_parse_addressfield($field_item);
    return geocoder_mapquest_nominatim($address, $options);
  }
  if ($field['type'] == 'location') {
    $address = geocoder_widget_parse_locationfield($field_item);
    return geocoder_mapquest_nomination($address);
  }
  if ($field['type'] == 'taxonomy_term_reference') {
    $term = taxonomy_term_load($field_item['tid']);
    return geocoder_mapquest_nominatim($term->name);
  }
}

function _geocoder_mapquest_nominatim_geometry(&$data) {
  if (!isset($data[0]->lon)) {
    return NULL;
  }
  geophp_load();
  return new Point($data[0]->lon, $data[0]->lat);
}

but what or where is this file called: plugins/geocoder_handler/mapquest_nominatim.incundefined in #23?
i don't have it. can anyone tell me how to find it?

And as i've read i've now my Application Keys (AppKeys) from http://developer.mapquest.com/ but... i really don't find where to insert in to, cause here: admin/structure/openlayers/layers/settings api keys in drupal doesn't show anything else than BING, CLOUDMADE, GOOGLE.
Have I to modify anything else?
tnx a lot :)

This patch adds the data array to the point object for nominatim results so that they can be accessed by the Openlayers_geosearch module;

@davidserene, I suppose the xdebug_break() was not intended in the patch ;-)

Hi simon, thanks for the catch, here's a cleaned out copy.

Sorry the previous patch is a duplicate to this. I have attached the patch that builds up the mapquest results.

Please find attached a patch, there is no need to str_replace() spaces on $address variables, as is already handled by drupal_http_build_query().

merged both previous patches and fixed code formatting issues

So, what exactly is needed here, what would be the definitive patch? ;-)

I know that I'm only using patch in #1748412-19: Geofield Mapquest Geocoder , I have not tested all your patches for now. Shouldn't we combine everything (and make Mapquest nominatim really usable by the module)?

@simon-georges

I haven't looked at this issue in a while, but I know that in addition to #19 you'll also need #29, although as @Mac_Weber points out 29 is probably not handling encoding in an ideal way. However without something in that range the module won't work at all due to lack of API Key.

I haven't personally tested anything past that particular patch, but we've been running 29 in for over a year now without issue.

Ok, thanks for the feedback, I'll try to come up with something working for everybody.

The attached patch include all changes in this thread accept for API Key related changes. Those changes are not mapquest specific and should be addressed in another issue.

Specifically, the attached patch:

• Keeps the address from being URL encoded
• Keeps the country out of the 'q' parameter and adds a 'countrycodes' parameter
• Removes the limit of 1 so that all matches will be returned
• Adds all results to the point object so that they are accessible

I found that the patch in #43 was throwing a PHP warning if no results were returned in the request. I moved logic that performs a check for results sooner within the code, and rather than attempting to build geometries with no data, the module logs the request via watchdog. New patch attached.

Here's a version that returns address components, so you can also use this for reverse geocoding.

Also fixes missing 'q' in the url constructor so it actually returns results n stuff.

Also pass $options into the query. The field implementer expects this at least (countrycode)

Hi,

I applied the latest patch (geocoder-mapquest-nominatim-1748412-46.patch) but I get an ugly error:
Fatal error: Call to undefined function geocoder_mapquest_nomination() in /public_html/sites/all/modules/geocoder/plugins/geocoder_handler/mapquest_nominatim.inc on line 60
I'm using using Geocoder 7.x-1.2+18-dev to geocode from another field which is a Location field.

Hi,

I applied the latest patch (geocoder-mapquest-nominatim-1748412-46.patch) but I get an ugly error:
Fatal error: Call to undefined function geocoder_mapquest_nomination() in /public_html/sites/all/modules/geocoder/plugins/geocoder_handler/mapquest_nominatim.inc on line 60
I'm using using Geocoder 7.x-1.2+18-dev to geocode from another field which is a Location field.

@gge, I'm sure this is the issue:
https://www.drupal.org/node/2499771

I'm having an issue with the Mapquest service as well. The issue comes from the encoding of the q parameter. I was able to confirm this by debugging into the query and seeing the full url for a field containing the text "Flinders street Melbourne" was:

http://open.mapquestapi.com/nominatim/v1/search?q=Flinders%2Bstreet%2Bme...

Where it should be

http://open.mapquestapi.com/nominatim/v1/search?q=Flinders+street+melbou...

The google service works perfectly for me, I also noticed when stepping through the code that there's an issue with your comparison when it does the diff to decide if there is any point geocoding again (line 239 of widget.inc) one of the arrays has the keys 'format' and 'safe_value' and the other doesn't so it's always geocoding.

This patch fixed my issue, rolled against 7.x-1.2

#51 Works for us. Thanks!

Thanks!