Many listings are getting a 'Failed to Geocode' error and I think it is because they have a ')' at the end of the postal code, the listings that are geocoded, do not have the parentheses
Saving item 1640345 [1142.74 sec, 89.51 MB] [notice]
Item 437 of 500 [1142.74 sec, 89.51 MB] [notice]
Failed to Geocode: 601 Bobwhite Ct , Lodi, WI 53555) [1142.85 sec, [notice]
89.54 MB]
Saving item 1640152 [1142.88 sec, 89.57 MB] [notice]
Item 438 of 500 [1142.88 sec, 89.57 MB] [notice]
Failed to Geocode: N10828 Rainbow Terr , Fox Lake, WI 53933) [1142.99 [notice]
sec, 89.6 MB]
Saving item 1639300 [1143.01 sec, 89.64 MB] [notice]
Item 439 of 500 [1143.01 sec, 89.64 MB] [notice]
Failed to Geocode: 2107 Lewis St , Cross Plains, WI 53528) [1143.13 [notice]
sec, 89.67 MB]
Comments
Comment #1
jday commentedThe problem might have been a daily limit on geocoding...not sure - but I've run the import script several more times now and haven't gotten any failures.
Comment #2
jday commentedok so now I have a lot of listings that have not been geocoded and re-running the rets-import does not generate that info. I'm guessing it won't happen until something in the listing changes, is there a way to run the geocode separately from the import? like the photo processing?
Comment #3
jday commentedUPDATE
I added another field to collect during imports to see if that would trigger the geocoding for the listings missing that info, and it looks like my theory that the failures might be due to a daily limit was wrong. I continue to get the 'Failed to Geocode' errors for any listing that has a right parentheses at the end of it.
Comment #4
camidoo commentednot really sure where that right parentheses is coming from. Yea there isn't really a way to re-geocode like re-downloading the images. However, when you add a field mapping it will reset the hash to '', causing the import process to re-process that listing. So you can either add and then remove a field mapping, or open mysql and run a query to UPDATE drealty_listing SET hash = ''; or something similar to force all the listings to be reprocessed.
Which geocoder handler are you using to do the geocoding? I used to use Google, however they have a limit of 1500 per day, i have started using Yahoo! Placefinder and the results are just as accurate, and they're daily limit is 20k or something like that.
Comment #5
kevinquillen commentedThis happens to me too with Innovia vendor for properties that already have lat/long coordinates saved. When I update the listing in drush, it throws that error for every listing. If I clear coordinates out and rerun, no error.
Comment #6
jday commentedI am using Yahoo placefinder and the listings that are failing have not previously been geocoded. Looking at the database, the latitude and longitude fields are NULL or 0.000000 and the zip code fields do not have the mysterious parentheses that appears in the failure messages.
UPDATE:
After adding another field and running the import again, I'm seeing that the failures happen on listings that have '0.000000' values in the lat/long fields. So I'm guessing the RETS feed is supplying those values...? and therefore sabotaging the geocoding by making the geocoding function believe the task is already done....
Comment #7
kevinquillen commentedI saw that happen too. What's the harm in overwriting the value or having them set as NULL instead of 0?
Comment #8
camidoo commentedactually, the geocoding does overwrite the values...
basically the when a listing is processed it loops through all the mappings that are defined...
Once it's done doing the field mappings, then it checks to see if we are geocoding this 'class'
The two lines above essentially overwrite the value of anything that was pulled from the rets feed for lat and lon.
** well i figured out where the ')' was coming from this is on line 474 and 477 of drealty.daemon.php.
So, something is causing the geocoding to fail. Since you are using the yahoo placefinder, did you set the yahoo app id in the:
admin/config/content/geocoder
reason i ask is because, it won't complain if you don't have it set and will 'act' like it is set, but fail to geocode anything.
Let me know.
Comment #9
jday commentedYes I have my api code from yahoo entered into the geocoder admin page. Out of 11K+ listings, 8,480 are failing and 1,914 are geocoded correctly...
I've copied some of the failed addresses into the map.yahoo.com site directly and it found most of them, two addresses it couldn't find. Maybe I'll try google and see what happens.
Comment #10
camidoo commentedI think I 'may' have figured out the problem.
right now i'm concatenating the address out of these fields:
However if you've mapped:
They never get put into the address, so the address should be put together like this:
Let me know if you've mapped either of those fields. If so i'll make the changes post haste and get them pushed up to see if it works. If not, then back to the drawing board.
Comment #11
jday commentedinteresting...I do have street_dir_suffix mapped, but now I'm thinking that since the feed only has one value for direction it should be mapped to the street_dir_prefix field (attached screen shot)
Comment #12
camidoo commentedwell i just pushed up a new release that should address not mapping those fields. so if they're mapped they'll now be included:
is how the address is built now.
Comment #13
camidoo commentedI'm also thinking that it might not be a bad idea to include an option for encoding based on the unparsed_address or encoding based on a concatenated address, i'll add that to the todo list.
Comment #14
jday commentedThanks for your attention to this camidoo
I moved the mapping of the address direction to the street_dir_prefix field but looks like it's coming in before and after the street name
Comment #15
camidoo commentedoh snap, i see the problem, thanks!
Comment #16
camidoo commentedthere's a new release up there or you can grab the latest from git, via a 'git pull'.
Additionally, can you check to ensure that you don't have the field 'L_AddressDirection' mapped twice?
you can open phpmyadmin or another mysql tool and run this query:
and see if it's there more than once.
Comment #17
jday commentedwoe is me
1. I unchecked the Street Dir Suffix box, deleted the mapping
2. Checked Street Dir Prefix box, added the mapping
3. Ran the import - got the direction before and after the street name
4. Went into the database and deleted the 'street_dir_suffix' fields from the drealty_fields, drealty_field_mappings, and dreatly_listing tables
5. Ran the import
6. Error ->WD php: PDOException: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'base.street_dir_suffix' in 'field list'
So I'm guessing I'll have to trash the db and start from scratch, which leads me to a new problem with the connection export/import feature for which I'll start a new issue
Comment #18
jday commentedAfter dumping the database and starting again, it looks like adding those fields to the $geoaddress has fixed the 'failed to geocode' errors I was getting. Camidoo is a genius.
Comment #19
camidoo commentedGlad to hear that it's now working for you!
Can anyone else confirm that this is working for them?
Comment #20
stockliasteroid commentedI was having issues in my setup where the address_number and address_street were right up against each other... This was with beta18. In my case it was because the double-space replace in the geoaddress is replacing double spaces with no space at all. It seems like it should replace a double space with a single space, correct? Thus:
// remove any double spaces
$geoaddress = str_replace(" ", " ", $geoaddress);
That fixed my issue. anyway...
Comment #21
camidoo commentedYea, i feared that may cause problems, there has to be a more elegant solution in how i'm putting together the address. I'll do some research and see what i can come up with.
Comment #22
kevinquillen commentedSeems like this should be closed? 3.x uses AddressField/Geo.
Comment #23
kevinquillen commentedUsing AddressField + GeoField + Geocoder has been pretty straight in 3.x. Please re-open if this continues to occur.