Dom
Member
Registered: 13th Sep 03
User status: Offline
|
I'm on the hunt for a up-to-date geo PAF (Postcodes against their long/lats) database/dataset. In the past i've used the PAF data from Linuxbox.co.uk and it's been spot on but it hasn't been updated since 2009 and there's now a fair few missing postcodes.
I've had a look at the free offerings from OS and Geonames.org but both datasets need a reasonable amount of work to put it into a useful format - OS requires converting from eastings/northings (possible issue with accuracy?); Geonames isn't delimiter separated and is messy as hell.
So has anyone got any recommendations apart from handing over £4-6k to RM?
Alternatively does anyone know of free lookup sites that i could possibly scrape?
I know there are commercial postcode lookup services but you're charged per query and this would be the last resort.
Chars 
|
evilrob
Premium Member
Registered: 16th Mar 12
Location: Your mum's house
User status: Offline
|
I use postcodeanywhere for my geocoding requirements - got a great API if you need transactional geocoding for not silly money.
|
evilrob
Premium Member
Registered: 16th Mar 12
Location: Your mum's house
User status: Offline
|
What are you trying to do? Address autocompletion/validation from postcode or just getting lat/long from postcode?
|
evilrob
Premium Member
Registered: 16th Mar 12
Location: Your mum's house
User status: Offline
|
Javascript to convert eastings/northings to lat/long:
http://www.movable-type.co.uk/scripts/latlong-gridref.html
|
Sam
Moderator
Premium Member
Registered: 24th Dec 99
Location: West Midlands
User status: Offline
|
I think Google Maps API does this for free now.
|
Dom
Member
Registered: 13th Sep 03
User status: Offline
|
quote:
Originally posted by evilrob
What are you trying to do? Address autocompletion/validation from postcode or just getting lat/long from postcode?
Need Long/Lat for rudimentary distance calculations in search results and last resort is to use a paid API service to query missing (current 448) postcodes, also be a royal PITA to implement hence on the hunt for a direct-replacement or something i can easily merge with the current PAF data.
Sam - Google's free service is only for JS and even then you have to display a map (same for Bing etc). Their API lookup is a commercial product and they charge per query the same as Postcode Anywhere/other query based services.
[Edited on 10-12-2012 by Dom]
|
Steve
Premium Member
Registered: 30th Mar 02
Location: Worcestershire Drives: Defender
User status: Offline
|
You can download it all for free from somewhere osmaps Itthink
|
Dom
Member
Registered: 13th Sep 03
User status: Offline
|
quote:
Originally posted by Steve
You can download it all for free from somewhere osmaps Itthink
You mean Ordnance Survey?
I've already got their free Code-point data but it needs converting to WGS84 Long/Lat and it's a bit more farting around than i really wanted to do.
Was hoping there would be a decent direct-replacement supplier of PAF but apart from RM and their stupid prices i can't seem to find anything.
|
evilrob
Premium Member
Registered: 16th Mar 12
Location: Your mum's house
User status: Offline
|
You can do 2,500 Google Maps API calls a day, but as you say you've got to display a map - so you could scrape the coordinates, but that would be more work than just converting the Ordnance Survey bonky codes.
[Edited on 10-12-2012 by evilrob]
|
evilrob
Premium Member
Registered: 16th Mar 12
Location: Your mum's house
User status: Offline
|
I know you don't want to pay for it, but if you do end up going down that route, the Postcode Anywhere APIs are a piece of piss to implement and well documented IMO.
|
evilrob
Premium Member
Registered: 16th Mar 12
Location: Your mum's house
User status: Offline
|
Dunno if this is any good:
"Spreadsheet for coordinate calculations
From this page you can download a Microsoft® Excel 2002 spreadsheet which will carry out common calculations with coordinates. All of the concepts and formulae given in A guide to coordinate systems in Great Britain, which is the user manual for these calculations, are detailed in the spreadsheet. The guide gives some important warnings about the use of its formulae, which you should be aware of. There are also other projection functions such as scale factors, convergence and t-T in the spreadsheet that are not in the guide."
http://www.ordnancesurvey.co.uk/oswebsite/gps/osnetfreeservices/furtherinfo/spreadsheet.html
|
Dom
Member
Registered: 13th Sep 03
User status: Offline
|
quote:
Originally posted by evilrob
You can do 2,500 Google Maps API calls a day, but as you say you've got to display a map - so you could scrape the coordinates, but that would be more work than just converting the Ordnance Survey bonky codes.
It'd be messy as hell to try and implement and then scrape it.
I've got no issue purchasing PAF data, did it in the past with the Linuxbox dataset (£100 for 3m rows), just as long as it doesn't cost the earth (yet to find anything reasonable) and i can either do a straight swap or merge it with the data i already have.
Cheers for the help though Rob & co, much appreciated 
|
Ian
Site Administrator
Registered: 28th Aug 99
Location: Liverpool
User status: Offline
|
OS Code Point Open?
Bit of a faff to download but that should do.
I'd personally prefer northings and eastings for distance calcs, pythag on the x and y differentials is far less work than longs and lats on a spherical plane. Plus your answer is in meters which is also easier to work with.
Mine are inside the DB query so you can use the SQL ORDER BY on that column and you don't even need to process it inside the programming. Not sure I'd fancy that with longs/lats.
Also bear in mind that distance calcs are not accurate anyway so its not essential that you have absolute accuracy on the points. Unless the distances are very small then there's no real difference between going TR7 to AB32 than if you have the full postcodes for both places.
In that respect, API calls are far nicer because road distances are far closer to the truth. Just more difficult to do in bulk and order on.
[Edited on 11-12-2012 by Ian]
|
Ian
Site Administrator
Registered: 28th Aug 99
Location: Liverpool
User status: Offline
|
quote:
Originally posted by Dom
last resort is to use a paid API service to query missing (current 448) postcodes
You got that as a list somewhere?
|
Dom
Member
Registered: 13th Sep 03
User status: Offline
|
quote:
Originally posted by Ian
OS Code Point Open?
Bit of a faff to download but that should do.
I'd personally prefer northings and eastings for distance calcs, pythag on the x and y differentials is far less work than longs and lats on a spherical plane. Plus your answer is in meters which is also easier to work with.
Mine are inside the DB query so you can use the SQL ORDER BY on that column and you don't even need to process it inside the programming. Not sure I'd fancy that with longs/lats.
Also bear in mind that distance calcs are not accurate anyway so its not essential that you have absolute accuracy on the points. Unless the distances are very small then there's no real difference between going TR7 to AB32 than if you have the full postcodes for both places.
In that respect, API calls are far nicer because road distances are far closer to the truth. Just more difficult to do in bulk and order on.
Already managed to grab the OS Code Point DB but as mentioned need to convert to Long/Lat - the PAF data i purchased was long/lats (long/lats are to 7 decimal places, so pretty accurate), which in turn got merged with the RM PAF data (gained another 80k postcodes; again in long/lat), and the app was built around using long/lats so it'd be a royal PITA to change.
But so far, using the OS data looks to be the best option to fill in the bulk of the missing postcodes. Then perhaps use Postcode Anywhere (no idea what their TOS is on storing the data; most don't allow you to) to keep it up-to-date.
quote:
Originally posted by Ian
quote:
Originally posted by Dom
last resort is to use a paid API service to query missing (current 448) postcodes
You got that as a list somewhere?
Will have access to the DB again tomorrow, so yeah can grab a list of the missing postcodes - you think you might have the data?
By all means, if anyone wants the PAF data i have (again, it's circa 2009 but it's over 3m postcodes) then let me know and i'll host it somewhere.
|
Ian
Site Administrator
Registered: 28th Aug 99
Location: Liverpool
User status: Offline
|
No doubt you could hack in some sort of API call somewhere as a plan B to fill the blanks on the fly.
I was just curious to see what the missing rows were. I don't think I have long/lat or if I do it'll be the original massive data set before I trimmed out the columns I didn't need.
|
