Flutterby™! : Yahoo's Open Location project 2009-04-15 15:58:26.774948+00

Yahoo's Open Location project

2009-04-15 15:58:26.774948+00 by Dan Lyke 3 comments

One of the things I've hacked together recently is a little Perl script that searches through Petaluma's Accela Citizen Access database, extracts recent building permits, geocodes them, and outputs them to a GeoRSS feed for inclusion into various online map services.

It's kind of a proof of concept for ways to re-think how Petaluma is approaching its data publishing. There are strong and weak reasons why raw data isn't getting published right now, and I'd like to find ways to build up citizen created applications around information that the city has so that we can set up a symbiotic ecosystem for publishing data and interpreting it.

So I've started to pay attention to geography stuff, especially where geocoding services or map data is available for free for inclusion into such apps. This Where 2.0 Preview - Tyler Bell on Yahoo's Open Location Project looks like it might have some interesting leads, even though it starts out with the wonderfully ignorant:

Location can be a vague concept to pin down. To a surveyor, location means latitude and longitude accurate to a few millimeters ...

Thanks to experiences with GPS accuracy and talking with Eric, I've found that I can gain tremendous credibility with people who locate stuff for a living if I acknowledge that latitude and longitude are generally not the primary way location gets described. Especially to surveyors (one of the fun comments out of a recent meeting was "surveyors never tell you where something is, they issue an opinion..."). Luckily, Tyler Bell, the subject of the interview, seems better informed.

Bonus link: Aaron talks about Python, maps, clustering Flickr apps, and Amazon EC2, which includes this great quote from John Allspaw:

You know what is ASTRONOMICALLY FUCKING EXPENSIVE? Leaving an EC2 instance running for two weeks doing nothing, by mistake.

Careful with your cloud computing, folks.

On Surveyors: there are really two kinds - the kind that survey for engineering purposes and the kind that survey for legal purposes. Engineering surveyors, of course, need to make sure that both ends of the bridge match up. Legal surveyors just provide a legal opinion of where something is backed up by their professional insurance.

But as a computer geek, you should beware of the pitfalls that come with the easy index of spatial location. By declaring where something is you are not just fixing it's location, you are also fixing it's existence at that location. This is why GISers are constantly struggling with ontologies - both the philosophical and the comp sci versions.

Look at the recent brouhaha over the LAPD Crime Maps. Not only is the location wrong for the crimes, but there is also potential impact on the incorrect location.

Yeah, that LAPD issue is exactly the sort of thing is exactly why the city's computer folks balk at providing data that's re-usable. In my little script, sometimes the geocoder's failing to find the location, and I need to do something that's intelligible to the user rather than simply drop the code from the RSS, lest all the addresses on south McDowell Blvd fail to show up on the final maps.

And this is also why I'm extremely sympathetic to arguments that systems which take an hour or two of Perl coding to implement can cost a hundred thousand bucks to deploy.

#Comment Re: made: 2009-04-15 19:10:09.968007+00 by: Mark A. Hershberger

To get around the "oops, I left it on" problem, you can Use Hadoop+S3 to deploy EC2 clusters on demand. They'll spin down automatically when you're done with them.