I wrote more about this for the Knight Digital Media Center at USC site: US Census upgrades American FactFinder tool, new data coming soon | Knight Digital Media Center.

,http://m.cnn.com/primary/_E5uyqH-isfp12PXn2

Recently I wrote about how a Los Angeles Police Dept. geocoding data glitch yielded inaccurate crime maps at LAPDcrimemaps.org and the database-powered network of hyperlocal sites, Everyblock.

On Apr. 8, Everyblock founder Adrian Holovaty blogged about the two ways his company is addressing the problem of inaccurate geodata.

1. Latitude/longitude crosschecking. “From now on, rather than relying blindly on our data sources’ longitude/latitude points, we cross-check those points with our own geocoding of the address provided. If the LAPD’s geocoding for a particular crime is significantly off from our own geocoder’s results, then we won’t geocode that crime at all, and we publish a note on the crime page that explains why a map isn’t available. (If you’re curious, we’re using 375 meters as our threshold. That is, if our own geocoder comes up with a point more than 375 meters away from the point that LAPD provides, then we won’t place the crime on a map, or on block/neighborhood pages.)
2. Surfacing ungeocoded data. “Starting today, wherever we have aggregate charts by neighborhood, ZIP or other boundary, we include the number, and percentage, of records that couldn’t be geocoded. Each location chart has a new “Unknown” row that provides these figures. Note that technically this figure includes more than nongeocodable records — it also includes any records that were successfully geocoded but don’t lie in any neighborhood. For example, in our Philadelphia crime section, you can see that one percent of crime reports in the last 30 days are in an ‘unknown’ neighborhood; this means those 35 records either couldn’t be geocoded or lie outside any of the Philadelphia neighborhood boundaries that we’ve compiled.”

These strategies could — and probably should — be employed by any organization publishing online maps that rely on government or third-party geodata.

Holovaty’s post also includes a great plain-language explanation of what geodata really is and how it works in practical terms. This is the kind of information that constitutes journalism 101 in the online age.

(NOTE: I originally published this post in Poynter’s E-Media Tidbits.)

lacrime

LAPDcrimemaps.org has some recently revealed geodata flaws.

Crime maps are one of the most popular and (in urban areas) ubiquitous types of geo-enabled local news — and they’re a staple of the Knight News Challenge-funded project Everyblock. This data comes from local police departments — but how reliable is it?

On Sunday, the Los Angeles Times reported a problem with the Los Angeles Police Department’s online crime map, launched three years ago…

“LAPDcrimemaps.org is offered to the public as a way to track crimes near specific addresses in the city of Los Angeles. Most of the time that process worked fine. But when it failed, crimes were often shown miles from where they actually occurred.

“Unable to parse the intersection of Paloma Street and Adams Boulevard, for instance, the computer used a default point for Los Angeles, roughly 1st and Spring streets. Mistakes could have the effect of masking real crime spikes as well as creating false ones.”

Apparently the LAPD wast not aware of the error until alerted by the Times…

LAPD spokeswoman Mary Grady told the Times that “the department will work with its contractor to make the map as accurate as current technology allows.”

Also, the Times reported: “Alerted to the findings, Lightray Productions, the contractor that designed the LAPD site at a cost of at least $362,000, has promised to fix the problems. …One reason the errors were not caught earlier may be that the LAPD site retains crimes for only six months and allows viewers to see only a seven-day period at a time. The presentation makes some trends, such as the large accumulation of crimes mapped at Civic Center, more difficult to spot.”

Distorted or erroneous geodata, especially from official sources like police departments, can have ripple effects. In this case the LAPD crime data was automatically pulled into, and displayed by, Everyblock — an experimental project funded by the Knight News Challenge. (UPDATE: Everyblock has since implemented some corrective measures to spot and fix source geodata problems in its crime maps.)

In the All Points Blog from Directions Magazine, Adena Schutzberg noted:

“While the Times article highlights some key points about the process of geocoding and why the error was not found sooner (the app only shows data back a week, making such spikes less obvious) it missed out on some other points including:

• Different applications use different geocoding algorithms.
• Different applications use different data against which to geocode.
• Sharing raw data (vs. maps) can help identify such errors.

Schutzberg concluded: “My main question is this: Everyblock took the same data feed for its L.A. maps, and it seems, ended up with same inaccuracies. Is that because they use the same geocoding and data against which to geocode? That’s not clear from the Times article.”

If your news organization is using geodata to create interactive online features, you might want to consider ways to double-check for possible accuracy issues, perhaps by checking the results yielded by a different tool set to see if and how it handled the data differently.

(NOTE: I originally published a slightly different version of this article in Poynter’s E-Media Tidbits.)