The Chicago Tribune recently reported that it has halted a “short-lived research project in which the Chicago Tribune solicited responses from current and former subscribers to descriptions of Tribune stories before they had been published.”

The project — a collaboration between the paper’s editorial and marketing departments — was stopped because reporters raised journalistic concerns. Originally it had only surveyed selected “would-be readers” about general topics and previous Tribune coverage. But in the last two weeks, participants had begun being surveyed about their preferences on synopses of stories currently in the works.

In all, 55 reporters and editors voiced their complaint in a letter to Tribune editor Gerould Kern and managing editor Jane Hirt. The letter “expressed concern that providing story information to those outside the newsroom prior to publication seemed ‘to break the bond between reporters and editors in a fundamental way.’”

Here’s more detail about how the research was conducted: “Surveys were sent by e-mail to around 9,000 would-be readers on two occasions. About 500 responded to each, indicating which of 10 story ideas they preferred. Kern said the stories ‘tended to be news features,’ and the results never made it to him or had any impact in how stories were handled.”

I can understand the reporters’ complaint if their story ideas were shared outside the newsroom without their prior knowledge and consent. However, if that consent can be obtained, I personally think this type of research could be surprisingly useful. Especially if the people being surveyed truly represent younger people (i.e., the news organization’s future market) as well as demographics that historically have not been well served by the news organization…

I’d even take it farther — rather than just vote on a packaged list of story ideas, I’d survey them about which angles on those stories would most interest them. And I’d give them room to critique the story ideas, and get new story ideas. A combination of qualitative and quantitative data could shed light on how news organizations can make their news more relevant by being willing to step outside their comfort zone.

Chicago Reader sees it differently, however. There, Michael Miner wrote:

“Tell any self-respecting reporter that the subject of his or her latest work in progress just laid an egg with a focus group, and the reporter will reply, ‘Maybe so, but wait till they see what I do with it!’ (While thinking, ‘What in God’s name has happened to our business?’)”

…I don’t doubt that many longtime newspaper reporters would feel that way. But I don’t think it has as much to do with “self respect” as it does with pride and fear. It seems to me that many journalists prefer to only present their perfect, finished work to the public in order to pretend that their reporting is more independent and infallible than is actually the case. They’re easily threatened by the thought that someone might witness their messy sausage-making process. It used to be that this pretense of perfection was assumed to support the veneer of credibility. In fact, these days being aloof from your community and pretending you’re perfect only undermines credibility.

When Jim Romenesko noted this news a few days ago, Poynter reader Gary McCardle commented: “Marketing people do what marketing people always do. Aside from special themed sections, don’t let marketing people know about stories in advance of publication.”

That comment lays bare the distrust of marketing — and perhaps indirectly of efforts to involve community members up front in journalistic processes — so deeply ingrained in traditional mainstream newsroom culture. And I’d dare say that it’s a big reason why news organizations are struggling for relevance and revenue these days. It’s hard to update your business model when an important part of your organization is inherently wary of market research.

NOTE: This is an expanded version of an article I originally published in Poynter’s E-Media Tidbits.

Recently I wrote about how a Los Angeles Police Dept. geocoding data glitch yielded inaccurate crime maps at LAPDcrimemaps.org and the database-powered network of hyperlocal sites, Everyblock.

On Apr. 8, Everyblock founder Adrian Holovaty blogged about the two ways his company is addressing the problem of inaccurate geodata.

1. Latitude/longitude crosschecking. “From now on, rather than relying blindly on our data sources’ longitude/latitude points, we cross-check those points with our own geocoding of the address provided. If the LAPD’s geocoding for a particular crime is significantly off from our own geocoder’s results, then we won’t geocode that crime at all, and we publish a note on the crime page that explains why a map isn’t available. (If you’re curious, we’re using 375 meters as our threshold. That is, if our own geocoder comes up with a point more than 375 meters away from the point that LAPD provides, then we won’t place the crime on a map, or on block/neighborhood pages.)
2. Surfacing ungeocoded data. “Starting today, wherever we have aggregate charts by neighborhood, ZIP or other boundary, we include the number, and percentage, of records that couldn’t be geocoded. Each location chart has a new “Unknown” row that provides these figures. Note that technically this figure includes more than nongeocodable records — it also includes any records that were successfully geocoded but don’t lie in any neighborhood. For example, in our Philadelphia crime section, you can see that one percent of crime reports in the last 30 days are in an ‘unknown’ neighborhood; this means those 35 records either couldn’t be geocoded or lie outside any of the Philadelphia neighborhood boundaries that we’ve compiled.”

These strategies could — and probably should — be employed by any organization publishing online maps that rely on government or third-party geodata.

Holovaty’s post also includes a great plain-language explanation of what geodata really is and how it works in practical terms. This is the kind of information that constitutes journalism 101 in the online age.

(NOTE: I originally published this post in Poynter’s E-Media Tidbits.)

However, many news organizations still could take a lesson from the second item on HuffPost‘s list:

“2. Do research and include links to back it up. Whether you are referencing a quote, statistic, or specific event, you should include a link that supports your statement. If you’re not sure, it’s better to lean on the cautious side. More links enhance the piece and let readers know where you’re coming from.”

It amazes me how often I still see mainstream news stories which completely lack links, or which ghettoize links in a box in a sidebar or at the bottom of the story…

In online media, links enhance credibility and invite engagement. Yet many (perhaps most) major news organizations (including the Associated Press, Wall St. Journal, and USA Today) still include few or no outbound links to sources or context from their stories.

I know from speaking to many, many journalists that in some news organizations, outdated print- or broadcast-focused content management systems make it cumbersome for reporters to add links to stories. In other cases, reporters either don’t know how to add links, or don’t bother to do so. And in a few cases, editors still actively discourage reporters from adding links to stories due to mistaken ideas about what drives online traffic and demonstrates value to readers.

Look at this from the perspective of the community you’re trying to engage online: Dead ends are bad news on the internet. A story without source or context links (especially obvious ones) may appear suspect, as if the news org hopes to discourage independent followup. Yes, it’s a good idea to link to your own related stories — but if you only link to your own news from your own news, you risk looking like an echo chamber. The more value you offer (which includes useful external links), the more likely it is that your online news will attract repeat traffic, inbound links, and personal referrals.

Of course, links are not everything. It’s true that original research (including interviewing) still matters in journalism. It’s also true that not every fact reported has an online link for reference. That said, much (perhaps most) of the information and context that journalists gather and assemble into most stories does indeed have some sort of primary online reference.

Avoiding those links implies hubris (the impression that the news organization is pretending to be the original source/gatherer of all the information presented), laziness (the news org couldn’t be bothered to link), or cluelessness (the news org doesn’t recognize the value of links).

Also, when linking to sources of context, quality counts. I’d have preferred it if HuffPost had stated a preference for links to primary sources, and qualification for any link that is not a primary source. But still — when soliciting news and reporting from amateur journalists, this list is an adequate starting point.

(NOTE: I originally published another version of this article in Poynter’s E-Media Tidbits.)

Huessner interviewed me for this story because I’ve been blogging about it on Contentious.com and on E-Media Tidbits. She chose to include a few highly edited and interpreted quotes from me that I think grossly misrepresent my own views and the character of our conversation.

Yeah, being a journalist, I know that no one is ever completely happy with their quotes. I’ve been misquoted plenty in the past, and normally I just roll with it. But this particular case is an especially teachable moment for my journalist colleagues in mainstream media about understanding and covering the role of social media in today’s media landscape.

Today’s a pretty busy day for me, but I didn’t want to let this go unsaid any longer. So I made a little Seesmic video response to this story. Here I am speaking strictly for myself — not on behalf of any of my clients or colleagues. Yes, I am very emphatic here and somewhat critical. Please understand that my frustration is borne of seeing this particular problem over and over again.

For example, see Inquisitr.com: Indian Government trying to block Twitter as Terrorists may be reading it.

Rumors — even fairly innocuous ones — really bug me. Mainly because they’re so easy to prevent!

I’m trying to track this particular rumor down, but haven’t been able to confirm anything yet. At this point I’m skeptical of this claim. Here’s what I’ve found so far…

Newstrack India reports:

“[Yesterday] evening, there were suddenly a lot of ‘tweets’ reporting that the Indian government had asked that there should not be online updates of military operations against the holed-up terrorists, citing a BBC news source. But, the BBC actually quoted ‘tweets’, which in fact had no independent confirmation.”

BBC’s timeline of the Mumbai attacks (which is an excellent resource, by the way!) reported at 11:08 GMT:

“Indian government asks for live Twitter updates from Mumbai to cease immediately. ‘ALL LIVE UPDATES – PLEASE STOP TWEETING about #Mumbai police and military operations,’ a tweet says.”

The BBC included no link to that alleged tweet from the Indian government. A simple Twitter search for “please stop tweeting” showed the earliest occurrence of this phrase in connection to the Mumbai attacks came from the MumbaiUpdates account, which appears to be run by a Twitter user named Mark Bao — a high school junior based in Boston, who apparently is not in Mumbai at the moment.

Several hours ago, MumbaiUpdates tweeted: “ALL LIVE UPDATERS – PLEASE STOP TWEETING about #Mumbai police and military operations.”

Prior to that, he tweeted (in chronological order):

1. “Due to military action happening very soon, @mumbaiupdates may have little information to report to protect the rescue operations”
2. “I am not updating on any details about #mumbai operations until futher notice to protect the operatoin”
3. “Indian government is asking that the twitter search page #mumbai be shut down.”
4. “or possible clarification: to just stop live updating about the situation pertaining to #mumbai”

UPDATE: Bao just e-mailed me to let me know that his tweet was not the original report on this event. Here’s what he said:

“The rumour started on via another twitter post that retweeted from another person that was a trusted source IN mumbai. Later, it was confirmed on video that the police wanted live updates of the operations to be stopped, though they did not mention the hashtag #Mumbai, though they asked media outlets to stop reporting live.

“The purpose of [the MumbaiUpdates] stream was to disseminate info from the CNN-IBN, NDTV and those twittering from Mumbai. With any news reporting and re-reporting it’s possible errors got in the way. I’m sorry if it caused any confusion.

“If anything, even if NDTV and CNN-IBN were still reporting, it is best practice, and I think justified, to stop tweeting and disseminate more information on the operation that could be spread and could be useful to those that we don’t want to let know the info.”

Also, on Twitter he wrote:

“It was confirmed by Mumbai police on video that they don’t want live updates. Don’t think they mentioned Twitter but it is possible that they did. If not, then that is the rumour that evolved, yet still good practice.”

I’m really glad that Bao elaborated on this, and I’d like to say that I think he did a good job with quickly starting the MumbaiUpdates account to aggregate information on the attacks in India.

Parsing out Bao’s response, it looks like we still don’t know the exact source of this rumor’s first report, but apparently it might have come from a Twitter user in Mumbai. He also said it was “confirmed by video” — but we don’t know where that video was, whether that confirmation was an on-camera statement by police, whether someone was relaying on video information they’d gotten first-hand from the police, or whether someone was simply repeating an unsourced rumor on video.

If you have any further information on this (especially specific links, cites, video clips, etc.), please leave that information in a comment to this post.

TAKEAWAY LESSONS

Media is increasingly unmediated. People are communicating directly, on a global level. We don’t all have to be journalists — but we’d all be better off by adopting stronger media-literacy skills.

Specifically, when you hear something that sounds surprising or important, CHECK OR ASK FOR THE PRIMARY SOURCE before you share the news. It’s not hard to do, and it’s a crucial step.

If something just sounds like common sense (like, “Hey, tweeting details of police movements here might endanger police and hostages, so don’t do that!”), there’s no need to appeal to authority (i.e., saying the police said so) to make people listen. A true common-sense message stands on its own — and in social media like Twitter, it could  carry more credibility as a peer recommendation than if positioned vaguely as an order from “above.”

In this case, many, many well-meaning Twitter users simply repeated the alleged government/police request as if it were established fact. This could cause ripple effects in future interactions between the Indian police and the public (in person and online). There’s a power dynamic in play here that deserves attention and care.

Of course, if an important primary source (like a government official) does offer crucial or interesting information, attribute it clearly. Just a like when professional journalists rely on source documents or press releases, transparency counts! It doesn’t take much time to include a link in your tweet, or just say you heard it firsthand.

Rumors and misinformation, even if well-meant, don’t help. In this case, what if the Indian government made no such request regarding Twitter? What if this call for social media restraint actually arose from concerned Twitter users? Actually, that might be far more interesting than a governmental request.

Imagine the precedent that a true government or police request regarding live tweeting might set for possible future police policy or requests during other events, such as political demonstrations, natural disasters, or a food riot.

Finally, if there was a police or government request, it may have had nothing to do with social media. It’s possible that any official move to get people to stop tweeting details of police/military location and actions, or victims’ locations and circumstances may have actually been a side effect of incautious TV coverage. Many people in India and around the world were watching network TV coverage (especially NDTV and IBN) and tweeting what they saw.

I think in this case it would be useful to know whether the police were mainly requesting cooperation from TV news organizations, or from individuals with cell phones. If the latter, that might mark an interesting turning point in the intersection of government, public safety, and free speech.

——————

NOTE: The original version of this post included the following information about my initial assessment of the situation and attempt to get clarification:

So far I can tell, the source for this alleged request by the Indian government is someone based in the U.S. who is monitoring the situation by remote online. He did not cite or link to a primary source for his allegation. It’s unknown whether he got this news firsthand, is repeating what he heard secondhand, or simply made it up. (I’m not saying he would fabricate that info; I’m just saying it’s possible that he could have done so — and that possibility needs to be ruled out before making this news worthy of repeating as fact.) On that basis, I personally would not repeat this rumor as fact.

That’s why, just now I asked Bao about his source: “@mumbaiupdates: What’s the source of your info that Indian government was seeking to curtail tweeting about #mumbai? Link or cite, please?”

UPDATE: I just received this response from Bao:

MumbaiUpdates: “@agahran ~14 hours ago police were asking that the live updates (incl from media) stop. not sure if it is still in effect.”

…Which really doesn’t answer my question. So I just asked for clarification:

@mumbaiupdates My question is, did you get that info 1st-hand? If not, what’s yr source? Also, did the police specifically mention Twitter?

…Soon after Bao transmitted his “PLEASE STOP TWEETING” request, he noted via MumbaiUpdates:

• “Requesting that if live updates are stopped, that when operations are happening, that NDTV and CNN/IBN stop broadcast also. #Mumbai”
• “This is exactly what #Mumbai doesn’t need: a certain tv station following the configuration of the police. That’s what I’m getting at.”
• “SUCCESS – the NDTV website is no longer broadcasting live video from the #Mumbai front. Thank you NDTV.”
• “TV MEDIA, BE RESPONSIBLE. RT @MumbaiAttacks CNN-IBN just gave out a room number from a guest that called them. What are they thinking?”
• “If in #Mumbai pls call +91-120-4341818, or if anywhere email editor@ibnlive.com to tell to stop broadcstng operations info.”

Yesterday I wrote about how the New York Times missed an obvious opportunity for transparency by failing to link to (or publish) source documents released during a court case.

But also, a recent flap in Columbia Journalism Review has got me thinking about transparency. This flap concerns the role of press releases in science journalism. Freelance journalist Christine Russell kicked it off with her Nov. 14 CJR article, Science Reporting by Press Release. There, she wrote:

“A dirty little secret of journalism has always been the degree to which some reporters rely on press releases and public relations offices as sources for stories. But recent newsroom cutbacks and increased pressure to churn out online news have given publicity operations even greater prominence in science coverage.

“‘What is distressing to me is that the number of science reporters and the variety of reporting is going down. What does come out is more and more the direct product of PR shops,’ said Charles Petit, a veteran science reporter and media critic, in an interview. Petit has been running MIT’s online Knight Science Journalism Tracker since 2006. …In some cases the line between news story and press release has become so blurred that reporters are using direct quotes from press releases in their stories without acknowledging the source.

“This week, Petit criticized a Salt Lake Tribune article for doing just that. In an article about skepticism surrounding the discovery of alleged dinosaur tracks in Arizona, the reporter had lifted one scientist’s quote verbatim from a University of Utah press release as if it had come from an interview. ‘This quote is not ID’d as, but is, provided by the press release,’ Petit wrote in his critique. ‘If a reporter doesn’t hear it with his or her own ears, or is merely confirming what somebody else reported first, a better practice is to say so.’” (Note: I added the direct links to the article and release here.)

In other words, Petit is arguing for transparency. He recommends using extra words as the vehicle for transparency (i.e., adding something like “according to a university press release”). That is indeed a useful tactic. But we have more tools than words — we have links…

Petit called for transparency on the basis of professional ethics. That’s one very good reason to be transparent. But there’s also a practical consideration: avoiding embarrassment.

Your audience expects transparency. Furthermore, if they catch you not being transparent, they will “out” you — probably in a very public, findable, documented, linkable, and not-fun way. This will not help your credibility.

So if you’re quoting from anywhere, including a press release, DO cite your source and link back to it. Don’t be lazy about finding the link — almost every press release is published on the web, at least in PDF format. But if you really can’t find it online (or if it’s presented in a stupid way that’s not linkable, like Flash or a Javascript-driven pop-up window), then post the release on your own site or blog and link to that. I’m virtually certain you won’t get into copyright trouble for that with a release. But if you are quoting from copyrighted material (like a white paper or report), then post an excerpt and provide specific citation info.

WHY DON’T MORE NEWS ORGS LINK TO PRESS RELEASES?

…I know that many talented, ethical journalists in the mainstream media do lift quotes and info from press releases without citing the release specifically. Usually, the closest that they come is saying: “According to company X” when citing facts. Almost never do I see a link from a mainstream news org to a press release.

There are many possible explanations for this — from journalists not wanting to take a moment to look up links, to news orgs using content management systems which make it hard or impossible to create links from within stories, to editors discouraging press release references for “style” reasons.

But I suspect that a core reason for this mild obfuscation is about appearances: Though they rarely admit it, news orgs and journalists dislike revealing how much of the everyday practice of journalism simply is not rocket science. Today, anyone with a computer can quickly acquire most of the source information which journalists rely upon.

And Russell’s correct that newsroom cutbacks only make it more tempting and practical to lift from press releases. Not just because of lack of time — but for a need to bolster the appearance of their own importance.

DON’T TAKE RESPONSIBILITY FOR OTHERS’ STATEMENTS

In fact, citing and linking to press releases isn’t just more transparent; it’s also safer. Not all press releases are created equal. Some are high-quality, well-sourced, and responsible — and they’re frank about their own spin, agenda, or interests. Others are, um, not.

When you lift quotes or info from press releases or other sources without attribution, you’re implicitly vouching for the accuracy and authenticity of those statements. In effect, you’re saying “This is what this person said,” or “This statement of fact or context is correct” — even though that information is actually secondhand. That’s not always a wise bet. Everyone messes up sometime. Just ask Judy Miller.

I was tipped off to the CJR flap over press releases by my colleague Earle Holland, a public information officer at Ohio State University who penned a sharp rebuttal to Russell’s salvo. CJR ran it on Nov. 19: Press Release by Science Reporting. Holland wrote:

“As to the quotes I’ve included in the [research-related] press releases I’ve done, they’re all direct statements by the sources, approved by the sources, and aren’t vetted by anyone else. No administrator okays them, nor do PR gurus spin them in any direction. …Also, in most cases, the research in question is work I’ve followed for years, giving me the same kind of knowledge advantage that a beat reporter has over a general assignment reporter in the newsroom. Does anyone really believe that a reporter’s blind call from even the most prestigious news media will yield the kind of information that comes from a reporting relationship that’s grown over years? I don’t think so. The last decade or so has seen top science PIOs shift their prime goal from coverage to credibility, since they know that the former depends on the latter.”

I’d argue that conscientious PR is in fact a valuable kind of news reporting. It’s not traditional objective journalism, but that doesn’t make it worthless. However, all PR should be presented as what it is: information (probably promotional) from an involved party.

When an organization publishes a press release on their own site, they’re being honest about that context, and they’re taking responsibility for their own statements. When a news org fails to attribute or link to that same press release, they’re misrepresenting context and assuming responsibility for the quality of another party’s information.

Personally, I’d rather not gamble on whose PR is high-quality. That can vary considerably by organization, PR person, time, and topic. If I don’t have time to or cannot contact the source directly, I’m all for passing the buck by linking back to the release. I’d rather be faulted for not confirming quotes directly than for glibly passing them along, implying I’d gathered or confirmed them independently.

…For that matter, this caution also applies to repeating quotes or other information from traditional news stories. Don’t blindly assume that the reporter got the quote or facts right. Instead, say: On [DATE] the Podunk Tribune quoted Joe Schmoe as saying… and link to the article.

Anyway, that’s my take on this issue. What’s yours? Please comment below.

Public documents are the crux of this corruption story — specifically, “e-mails and internal documents from Johnson & Johnson made public in a court filing.”

The article included lots of detailed background on this complex case. However, it failed to supply or link to the source documents — or even cite the case (court, case name, docket number) in a way that would allow interested people to find the documents on their own.

I see this a lot, and it confounds me. Here, the New York Times evidently believes its readers are savvy enough to understand the risks of commercial interests undermining scientific research and — in this case — possibly putting kids’ physical and mental health at risk.

…But they expect me to just take their word about what those documents said? They don’t think I’d care to see the original context in which the statements they quoted were made? They don’t even think I might want to be able to look up the documents, or follow the case?

Obviously, the New York Times has these documents. Also, these documents are public information — so you don’t have to worry about breaking copyright or confidentiality. So why didn’t the Times simply present them?…

Assuming these documents are available online, the Times could have linked to them, either from the story or in a sidebar. If not, they could scan the most relevant ones and post them as downloadable PDFs. Or at the very least, they could cite the court case well enough to facilitate independent follow-up.

But no. The article doesn’t even say which court is hearing this case.

Well, screw that!

If you’re interested in this case (which involves Johnson & Johnson, Massachusetts General Hospital, the famed child psychiatrist Joseph Biederman and the controversial antipsychotic drug Risperdal often prescribed for kids diagnosed with bipolar disorder), here are the documents. I got that MS Word file, which contains scans of the released documents, from the blog Pharmalot (run by journalist Ed Silverman).

Took me five seconds in Google to find that. Still, why did the NY Times make me turn elsewhere?

Unfortunately, those docs don’t indicate the court case information in any way that’s easily evident to a layperson like me. So I Googled around and quickly found the class action suit. The complaint document for the suit indicates the case was filed in the US District Court of NJ as two civil actions: 3:06-cv-03044-FLW-JJH, and 3:07-cv-02224-FLW-JJH.

WHY BOTHER?

Is this more detail than most people would want? Probably. But providing that information and making those links inobtrusively demonstrates a willingness not just to inform, but to empower.

Providing options for action is a service. It demonstrates awareness and respect for the agency of readers, many of whom aren’t nearly as passive as they once were assumed to be. And it doesn’t have to clutter the story for more casual readers.

It’s the kind of touch that makes an impression. In short: it’s a brand-builder.

Right now, mainstream news organizations are losing their audiences. Little touches like this can make a news brand stand out and earn continued respect based on today’s criteria. So if you already have source information, why not share it?

Again, it confounds me why I don’t see more mainstream news orgs routinely requiring source links. This should not be optional.

I’d have noted my consternation directly to the NY Times but — surprise — they don’t allow comments on their site. I have e-mailed Harris via the Times site to request his input. Hopefully he’ll respond in a comment here or via e-mail.

This series is a work in process. I’m counting on Contentious.com readers and others to help me sharpen this discussion so I can present it more formally for the Knight Commission to consider.

So please comment below or e-mail me to share your thoughts and questions. Thanks!

To compensate for our government’s human-unfriendly info systems, some people have developed civic info-filtering backup systems: news organizations, activists, advocacy groups, think tanks, etc.

In my opinion, ordinary Americans have come to rely too heavily on these third parties to function as our “democracy radar.” We’ve largely shifted to their shoulders most responsibility to clue us in when something is brewing in government, tell us how we can exercise influence (if at all), and gauge the results of civic and government action.

Taken together, these backup systems generally have worked well enough — but they also have significant (and occasional dangerous) flaws. They’ve got too many blind spots, too many hidden agendas, insufficient transparency, and too little support for timely, effective citizen participation…

In other words, the patchwork network of backup systems often fail to supply enough civic information to precisely those people who are most likely to be involved or affected by civic issues, in ways that engage them and support participation. Also, often the civic info they offer generally reflects the providers’ own agendas, assumptions, habits, and preferences — about which they may or may not be conscious or transparent.

Yes, having these backup civic info systems is certainly better than relying solely on the government’s own information systems — but too often, not by much. And sometimes they can even be much worse.

JUST GIVE ME THE DATA

The Knight Foundation has been supporting some efforts to make civic and public info more user-friendly and direct, like Adrian Holovaty’s Everyblock project. This is another third-party civic info “backup system” that aims to provide a more direct experience of civic info. They try (and mostly succeed) to improve upon government communications by enhancing relevance and usability. Everyblock empowers users to search and filter civic info as they choose (at least within a geographic context).

…But there’s a big catch to offering this valuable service: Everyblock must cope with the fact that usually getting raw civic info from government and public sources is a huge pain. It requiring considerable tweaking and maintenance to constantly adapt their “screen scraping” processes.

Screen scraping is a painstaking, cumbersome programming technique. A screen scraper program extracts data from the final display output of another program (what gets shown in, say, your web browser). According to Wikipedia: “The key element that distinguishes screen scraping from regular parsing is that the output being scraped was intended for final display to a human user, rather than as input to another program, and is therefore usually neither documented nor structured for convenient parsing.” That means the whole process is inherently pitfall-prone and inefficient.

Blogger and author Jon Udell nailed the underlying problem of data friction inherent in situations where civic media are forced to resort to screen scraping to obtain public information:

“Data friction can be intentional or not. When it’s intentional, you might have to file a FOIA request to get it. But in a lot of cases, it’s unintentional. The data is public, and intended to be widely seen and used, but isn’t readily reusable.

“…Now it’s time to grease the wheels. Here’s one way that can happen. An enlightened city government can decide to publish [its] data in a reusable way. I’ve written extensively about Washington DC’s groundbreaking DCStat program which does exactly that. I can’t wait to see what happens when EveryBlock goes to Washington.

“But city governments shouldn’t have to go out of their way to provide web-facing data services and feeds. Databases should natively support them. That’s the idea behind Astoria (ADO.NET Services), which is discussed in this interview with Pablo Castro. If the NYC Department of Health had that kind of access layer sitting on top of its [restaurant inspection] database, it wouldn’t put EveryBlock’s screen-scraper out of a job — it would just make that [person's] job a whole lot more interesting and effective.”

This all leads back to why I like what the Knight Commission on the Information Needs of Communities in a Democracy is doing: They’re flipping the focus around, to put people’s needs first.

They’re doing this by starting from the question “What kind of information do communities need?” — rather than simply settling for “How can we tweak the badly designed, human-unfriendly entrenched patchwork system of civic information so that it becomes at least slightly less painful or more useful?”

…Well, they’re doing that to a point, anyway. The crucial limitation I see in their approach lies in how the Knight Commission has chosen to define “community.”

(COMING THURSDAY: Part 3, Beyond Geography…)

Fortunately afterward I caught up with one of the panelists, Geneva Overholser, who’s about to take the helm at USC’s Annenberg School of Journalism. She was kind enough to offer some thoughtful answers to Khan. Here’s what she had to say.

(Note: My apologies for the different audio levels between the intro and the interview. I recorded on two different devices and edited in iMovie HD, which I don’t yet know very well, so it’s a little clunky. I’m still learning.)

Here’s more info about who was on this panel…

From the SPJ event info page:

KEYNOTE: New Money, New Media, New Hope

• Rose Aguilar hosts the daily public affairs show Your Call on KALW-FM. Her forthcoming book, “Red Highways,” will be out in September. The book collects political interviews with people living and voting in so-called “red states,” and calls for a more thoughtful and productive dialogue in the media and between people with differing views. She will speak about what the public wants from journalism, and what it gets.
• Persephone Miel is a Fellow at the Berkman Center for Internet & Society at The Harvard Law School where she directs the Media Re:public project, examining the impact of participatory journalism on the information environment. Prior to joining Berkman, she spent more than 12 years with Internews Network, an international NGO supporting independent media around the world.
• Geneva Overholser is the newly appointed director of the School of Journalism at the University of Southern California’s Annenberg School of Communications. She previously held the Curtis B. Hurley Chair in Public Affairs Reporting for the Missouri School of Journalism, in its Washington, D.C., bureau. She is a frequent print, broadcast and online media critic, and the author of “On Behalf of Journalism: A Manifesto for Change.”
• David Talbot, the founder and former editor-in-chief of Salon.com, is also the author of New York Times bestseller “Brothers: The Hidden History of the Kennedy Years.” He recently launched a media and entertainment company called The Talbot Players with his brother Steve, executive producer of PBS’ Frontline World. He is also helping develop the San Francisco Free Press, a nonprofit Bay Area news engine that aims to combine the best of professional and citizens’ journalism.
• Moderator: Cynthia Gorney is a professor at the Graduate School of Journalism, U.C. Berkeley, a magazine writer (with regular contributions to National Geographic, The New York Times Magazine, The New Yorker, Oprah, Runners World and Harpers, among others), an occasional radio host of KQED-FM’s Forum, and the author of “Articles of Faith: A Frontline History of the Abortion Wars.”

(NOTE: I originally posted this article on Poynter’s E-Media Tidbits. But I thought Contentious readers might be interested in it, too.)

Most of what I do is help journalists and news orgs wrap their brains around the Internet. Generally I enjoy that work. Lately, though, I’ve been getting quite aggravated at the close-minded and helpless attitudes I’m *still* encountering from too many journalists about how the media landscape is changing. Those attitudes are revealed by statements, decisions, actions, and inaction which belie assumptions such as:

• The only journalism that counts is that done by mainstream news orgs, especially in print or broadcast form. Alternative, independent, online, collaborative, community, and other approaches to news are assumed to be inferior or even dangerous.
• Priesthood syndrome: Traditional journalists are the sole source of news that can and should be trusted — which gives them a privileged and sacred role that society is ethically obligated to support.
• Journalists and journalism cannot survive without traditional news orgs, which offer the only reliable, ethical, and credible support for a journalistic career.
• Real journalists *only* do journalism. They don’t dirty their hands or distract themselves with business and business models, learning new tools, building community, finding new approaches to defining and covering news, etc. As Louisville Courier-Journal staffer Mark Schaver said just this morning on Twitter, “[Now] is not a good time [for journalists] if you don’t want your journalism values infected with marketing values.”
• Journalistic status and authority demands aloofness. This leads to myriad problems such as believing you’re smarter than most people in your community; refusing to “compromise” yourself professionally by engaging in frank public conversation with your community; and using objectivity as an excuse to be uncaring, cynical, or disdainful.
• Good journalism doesn’t change much. So if it is changing significantly, it must be dying. Which in turn means the world is in big trouble, and probably deserves what it will get.

There’s a common problem with all these assumptions: They directly cut off options from consideration. This severely limits the ability of journalists and journalism to adapt and thrive…

I realize that right now is a scary time for journalists who crave stability. I have immense sympathy for good, smart people (many of whom have families to support and retirements to plan) who fear the unknown. Many of the news orgs that have sheltered and supported these journalists as they ply their craft are crumbling due to their inability or unwillingness to adapt their business models — leading to layoffs, buyouts, attrition, dwindling resources, overwork, and general demoralization.

I also know — first hand — that the prospect of learning new skills can be daunting. (That’s why, after all these years, I still don’t speak any language but English, and I still don’t know how to write computer code.) Plus, many of us have spent lots of money on j-school and many years in professional journalism honing our writing and reporting skills. We don’t *want* to learn how to think like an entrepreneur, or an information architect, or a community manager! We just want to keep doing what we know how to do; we didn’t sign up for all this extra stuff.

And I also understand journalists’ pride in the unique nature and value of our work. I wouldn’t want to live in a world that didn’t include solid reporting done by skilled journalists. That’s why I’m so passionate and persistent about helping to move journalism forward.

However… Even though despair is a natural result of prolonged fear and difficulty — when too many people in any culture are in despair, that culture can easily become toxic (overwhelmingly negative to the point of becoming self-destructive or self-defeating).

It’s worth looking over the Adbusters special series, Toxic Culture USA, by Kalle Lasn and Richard Degrandpre. Pay special attention to the research bibliography, citing several studies relating to the prevalence of despair and depression in certain communities and how that has a strong negative, destructive effect on those communities.

If this was nearly any other industry undergoing a sea change, I think my compassion and patience for the impact of workers’ despair would last longer. But regarding journalists, my personal patience is starting to wear thin.

The way I see it (and I’m far from alone in this view), right now is a time of immense opportunity for journalism and journalists to take on a broader and even more vital role in society. It’s a chance for journalists to not only continue doing good work, but maybe also to have more impact than ever before. If they can make this progress within updated, adapted news organizations, fine. But if not, they can find ways to do it independently, collaboratively, or by founding new supporting institutions or businesses.

Plus, new approaches to journalism can simply be more fun. As a group, journalists don’t seem to have nearly enough fun. In particular, engaging directly with your community can be fun and rewarding. Learning to monitor and improve the spread and impact of your work can be fun. And the process of learning anything new at all also can be a lot of fun. In fact, that basic craving for continual learning is what drew many of us to journalism in the first place. Remember that?

Yes, we need to solve the immediate problem of updating our business models, tool set, and skill set. This doesn’t have to be arduous — it can be playful. I’m looking forward to the NewsTools conference later this week to help address some of these issues in a constructive and (dare I say it) fun way.

Here’s the heart of the matter: Journalists (more so than most other professions) are supposed to be fundamentally curious and profoundly interested in what’s happening around them. Right now is no time for despair. It’s time to stop discouraging each other and start applying our innately enterprising abilities to our own field.

And let’s have more fun while doing it.