“the new discipline of physics could not proceed until Isaac Newton appropriated words that were ancient and vague—force, mass, motion, and even time—and gave them new meanings. Newton made these terms into quantities, suitable for use in mathematical formulas.”

The term information was similarly amorphous until Claude Shannon, while working at Bell Labs, quantified the concept in bits.

* * *

The journalism goals and business goals for news organizations are out of sync.

Pageviews. Unique visitors. Time on site.

Some journalism might be best quantified partly or wholly by one or more of those ways, but we need to explore deeper beyond these fairly simplistic metrics.

We know how these terms are defined, but what do they really mean? What do they help us achieve?

In creating a theory of information and quantifying information in bits, Shannon aimed to remove meaning. “Shannon had utterly abstracted the message from its physical details,” Gleick says.

For journalism, the goal should be to add more meaning to the information we use to measure our work. Granted, our current metrics aren’t meaningless. We use them because they do have meaning: views, comments, shares, etc. each has a meaning and can be measured based on that one-dimensional measure. The quantities of metrics increase because the works of journalism they describe are meaningful. Or, put another way, impactful.

So, what if we measured journalism by its impact?

Impact is something journalists have long valued. Whether it’s to inform, to hold accountable or to entertain, this more qualitative effect is an oft-cited goal.

Impact also addresses, at least in part, a fundamental disconnect between our main modern metrics and our journalistic values. Setting goals in terms of pageviews, for example, offers the wrong incentives. Yes, sometimes accountability journalism, for example, can drive as many pageviews as the latest celebrity gossip and the two goals align, but it’s far from a perfect model for success.

More specifically, these widely-used one-dimensional metrics value quantity over quality. True, we do have two-dimensional metrics that focus more on quality (time on site, pages per visit, etc), but quantity still seems to reign because they are at the heart of our models. There’s a danger in narrowly focusing on the one-dimensional “how much?” and ignoring the “so what?” of varied, complex, nuanced results of journalistic work. But, really, this is a problem because we’re looking at the wrong kind of “how much.”

Modern technology enables better quantitative and analytical tools, conceivably offering better ways to evaluate the results of journalism. Specifically, by weighing various measures that were not available in the past, it’s possible to devise a way to more concretely — albeit still imperfectly — define what impact means.

Yes, it’s true that this concept of impact — as currently defined — is very subjective and complex, as information used to be. It’s not easy, nor is it perfect or precise, but it deserves experimentation and effort in trying. And it won’t improve if no one starts somewhere.

* * *

Currently, works of journalism (articles, videos, galleries, graphics, etc.) no matter what subject (news, sports, entertainment, business, features, investigations, etc.) are quantitatively measured the same. An investigative piece that might be nowhere near as popular in pageviews across a mass audience (yes, sometimes, they can be) is quantitatively measured the same way a celebrity death story is. Either story could make a sensational splash, truly connect emotionally with readers, or both. Each has value, but there are different kinds of values across different subjects journalists cover.

If we value impactful accountability journalism, why are we quantitatively equating it one-to-one to entertainingly impactful news? For example, when an investigation is published that saves taxpayer money or even human lives, we should instead try to measure these in a more multi-dimensional way — instead of merely the simplistic ones — and measure them differently from journalism works that have different goals. We should do this not just because the quantification would be more accurate (again, still imperfect), but because it would be a better model of the complex real-world response.

Yes, this is already done in general, but informally and mostly qualitatively.

Along with weighing disparate works differently, we should consider that the immediate impact at a wide scale maybe be minimal, while the immediate impact on a small scale may be huge and that, in turn, leads to wider impact. Moreover, impact also needs to be measured over time — whether it grows, diminishes or stays the same — and perhaps even at what rate.

* * *

If metric goals are defined by executives, the business department and top editors, how can we build a bridge between the business reality and journalism reality? Where’s the middle where the two meet?

“How can we possibly meet these goals?” some in the newsroom might ask. “What does this have to do with good journalism?”

An ad-supported model traditionally sustained most newspapers. New models for selling ads online — based on CPM, for example — evolved to where we are today.

Relying on the one-dimensional metrics that comprise many of these online ad models can also be inaccurate.

We should not longer passively accept this status quo.

Define your own metrics. That was the key lesson of an unconference session Alexis Madrigal led at the first SparkCamp in New York City last June. It resonated and stuck with me, although mostly as a good idea that I had no idea how to implement.

He dubbed the example metric as something along the lines of “awesomeness,” which would be calculated by an equation weighting different metrics against each other. Really, what he’s talking about is a multi-dimensional metric:

(Pageviews / 1,000) + (video views / 100) + … etc

Tools like ChartBeat and its niche product NewsBeat offer a distinction between active and non-active users, as well as how many are merely reading or writing on the site. They are also developing a new metric called “engaged minutes.” These measures are important and helpful, but I think — invoking Madrigal again — we need to define our own metrics.

Impact makes sense because different organizations can define it in different ways. It’s not a set formula — it’s framework for which to think about how we measure journalism.

If Madrigal’s session was the unlit match, reading Melissa Rach’s second post on the value of content provided the spark when she said, “the baseline will help you measure the impact of your future content work .” Besides that inspiration, I found this nugget particularly insightful:

“most scientists, mathematicians, and statisticians say exact measurement is a myth. To them, the goal of measurement is to reduce uncertainty. Get this: it’s impossible to eliminate uncertainty all together– all measurement is based on assumptions. That means, when measuring content value, you don’t have to come up with precise numbers.”

We should consider the same in measuring impact.

And, to be clear, impact would not replace all other metrics. Instead, it could be an umbrella, a supplement to current measures or a step to something better than where we are now.

I envision a tool that allows different organizations, their departments and other levels (education under local, finance under business, etc.) to customize how they define impact and have that transparently communicated to journalists, readers, executives, advertisers, sponsors, etc.

You could factor in all the usual metrics of pageviews, pages per visit and time on site along with others such as comments, social mentions of a story (and by what kind of people) and links. You could track the larger conversation around a story (hat tip to Marc Lavallee for his influence in my thinking on that). You could also account for actions taken by governments, non-profits, community groups, registered voters, parents and others.

* * *

Upon further reflection separate from my initial writing, two words that have been mentioned several times deserve particular attention: weight and value.

In one sense, the current standard measuring tools are like different variations of the same ruler when instead we should really be using different kinds of scales to measure the weight of our work. Weight and impact are not the same, but they’re related.

At the first NewsFoo in December 2010, Tim O’Reilly led a discussion on philosophy and remarked on building a product to reinforce a value. He later shared how he built his business around values rather than business models and emphasized the need to ask, “Are those the right values?”

So, are pageviews, pages-per-visit, time on site, even engagement (which is moving in a more quantitative direction) conveying the right values?

* * *

Many of these core ideas formed in my mind before fully exploring if this specific idea had been discussed in a journalism context and, as I found, it has. Here are a few interesting ones I found, but — even though they didn’t influence the thoughts above because of timing — I thought I should share them:

- Metrics for civic impacts of journalism by Ethan Zuckerman

- 5 Needs and 5 Tools for Measuring Media Impact and Is There a Master Metric for Evaluating Public Media? by Jessica Clark

Another interesting thread I learned about after the main ideas above formed involves new ways to measure science and altmetrics (and thanks to Jonathan Stray the heads-up on the almetrics manifesto).

A direct inspiration, as is obvious above, was my reading and partial re-reading of Gleick’s The Information. Some less direct inspiration as I conceived the ideas above include many of Stray’s excellent posts on rethinking journalism; specifically, designing journalism to be used. Journalism is “used” because it has impact, meaning and a purpose. A number of journalists might be concerned about thinking about their work this way, but it’s true — whether direct or indirect. By observing and reporting, you’re on the field (to borrow a Jay Rosen metaphor).

Maybe you could even compare journalism to quantum mechanics, in a limited way. By your mere presence and observation, you can affect the outcome. And then by publishing journalism — from citing facts to quoting opinion to offering analysis — you are having an impact.

That impact is what we should measure.

* * *

I plan to soon follow up with some ideas on how we could measure impact. I have some initial thoughts and aim to get more from a Spark Camp Austin session I’ve proposed on measuring journalism.

But, overall, my goal is not necessarily to specifically outline the framework for how we could measure impact — though I do plan to try. Instead, the main goal is to advance the conversation and reality of how we measure journalism. What do those metrics mean? What do we value? How do we weigh different types of journalism in ways that are appropriate to their purposes? And, fundamentally, are our journalistic goals and business goals in line?

This post has been updated with minor tweaks and two specific quotes from Melissa Rach’s post. Also, thanks to several people who I explained the idea to before writing this, including Albert Sun and Max Cutler for their feedback as I worked on finishing the draft en route to Spark Camp in Austin.

Update: ASNE also Storified the chat.

Saturday

Daniel Bachhuber and I want to know — and we’ll compile a list based on the responses. Please comment below and include links, if possible. Thanks!

I’ve had a great experience working with the health, science, environment and wellness reporters and editors since I joined the Post in December and am very excited at this new opportunity be a full member of the foreign editing team — truly integrated with the section.

Stay tuned!

What was the highlight of the chat for you?

http://twitter.com/#!/alykat/status/24865749973602304

http://twitter.com/#!/kathleensulli/status/24867424381698048

http://twitter.com/#!/matthewbyrnes/status/24873613823836160

http://twitter.com/#!/excdinglyrandom/status/24892284650651650

http://twitter.com/#!/trishanderton/status/24896231633321987

http://twitter.com/#!/sdoig/status/24912285742931968

http://twitter.com/#!/fricto/status/24914985498972160

http://twitter.com/#!/katdowns/status/24917106692718592

http://twitter.com/#!/suzanneyada/status/24917609505890305

http://twitter.com/#!/davewrightjr/status/24918171098030080

http://twitter.com/#!/RobinJP/status/24922525158871040

http://twitter.com/#!/rosenbaumk/status/24933005969592321

http://twitter.com/#!/megterilli/status/24933641276624896

http://twitter.com/#!/karenkho/status/24933829315661824

http://twitter.com/#!/WithDrake/status/24934657334181890

http://twitter.com/#!/kingkool68/status/24934752645545984

http://twitter.com/#!/RebeccaKesten/status/24935215356977152

http://twitter.com/#!/jmestepa/status/24935809392058368

http://twitter.com/#!/LisaSink/status/24943431558111235

http://twitter.com/#!/williampd/status/24969096722386944

http://twitter.com/#!/JMBorchardt/status/24971075678896128

http://twitter.com/#!/michelleminkoff/status/24971080330383360

http://twitter.com/#!/wcochran/status/24991547237867520

http://twitter.com/#!/greglinch/status/24993955959541762

http://twitter.com/#!/vaguity/status/24996077383983105

http://twitter.com/#!/pergam/status/24997551006556160

http://twitter.com/#!/kimgrinfeder/status/24997950212022272

P.S. I love the Twitter Blackbird Pie plugin for WordPress.

Quick thoughts from this evening:

When you were in elementary school, what did you say when asked, “What do you want to be when you grow up?” Do you see any connections between that and your current job or interests?

Thinking about this makes me feel thankful for all the great opportunities I’ve had and continue to have in journalism and technology.

UPDATE: Read some of the responses.

• Networked reading: room 252 at 4:30 p.m.
• GitHub for News (some background: Quick thoughts on journalism and version control): room 252, 5:30 p.m.

I’ve been meaning to write about these once the ideas are baked a little bit more, so stay tuned for more details.

"Collaborative design process can be..."

(Disclosure: I attended as a guest of the Post; I am not — yet? — an ADCMW member.)

I’ve included my chronological livetweets from the #adcmwhappycog event (P.S. not my idea for a hashtag), plus some tweets with photos by @AnneLikesRed. Chris Cashdollar (@ccashdollar) and Kevin Hoffman (@kevinmhoffman) gave the presentation, which I will embed or link here after it’s posted online.

Read their great insights and comment below with your thoughts:

greglinch: #happycog is different from other shops bc everyone works on a project. Also, everyone works in more than just their speciality.

greglinch: The official hashtag for the @happycog event is #adcmwhappycog. They’ll be discussing their processes and more.

greglinch: .@happycog processes generally include: project definition, IA, viz design (design systems for ease in future), programming #adcmwhappycog

greglinch: Process values at @happycog: collaborative, iterative and flexible. #adcmwhappycog

AnneLikesRed: http://twitpic.com/380xjq #adcmwhappycog

greglinch: All of the things I’ve mentioned in the past few tweets is more of a framework then part of a rigid process. #adcmwhappycog

greglinch: For each project, @happycog asks: What is the design challenge at hand & what is the culture of the org they’re working with? #adcmwhappycog

greglinch: What the @happycog process could be like. RT @AnneLikesRed: http://twitpic.com/380xjq #adcmwhappycog

greglinch: .@happycog will sometimes disconnect and completely focus on the work — sometimes working on paper to start. #adcmwhappycog

greglinch: Sample sketches from Build-a-Bear project, which never launched but, bc of that, they learned a lot #adcmwhappycog http://yfrog.com/28jgmfj

greglinch: At wireframe stage, they focused on the upsell and the call to action — the two fundamental reqs. #adcmwhappycog http://yfrog.com/mn3wqj

greglinch: Another view of #adcmwhappycog event. RT @robleto: Happy Cog speaking at ADCMW @ Washington Post http://instagr.am/p/Srh1/

greglinch: “Fast results have a cost.” Also, they have no-meeting zones to focus on work — and those are disconnected from email, etc. #adcmwhappycog

greglinch: Where are all the #ONADC tweets, peeps? Share the love! Read about @happycog event with: #adcmwhappycog

greglinch: Project definition/IA when working with clients: Early, open discussion with focused meetings #adcmwhappycog

greglinch: Wireframes at @happycog focus on hierarchy + persistence, *not* visual design #adcmwhappycog

greglinch: When you’ve worked with a wireframe for weeks and need to scrap it, it can be hard to get out the thinking of that wireframe. #adcmwhappycog

greglinch: “Rules need to be understood to be broken” #adcmwhappycog

greglinch: At @happycog, they’re great at generating ideas — they need to improve at removing ideas. #adcmwhappycog

greglinch: Photo of @HappyCog commenting system that’s character-limited and can be tweeted. #adcmwhappycog http://yfrog.com/0le4ifj

greglinch: They put ideas on the @HappyCog blog that are incomplete so people can weigh in and build on them. #adcmwhappycog

greglinch: .@HappyCog is mostly waterfall, with only a bit of agile, for their process. #adcmwhappycog

greglinch: “Use deliverables for discussion” — @HappyCog. #adcmwhappycog

greglinch: Mental Models is a book about “aligning design with human behavior” they recommend http://j.mp/bPBZIK #adcmwhappycog

greglinch: Use “existing research and design team as guinea pigs” for project. For Philly tourism site, asked what they’d do on weekend. #adcmwhappycog

AnneLikesRed: http://twitpic.com/381br2 #adcmwhappycog

greglinch: “Balance classification with context” — make sure info is where it needs be, not just like indexes in a library card catalog #adcmwhappycog

greglinch: Roughy 80 people attending #adcmwhappycog event.

AnneLikesRed: So true! http://twitpic.com/381fal #adcmwhappycog

greglinch: “The actual dynamic of group decision making” — You need convergent and divergent thinking #adcmwhappycog http://yfrog.com/jau2rzj

greglinch: Book recs: Rework http://j.mp/c1PlYh and Facilitator’s Guide to Participatory Decision-Making http://j.mp/af7oqi #adcmwhappycog

greglinch: Core Values: full participation, mutual understanding, inclusive solutions, shared responsibility #adcmwhappycog http://yfrog.com/9fvddnj

greglinch: “Pathways to Action Model” — process design, vision space, problem space, solution space, make it real #adcmwhappycog

greglinch: Collaborative Design Process: creative concepts, client review, single concept iteration #adcmwhappycog http://yfrog.com/17izxqj

greglinch: Techniques they use w/ clients to learn more include a scale for balancing site’s feeling & MadLibs #adcmwhappycog http://yfrog.com/c95s9bj

greglinch: “Being dogmatic is for the birds” — find the unique process needs for each project. #adcmwhappycog

greglinch: During Q&A: “People who tell you CMSes are separate from design is lying to you” — it does impact design decisions #adcmwhappycog

greglinch: Quite a coincidence that an attendee mentioned Edward de Bono + his book — I just discovered him on Tues http://j.mp/bCF2Pe #adcmwhappycog

greglinch: .@happycog looks for models — such as from other industries — that can solve specific interaction problems. #adcmwhappycog

greglinch: And that’s a wrap! Hope you all enjoyed the #adcmwhappycog tweets. My fingers (and iPhone) get the rest of the night off.