I experienced one such path tonight that turned into a loop and gave me a mini “eureka!” moment, so I wanted to share:

I met a fellow journalist/geek, Keith Collins, at BarCamp News Innovation Philly on April 28. We were chatting about science and that, of course, led to RadioLab. He mentioned a segment he enjoyed about a pendulum. I did a quick search on my phone and sent myself the link to read later. When I returned to the post, it didn’t seem like I found the right item — this was a post on the Krulwich Wonders blog about a Pendulum Dance. Nonetheless, it fascinated me.

I tweeted it with a hat tip to Keith and he replied with the actual segment he had referenced on the Limits of Science. It did not disappoint. I responded to say that I’d enjoyed it and Keith replied with a link to one of the things mentioned in the segment called Eureqa, which is described as a

“software tool for detecting equations and hidden mathematical relationships in your data. Its goal is to identify the simplest mathematical formulas which could describe the underlying mechanisms that produced the data. “

After downloading the application for later and browsing the page, I happend to scroll down to the “more information section.” A link about symbolic regression, which led to the Wikipedia page on genetic programming, grabbed my interest.

I happened to scroll past the introduction to the history section and read the first line there:

“In 1954, GP [genetic programming] began with the evolutionary algorithms first used by Nils Aall Barricelli applied to evolutionary simulations.”

Baricelli is a prominent figure in the wonderfully insightful book I’m curerntly reading about the origins of the digital universe:  Turing’s Cathedral by George Dyson.

“Eureka!”

No. Scratch that.

“Eureqa!”

Now that’s what I call finding hidden relationships in your data.

—

Bonus discovery path: When I tweeted the Pendulum Dance post, Xerox PARC‘s @PARCinc Twitter account favorited it. It seems clear they found it after seeing my reply to a tweet from Scott Klein. That prompted me to look at their recent tweets to see if they were an account I wanted to follow (I did!). In a then-recent tweet, they shared a Wikimedia newsletter that included a summary of a PARC report titled Thermodynamic Principles in Social Collaboration. Gotta love the interwebs!

“We are excited to announce that Greg Linch will be moving into a new hybrid technology / newsroom role starting June 1.  Since coming to the Post in December 2010, he has desk-pedaled his way across a few sections.

Greg began by producing for the health, science and environment team.  Those months rekindled a childhood interest by feeding and growing his natural curiosity about the world.  He then put both halves of his journalism-political science double major to use during a short stint with the politics team before starting an exciting year working with the foreign and national security desks.

All the while he’s been improving his technical knowledge with the ultimate goal of doing better journalism, such as creating a few handy tools and helping to make some production tasks more efficient.  That’s no surprise, of course, considering the two start-ups he previously worked on — one for college media when he was at the University of Miami and one that made tools for newsrooms before he joined the Post.

At the Post we have section producers who primarily work in a CMS and engineers who build news applications, but nothing in between.  Greg will pioneer an experimental role to straddle web production and web development – a special projects and applications producer position that will focus on more technical and medium to long-term projects and solutions.

We see Greg as a person who can look beyond standard journalism forms to help develop technology that pushes the boundaries of storytelling alongside the newsroom.  He will partner with editors and technologists to
conceive and create tools that engage users with our journalism; current examples of apps in development include a polling interface and our new live blogging platform.  He will work with the entire newsroom, from
producers to reporters to designers to find places where development can come together to create new technology that serves our users and our journalism.

Greg will spend the first 3 – 6 months of this new role training exclusively with engineers: honing his development skills.  After he completes this initial embedding in web development, he’ll be working in
the newsroom through Cory Haik and be deployed on projects within news and alongside the embedded engineering group run by Washington Post Chief Architect Greg Franczyk.”

Job opening

With my move, there’s an opening for a world/national security producer. That posting is below. Let me know if you’re interested!

“The foreign desk is seeking a producer to play a central role in developing and executing a digital strategy for world and national security. An ideal candidate would bring a passion for foreign and national security news, a
commitment to teamwork, and a highly-developed facility for using the tools of digital journalism to enhance and augment our work.

The world producer is a full member of the foreign-desk team and also works closely with national security editors. With a mission of making our work more engaging, the producer will have primary day-to-day responsibility for the presentation of foreign and national security content on the web, mobile and tablet. The job encompasses roles in nearly every aspect of digital journalism, including blogs, search, social and multimedia. The producer works closely with foreign and national security correspondents
and editors but must also show initiative, an appetite for innovation and strong news judgment. The producer reports to Anup Kaphle, the digital editor for world.

The job will require some early-morning and weekend work. Interested candidates should contact Doug Jehl, Griff Witte or Peter Perl by May 13. We plan to fill the job no later than June 1, when the current producer, Greg Linch, moves on to another job in the newsroom.”

Better ways to measure journalism

If you’re interested, please up-vote/comment soon! Voting closes Thursday, April 12.

It would begin with an overview of interesting metrics in other fields, how we can learn from those and how we can possibly apply them to journalism. After that groundwork is laid, there would be a moderated discussion among the participants of other examples and how we could best implement new metrics.

For more background on the idea, check out:

Quantifying impact: A better metric for measuring journalism

Carnival of Journalism: Responses to “How can we better measure journalism?”

With this question, I wanted to broaden the possibile metrics beyond just impact. Higlights from the discussion are below. Enjoy!

Sheree Martin offered four important questions we must first consider:

• What is journalism?
• What is impact?
• How do we measure?
• Who is measuring?

Kathy E. Gill asked about similar fundamentals, “What matters? What is the role of journalism, our purpose, our challenge?” and raised a good question of balance:

how does “but it matters!” coexist in an environment where assessment is measured in large part by short-term page views and click-throughs?

Denise Cheng noted an opportunity to measure “inert engagement,” described as “the engagement that doesn’t want to come out of hiding as big steps like shares and comments.” She said:

measuring impact by measuring engagement are manifold. First and foremost, the modus operandi on my patch of the Internet is that journalism’s highest ideal is to equip its readership with information from which they can take judicious action

Specifically, she recommended defining metrics before you embark, measuring topical importance to the audience and acts accordingly, and that the sturdier your metrics become over time, the more of a road map you have.

Jonathan Groves looked at the issue on a more elemental level:

The root of journalism is truth, and the time-tested method that journalists have to uncover that truth is verification. If we want to measure journalism, it must begin here.

Addressing the quantitative vs. qualitative measurement distinction, he said:

Measurement assumes quantification, and some ideas — such as verification — are better evaluated qualitatively. Creating a measure requires including some attributes and excluding others; inevitably, such measures are always imperfect approximations, especially when it comes to complex concepts.

Clarisa Morales Roberts described a possible formula framed by the effects of journalism:

If we want to measure the impact of media and online journalism, we need to consider action. Action is what defines Effective Media (EM), and Effective Media can be measured by the Action that is a direct result of Quality Dialogue that is Shared

…

So, if we want to consider Impact by measuring Action, that measurement has to be proportional.

Michael Rosenblum emphasized the importance of finding niche instead of mass audiences:

As more and more content begins to fill the blogosphere and cyberspace and the cloud and wherever else ‘it’ all is, the competition for the Holy Grail of mass audience becomes ever more intense, and as such, the content itself becomes ever more amorphous.

Yet where is the ‘real’ value?

The web gives us access to discrete groups with specific interests. Our goal should be ‘narrowing the field’, not expanding it. Creating affinity groups with a common interest and common goals, and then, making it possible for those people to achieve those goal – whether its contributing to a new project – as in Kickstarter, or going on a golfing trip to St. Andrews.

Steve Outing looked at how social media is gaining an edge in the impact realm:

When I look at the question, I can’t help but get sidetracked into thinking how social media (i.e., “the crowd” utilizing digital social tools like Twitter, Facebook, and Change.org, among others, to amplify their voices) in a growing number of cases is having more impact than the traditional news media can achieve themselves — or is driving the mainstream news media to pay attention to stories that their editors fail to recognize as important.

Carnival ringmaster David Cohn also proposed an alternative approach :

I want to measure a different kind of impact. The impact of the dollars we spend in pursuit of journalism and its meaningful impact.

…

What we don’t appreciate is the strength of the little guy. What they don’t have in “impact” they do have in efficiency.

Steve Fox challenged the assumption of measuring something like impact:

Perhaps we all need to remember that the true “impact of journalism” rests with the impact we have on people’s lives. Have we given readers/viewers an amazing piece of writing or video that makes them appreciate parts of their life more? Have we created an “Oh, wow” moment for readers/viewer? Have we expanded someone’s universe? Isn’t that why we got into this business? Isn’t that what journalism has always done?

Perhaps the real question should be: “Why are we spending so much time measuring the “impact” of journalism?” Because, it really isn’t quantifiable now, is it?

What would be your ideal measure for journalism?

If you can’t see the embed above, view the discussion on Storify.

Journalism is a tool to be used, both by those who practice it and those who engage with it.

Journalism is inherently interdisciplinary, both in the subjects it covers and how those subjects are covered.

Journalism can applied to any subject, or theme. There are some basic themes you can start with and modify. Child themes can be derived from parent themes; for example, online learning as a child theme of education.

Journalism has core features. Research, reporting, verification, creation and more.

Journalism has more advanced functionality you could call plugins. These can improve the process or the product (here, meaning outcome). Analysis, visualization and feedback/participation mechanisms, for example. Sometimes that functionality gets incorporated into core.

The point is not to make an arbitrary comparison. And, yes, some of these comparisons are apples to oranges.

The point is to think more abstractly both about the concept of journalism and about journalism concepts. The basic ideas. The individual pieces. The fundamentals.

Journalism should be seen as a modular platform that we can customize, develop and improve.

Journalism is an open-source framework constantly in development.

No one owns journalism. No one controls journalism. Anyone can implement it. Anyone can fork it. Anyone can hack at its core.

What are you developing?

Update: I changed “practice” to “concept” in the first line. I think that’s more the frame I was looking for, as indicated by the 8th paragraph.

“the new discipline of physics could not proceed until Isaac Newton appropriated words that were ancient and vague—force, mass, motion, and even time—and gave them new meanings. Newton made these terms into quantities, suitable for use in mathematical formulas.”

The term information was similarly amorphous until Claude Shannon, while working at Bell Labs, quantified the concept in bits.

* * *

The journalism goals and business goals for news organizations are out of sync.

Pageviews. Unique visitors. Time on site.

Some journalism might be best quantified partly or wholly by one or more of those ways, but we need to explore deeper beyond these fairly simplistic metrics.

We know how these terms are defined, but what do they really mean? What do they help us achieve?

In creating a theory of information and quantifying information in bits, Shannon aimed to remove meaning. “Shannon had utterly abstracted the message from its physical details,” Gleick says.

For journalism, the goal should be to add more meaning to the information we use to measure our work. Granted, our current metrics aren’t meaningless. We use them because they do have meaning: views, comments, shares, etc. each has a meaning and can be measured based on that one-dimensional measure. The quantities of metrics increase because the works of journalism they describe are meaningful. Or, put another way, impactful.

So, what if we measured journalism by its impact?

Impact is something journalists have long valued. Whether it’s to inform, to hold accountable or to entertain, this more qualitative effect is an oft-cited goal.

Impact also addresses, at least in part, a fundamental disconnect between our main modern metrics and our journalistic values. Setting goals in terms of pageviews, for example, offers the wrong incentives. Yes, sometimes accountability journalism, for example, can drive as many pageviews as the latest celebrity gossip and the two goals align, but it’s far from a perfect model for success.

More specifically, these widely-used one-dimensional metrics value quantity over quality. True, we do have two-dimensional metrics that focus more on quality (time on site, pages per visit, etc), but quantity still seems to reign because they are at the heart of our models. There’s a danger in narrowly focusing on the one-dimensional “how much?” and ignoring the “so what?” of varied, complex, nuanced results of journalistic work. But, really, this is a problem because we’re looking at the wrong kind of “how much.”

Modern technology enables better quantitative and analytical tools, conceivably offering better ways to evaluate the results of journalism. Specifically, by weighing various measures that were not available in the past, it’s possible to devise a way to more concretely — albeit still imperfectly — define what impact means.

Yes, it’s true that this concept of impact — as currently defined — is very subjective and complex, as information used to be. It’s not easy, nor is it perfect or precise, but it deserves experimentation and effort in trying. And it won’t improve if no one starts somewhere.

* * *

Currently, works of journalism (articles, videos, galleries, graphics, etc.) no matter what subject (news, sports, entertainment, business, features, investigations, etc.) are quantitatively measured the same. An investigative piece that might be nowhere near as popular in pageviews across a mass audience (yes, sometimes, they can be) is quantitatively measured the same way a celebrity death story is. Either story could make a sensational splash, truly connect emotionally with readers, or both. Each has value, but there are different kinds of values across different subjects journalists cover.

If we value impactful accountability journalism, why are we quantitatively equating it one-to-one to entertainingly impactful news? For example, when an investigation is published that saves taxpayer money or even human lives, we should instead try to measure these in a more multi-dimensional way — instead of merely the simplistic ones — and measure them differently from journalism works that have different goals. We should do this not just because the quantification would be more accurate (again, still imperfect), but because it would be a better model of the complex real-world response.

Yes, this is already done in general, but informally and mostly qualitatively.

Along with weighing disparate works differently, we should consider that the immediate impact at a wide scale maybe be minimal, while the immediate impact on a small scale may be huge and that, in turn, leads to wider impact. Moreover, impact also needs to be measured over time — whether it grows, diminishes or stays the same — and perhaps even at what rate.

* * *

If metric goals are defined by executives, the business department and top editors, how can we build a bridge between the business reality and journalism reality? Where’s the middle where the two meet?

“How can we possibly meet these goals?” some in the newsroom might ask. “What does this have to do with good journalism?”

An ad-supported model traditionally sustained most newspapers. New models for selling ads online — based on CPM, for example — evolved to where we are today.

Relying on the one-dimensional metrics that comprise many of these online ad models can also be inaccurate.

We should not longer passively accept this status quo.

Define your own metrics. That was the key lesson of an unconference session Alexis Madrigal led at the first SparkCamp in New York City last June. It resonated and stuck with me, although mostly as a good idea that I had no idea how to implement.

He dubbed the example metric as something along the lines of “awesomeness,” which would be calculated by an equation weighting different metrics against each other. Really, what he’s talking about is a multi-dimensional metric:

(Pageviews / 1,000) + (video views / 100) + … etc

Tools like ChartBeat and its niche product NewsBeat offer a distinction between active and non-active users, as well as how many are merely reading or writing on the site. They are also developing a new metric called “engaged minutes.” These measures are important and helpful, but I think — invoking Madrigal again — we need to define our own metrics.

Impact makes sense because different organizations can define it in different ways. It’s not a set formula — it’s framework for which to think about how we measure journalism.

If Madrigal’s session was the unlit match, reading Melissa Rach’s second post on the value of content provided the spark when she said, “the baseline will help you measure the impact of your future content work .” Besides that inspiration, I found this nugget particularly insightful:

“most scientists, mathematicians, and statisticians say exact measurement is a myth. To them, the goal of measurement is to reduce uncertainty. Get this: it’s impossible to eliminate uncertainty all together– all measurement is based on assumptions. That means, when measuring content value, you don’t have to come up with precise numbers.”

We should consider the same in measuring impact.

And, to be clear, impact would not replace all other metrics. Instead, it could be an umbrella, a supplement to current measures or a step to something better than where we are now.

I envision a tool that allows different organizations, their departments and other levels (education under local, finance under business, etc.) to customize how they define impact and have that transparently communicated to journalists, readers, executives, advertisers, sponsors, etc.

You could factor in all the usual metrics of pageviews, pages per visit and time on site along with others such as comments, social mentions of a story (and by what kind of people) and links. You could track the larger conversation around a story (hat tip to Marc Lavallee for his influence in my thinking on that). You could also account for actions taken by governments, non-profits, community groups, registered voters, parents and others.

* * *

Upon further reflection separate from my initial writing, two words that have been mentioned several times deserve particular attention: weight and value.

In one sense, the current standard measuring tools are like different variations of the same ruler when instead we should really be using different kinds of scales to measure the weight of our work. Weight and impact are not the same, but they’re related.

At the first NewsFoo in December 2010, Tim O’Reilly led a discussion on philosophy and remarked on building a product to reinforce a value. He later shared how he built his business around values rather than business models and emphasized the need to ask, “Are those the right values?”

So, are pageviews, pages-per-visit, time on site, even engagement (which is moving in a more quantitative direction) conveying the right values?

* * *

Many of these core ideas formed in my mind before fully exploring if this specific idea had been discussed in a journalism context and, as I found, it has. Here are a few interesting ones I found, but — even though they didn’t influence the thoughts above because of timing — I thought I should share them:

- Metrics for civic impacts of journalism by Ethan Zuckerman

- 5 Needs and 5 Tools for Measuring Media Impact and Is There a Master Metric for Evaluating Public Media? by Jessica Clark

Another interesting thread I learned about after the main ideas above formed involves new ways to measure science and altmetrics (and thanks to Jonathan Stray the heads-up on the almetrics manifesto).

A direct inspiration, as is obvious above, was my reading and partial re-reading of Gleick’s The Information. Some less direct inspiration as I conceived the ideas above include many of Stray’s excellent posts on rethinking journalism; specifically, designing journalism to be used. Journalism is “used” because it has impact, meaning and a purpose. A number of journalists might be concerned about thinking about their work this way, but it’s true — whether direct or indirect. By observing and reporting, you’re on the field (to borrow a Jay Rosen metaphor).

Maybe you could even compare journalism to quantum mechanics, in a limited way. By your mere presence and observation, you can affect the outcome. And then by publishing journalism — from citing facts to quoting opinion to offering analysis — you are having an impact.

That impact is what we should measure.

* * *

I plan to soon follow up with some ideas on how we could measure impact. I have some initial thoughts and aim to get more from a Spark Camp Austin session I’ve proposed on measuring journalism.

But, overall, my goal is not necessarily to specifically outline the framework for how we could measure impact — though I do plan to try. Instead, the main goal is to advance the conversation and reality of how we measure journalism. What do those metrics mean? What do we value? How do we weigh different types of journalism in ways that are appropriate to their purposes? And, fundamentally, are our journalistic goals and business goals in line?

Update 1: This post has been updated with minor tweaks and two specific quotes from Melissa Rach’s post. Also, thanks to several people who I explained the idea to before writing this, including Albert Sun and Max Cutler for their feedback as I worked on finishing the draft en route to Spark Camp in Austin.

Update 2: Read notes from a BarCamp News Innovation Philadelphia session I led on defining new metrics for journalism.

Update: ASNE also Storified the chat.

Steve Jobs’ greatest legacy is not the products he created, but what they enabled and who they inspired.

Soon after that, I thought of a lesson for journalism: we shouldn’t focus so much on what we do as much as what we enable, who we impact and what comes from all that.

Speaking of Jobs’ legacy, watch his 2005 Stanford commencement speech:

Update: Not long after publishing this post, I noticed a hashtag and decided to re-send the original tweet with #stevejobslegacy appended. And — whaddya know it — it was added to the New York Times’ homepage widget: