The statistician in me has been in denial for some time about some growing evidence in favor of the wisdom of crowds. The cover of James Surowiecki’s book has caught my attention during nearly every visit I’ve made to the business book section of local bookstores over the past two years. And yet, until recently, I couldn’t bring myself to buy the book. I mean, how can a group of non-experts collectively provide better answers than a sage?

My change of heart really started when I noticed that Microsoft has been developing its own prediction markets “as a serious alternative to blunt forecasting tools.” Google had already announced here that it was using them, but that didn’t surprise me much – Google is still a relatively young company with a reputation for innovation and challenging existing paradigms. In Microsoft, though, you have a thirty-year-old company with arguably the largest portfolio of information age assets — talent, technology, and cash — the world has seen. And as noted here by Scoble, a culture of centralized decision making has become well-ingrained at the company. When an asset-rich, old-school-managed company like that begins considering a approach based on decentralized decision making (even if it’s just in one department) I start thinking it’s worth a closer look.

Many others have already published reviews of the book…
Geoff Hutchinson
Shawn Callahan - Anecdote
Steve Hardy (Creative Generalist)
David Edery (Game Tycoon)
Jason Kottke
“Heeding the Herd Instinct” - Business Week
… so I won’t add another. For those who seek a simple explanation of Surowiecki’s premise, Kathy Sierra delivers it:

The wisdom of crowds comes not from the consensus decision of the group, but from the aggregation of the ideas/thoughts/decisions of each individual in the group. At its simplest form, it means that if you take a bunch of people and ask them (as individuals) to answer a question, the average of each of those individual answers will likely be better than if the group works together to come up with a single answer.

During my regular monitoring of business blogs and articles over the past month, a few recent publications related to information flow in organizations caught my attention as I pondered this collective intelligence concept. From these sources, I’ve identified [so far] three phenomenon which hamper an organization’s ability to make well-informed decisions.

Upward voice doesn’t flow freely in most organizations
Harvard Business School professor Amy Edmondson and Penn State professor James Detert have been exploring the challenges employees face speaking up to higher-ups at work. They define upward voice as “communications directed to someone higher in the organizational hierarchy with the perceived power or authority to take action on the problem or suggestion.” In the recent article Do I Dare Say Something? they discussed their working paper called “Latent Voice Episodes: The Situation-Specific Nature of Speaking up at Work”(abstract) with HBS Working Knowledge:

[In] the modern economy, most of us depend on hierarchical organizations and their agents (i.e., bosses) to meet many of our basic needs for economic support and human relationships. Thus, fear of offending those above us is both natural and widespread.

Yet, even in [a climate of open, honest communication] people have to speak up to specific individuals, and our research suggests that people can be afraid to speak up to their boss even when the overall organizational climate appears conducive to voice.

Information erodes on its trip through the management ranks
In The Inner Game of Work, author W. Timothy Gallwey describes how challenging it can be to upload a clear message from one mind to another:

It takes a lot of attention to really understand what is being said by another person. Even in the transfer of simple information, it has been well established by researchers that people generally don’t hear what others are saying. We hear what we expect to hear. Compounding the problem, people don’t say what they really mean. Meaning can be disguised by the speaker’s efforts to be polite, to avoid judgment, or to make a good impression.

(For proof of Gallwey’s point, try the telephone game.)

In a June 2004 article for Wired magazine, Surowiecki adds even more meat to the information-erosion bone:

[Executives] have a hard time getting the information they need — at so many firms the flow of information is shaped by political infighting, sycophancy, and a confusion of status with knowledge.

People tend to believe a smooth talker – even when the smooth talker is wrong, and especially when the smooth talker is a leader
I’d prefer to spread the love rather than credit Kathy Sierra with another good post, but she recently delivered one that covers this point too well to ignore: When only the glib win, we all lose

I see four musts for anyone who leads or manages others:

(1) read Wisdom of Crowds

(2) facilitate upward voice
Edmondson and Detert explain here:

[T]wo beliefs are essential preconditions for the free expression of upward voice: first, the belief that one is not putting oneself at significant risk of personal harm (e.g., embarrassment, loss of material resources) and second, the belief that one is not wasting one’s time in speaking up. In short, voice must be seen as both safe and worthwhile.
This makes facilitating voice every manager’s job.

(3) Think separately, then mix
Tom Peters provided an excellent example of this:

[Traditional Approach:] You’ve got a huge marketing decision to make post haste. You gather 10 experts in the field. Lock them in a room for 72 hours. Ask them to come up with a best estimate of, say, success of a New Product you’re close to launching. The process is better than nothing—maybe.

Alternate: Select 10 experts from disparate fields, some closely associated with the decision at hand, some not. Tell each one to stay isolated in his or her individual office, lock and bar the door, turn off all phones and computers—and come up with a best estimate in 72 hours, which will then be emailed to you. You, in turn, average their estimates and take the result as the collective output. This process/result is likely to be … Solid Gold!

(4) explore internal use of prediction markets

I’ll end this lengthy post with an interesting thought by Ben Casnocha, an 18-year-old high school student and successful small businessman from San Francisco (thanks to Dennis Howlett for the pointer):

“The more successful person is the one who can facilitate the intelligence in the room.”
(From Are You The Smartest Person in the Room?)

P.S. - I did purchase Surowiecki’s book – just five bucks at a local Half Price Books store!