Book Review: “The Wisdom of Crowds” by James Surowiecki

“The Wisdom of Crowds”

By James Surowiecki
Anchor; Reprint edition (Aug 16 2005)

James Surowiecki’s book is an entertaining summary of many, recent and not so recent, studies on crowd behavior. The book comes with many references, and provides quite a balanced assessment of current knowledge. Unlike what the title might suggest, Surowiecki’s book is not a praise of “The Wisdom of Crowds,” but rather an examination under which circumstances crowds are wise, and for which purposes this wisdom might be useful.

Surowiecki distinguishes between three different problems posed at a crowd: problems of cognition, coordination and cooperation. The book is divided into two parts. The first part offers a lot of examples for these problems and the crowds’ attempts to solve them. The second part looks into the question which conditions are necessary for a successful solution. The author identifies three such conditions: diversity, independence (of individuals from each other), and decentralization – though with qualifiers:

“[D]ecentralization works well under some conditions and not very well under others. In the past decade, it’s been easy to believe that if a system is decentralized, then it must work well. But all you need to do is look at a traffic jam – or, for that matter, at the U.S. intelligence community – to recognize that getting rid of a central authority is not a panacea. Similarly, people have become enamored of the idea that decentralization is somehow natural or automatic. [However,] it’s hard to make real decentralization work, and hard to keep it going, and easy for decentralization to become disorganization.”

This paragraph makes clear that understanding the ways crowds make decisions is necessary to set up a system such that decision making is smart. Intelligent organization requires thinking – and scientific research.

Surowiecki warns of factors that dumb down the decisions of groups, most notably skewing information, groupthink, and herding, all of which lead to suboptimal decisions, and potentially disastrous failures.

The cases discussed in the book draw on many examples, the recurring ones are betting markets, and the financial and economic system. The author also dedicates a chapter to the academic system, and ends with discussing politics. While the elaborations on the financial markets are extensive and insightful though somewhat repetitive, those on politics are well meant but vague, and those on academia are hopelessly naïve:

“The coin of the realm, for most scientists, is not cash but rather recognition. Even so, scientists are undoubtedly as self-seeking and as self-interested as the rest of us. The genius of the way science is organized, though, makes their self-interested behavior redound to the benefits of all of us. In the process of winning notoriety for themselves, they make the group – that is, the scientific community and then, indirectly, the rest of us – smarter.”

The chapter comes with very few, basically irrelevant references, and leaves me with the impression the author has next to no experience with academic research. Unfortunately, “the genius of the way science is organized,” is today severely affected by various pressures researchers are subject to in their decision making process, most notably financial and time pressure. That, together with, herding, lacking independence, a meanwhile completely thermalized information basis, and specialization and fragmentation which promotes groupthink, several of the factors Suroviecki previously identified as necessary for smart decision making are not fulfilled. And that doesn’t even touch on the question how well the “smartness” of the scientific community reaches “the rest of us.” For more details, see my post We have only ourselves to judge each other – which suggests the steps to be taking to allow the scientific community to be indeed a decentralized, smart crowd.

A central theme of “The Wisdom of Crowds” is that leadership by a single or few persons is unlikely to be superior to using the full available knowledge (of the group, the company, the community, ie the crowd). Humans tend to assign success and failure to single persons where it might instead have been simply a result of lucky or unlucky circumstances. A consistently better performance, so the author argues, cannot be achieved by picking “the” right person, but by accessing and aggregating the wisdom of the crowd.

It remains somewhat unclear throughout the book what role Surowiecki assigns to experts, though he writes in the afterword that the expert is of course necessary to provide information, for without information neither a crowd nor anybody else can make qualified decisions. He cautions however that many studies have shown that specialists - as all other people - tend to be overconfident about their knowledge and fall for the “Illusion of Knowledge,” ie they fail to see the limits of their own knowledge. If the role of the experts is to provide the crowd with information, then one should keep in mind that the way how information is communicated can spoil the ability to make good decisions, eg by making information seem more (or less) important than it actually is, or by attaching irrelevant details like what specific persons thought about it. (And let’s not even talk about the problem of simply inventing information.)

In any case, the book makes a very compelling case that there is a large unused potential in the wisdom of crowds and that, if we know how to tap onto this potential, we could use it to improve decision making processes in certain situations. Especially scientists as members of large communities, and as members of academic institutions, can learn a lot from better understanding under which circumstances which decision making processes have been successful. The book conveys an optimistic, but also a cautious message, since there are many examples of stupid crowds as well. While the examples in the book fit very well into the argument the author is leading, I am (as often) left to wonder whether there are examples that did not fit into the theme and thus are not to be found in the book.

All together, “The Wisdom of Crowds” is a very recommendable book, informative and well written. If this was an Amazon review, I’d give five stars.

Read my other book reviews.