Wisdom of the Crowds
January 31st, 2008 Richard Posted in web and new media, wisdom_of_crowds |
Our theme for this week is virtual communities/crowds/mobs. As far as the crowds, I saw an excellent talk at the 2006 SXSW interactive conference given by James Surowiecki. This whole 1 hour talk is on line on the sxsw site. The talk is loaded with many examples that support his ideas, and theories/research from a number of sources. If we accept these things, the most important question becomes: what factors are important in determining when “crowd wisdom” will be more accurate than expert wisdom, so I made a seven minute clip, from different places in his talk, that focus on these important factors: objective aggregation (my term), diversity, independence.
February 4th, 2008 at 12:41 pm
This was an interesting talk. It is clear that the “wisdom of the crowds” can expose or create new knowledge: the Page Rank algorithm is an excellent example. I did feel like I wanted more information about his model–what was his evidence that his assumptions are necessary and sufficient, and how does one determine which applications do so? I want a framework for when to use a public-at-large wisdom-of-crowds models and when to use expert systems and published papers.
Prediction markets, when played fairly, are also useful, but can be manipulated. I believe that as the government, media, and industry begin to pay more attention to prediction markets, there will be more temptation for these players to exploit loopholes or to bid their own positions way up. However, if important decisions are not based on short-term fluctuations, we can expect a certain amount of self-correction: if prediction markets are like the stock market, we can expect the loophole advantage to eventually get spread very thin over many people.
His three important factors are aggregating judgments, cognitive diversity, and independence. He implicitly bases these on the work on groupthink–simply reversing each factor. I think these are backed up by his citation of Asch’s work, which has also been used to conclude that smaller juries are more likely to bring guilty verdicts, especially against minorities. The big question, one he does address in the longer recording, is that independence can be hard to find. I think this is part of the initial power of the Page Rank algorithm, social bookmarking sites, etc.–it found patterns among things people had done passively, for their own use. How do we keep this from eventually turning to circularity, where everyone links to the top two results so that they are always the top two results, etc.?
February 4th, 2008 at 7:40 pm
When I first saw the post on Wisdom of Crowds, the topic sounded familiar and now I remember why.
http://headrush.typepad.com/creating_passionate_users/2005/03/one_of_us_iisi_.html
February 4th, 2008 at 8:07 pm
So I just finished listening to the 7 min version, which completely blew me away so i’m gonna have to listen to the hour talk now.
I posted the link in the previous post cause it was my first exposure to the idea of Wisdom in Crowds, sometime last year. I thought the idea of this was asinine. Jumped to conclusions I did. If you only knew how many times, with groups at work, I have been involved with some of the most hair-brained, well-intentioned people that the only benefit they brought was the days they missed calls. As was called out in the audio, I have left meetings where I DID feel dumber. Matter of fact, if your IQ could decrease cause of others, I would be in IQ debt.
This is the piece that makes this idea credible with me: The wisdom of crowds comes not from the consensus decision of the group, but from the aggregation of the ideas/thoughts/decisions of each individual in the group.
Forgive me, I have been jaded by 7 1/2 years in the corporate world
February 5th, 2008 at 7:52 am
This is my first exposure to Wisdom in Crowds. Like Shannon, I am perhaps jaded for other reasons; the politics and egos of academia. An idea may go forth simply because of who suggested it.
I typically cringe at the word “committee.” I have participated in many committees on campus and my experience has ranged from Shannon’s experience (basically a waste of my time) to one of satisfaction.
Ironically, one committee/group that I am a member of has changed my view: The Diversity Taskforce Initiative Committee. I was initially asked to participate as an IT adviser who helped them create a website. This group brings together faculty, staff, students, males, females, homosexuals, heterosexuals, blacks, whites, Hispanics, young, and old, in a monthly forum that consistently moves forward. The two important characteristics this group has which are reiterated in the clip: diversity and independence. Egos and politics are, for the most part, checked at the door.
The key is having a committee or group leader who is willing to utilize the concept of Wisdom of Crowds. Without that commitment, it’s business as usual.
February 5th, 2008 at 8:17 am
One clarification…I did not mean to insinuate that egos and politics are non-existent in the corporate world. Just that they are *extra-special* in the academic world.
February 5th, 2008 at 8:47 am
Some days I wonder if I am a designer or the un-distinguished senator from Tennessee. That’s when my ego kicks in and says their politics can kiss it. Granted i’d like to scream that from my soapbox many times.
February 5th, 2008 at 11:57 am
This was certainly an interesting talk. The use of examples was really helpful. When I come to think of it, the 3 important factors that he mentioned, aggregating judgments, diversity and independence seem really true.
Coming from south-east Asia, the only thing that ring bells in my head when someone says ‘crowd’ is the “tyranny of the masses”. That is one of the things I’ve been exposed to since childhood. Tyranny of the masses is the basic concept that sometimes the majority might be wrong. As Michelle mentioned, ‘independence’ can be hard to find. Whenever a group of people sit down to do something, there’s always an agenda. When there’s a lack of independence, crowd is regulated by the powerful few with their own agenda and bias. The mass mentality can be particularly frightening because once people as a whole are headed in a particular direction, there is an incredible amount of inertia that can be very hard to redirect. For me, the idea of ‘wisdom of crowd’ was certainly a change………like an ideal dream. To sum up, I really liked the notion that the wisdom of the crowd does not come from the consensus decision of the group but from the aggregation of the ideas/thoughts of each individual in the group.
February 5th, 2008 at 11:59 am
I concur with Holli that “The key is having a committee or group leader who is willing to utilize the concept of Wisdom of Crowds. Without that commitment, it’s business as usual”.
February 5th, 2008 at 12:37 pm
I think what he had to say was very true. Groups can be smarter with diversity but only if people think for themselves rather than going with a consensus. When people try and “go with the lowest common denominator” then groups are not that smart but if people think for themselves the group as a whole can be very productive.
February 5th, 2008 at 12:56 pm
Lowest common denominator = The dumbing down of projects in my world. Ticks me off that the assumption is made to plan to what people feel is the “dumbest person”. Only helps create “dumb” applications.
February 5th, 2008 at 6:15 pm
A couple of thoughts after class
Validity or factual nature of the internet:
Does this produce a more questioning society overall as the internet inserts itself into our lives? Is that a bad thing? Is this based on age?
I personally take everything I read on the internet with a few grains of salt….but not all do:
How do we (society) educate individuals to decipher the difference? Is that necessary?
Pedagogy and the Internet
As a young student, my only option was to learn from encyclopedias, books, etc. You read the book and that was it. The web seems to lend itself to proactive leaning. If users want to learn more from an article online, they simply click on a corresponding link within the text. To answer Greg/RH’s question, I think it encourages/increases our need for information. I am sure it mires some people but the same is true for any kind of learning. In very simplified terms: obtaining new info out about a type of music, cooking technique, technology, or a cool website, is learning. Transfer of new information=learning.
BTW- html tags are meant as a test