"Collective Intelligence"

[Spacebeard]

Bruce Sterling, to his credit, linked to an essay by Jason Lanier on the folly of the "collective wisdom" fad currently in vogue among trendy websites like Wikipedia and techno-utopians like Kevin Kelly. It's far too long to quote in its entireity, so here's the abstract:

The hive mind is for the most part stupid and boring. Why pay attention to it?

The problem is in the way the Wikipedia has come to be regarded and used; how it's been elevated to such importance so quickly. And that is part of the larger pattern of the appeal of a new online collectivism that is nothing less than a resurgence of the idea that the collective is all-wise, that it is desirable to have influence concentrated in a bottleneck that can channel the collective with the most verity and force. This is different from representative democracy, or meritocracy. This idea has had dreadful consequences when thrust upon us from the extreme Right or the extreme Left in various historical periods. The fact that it's now being re-introduced today by prominent technologists and futurists, people who in many cases I know and like, doesn't make it any less dangerous.

(link)

This is a subject I've been idly following for a while, and The Register has also published a number of articles on this fad:

Junk science - the oil of the new web

"People are fascinated by ways in which data-mining seems to represent some sort of over-mind. But sometimes there's no deep meaning at all. Dartboards are competitive with individual money managers - but nobody talks about the 'wisdom of darts'", he writes.

And today, Canadian hockey fans are rejoicing in the return of Maggie the Macaque. The simian (on the right) out-performed the experts in predicting the results of key games during the 2003 season. Could it be Maggie's diet of crabs, or could it be - "The Wisdom of Monkeys"?

One need only look at the composition of the internet to understand why the "Wisdom of Crowds" will never apply: the internet isn't representative of society, and even amongst this whiter-than-white sample, only a self-selecting few have any interest in participating in a given pseudo-market.

Nature mag cooked Wikipedia study

Hundreds of publications pounced on the Nature story, and echoed the spin that Wikipedia was as good as Britannica - downplaying or omitting to mention the quality gap. The press loves an upbeat story, and what can be more uplifting than the utopian idea that we're all experts - at whatever subject we choose?

Six startups from the Web 2.0 swamp

Can the "market" really tell where a hurricane will hit? We wouldn't bet New Orleans on it, but Inkling insists that the market is right about 75 to 80 per cent of the time, and claims to have data to support the premise. [ Chimps produce similar results - but you don't hear about the Wisdom of Chimps - ed ]. Veterans also have their doubts.

For my part, I started reading Digg shortly after first hearing about how superior its open, collective story selection process was to Slashdot's editorial staff. At first, those claims seemed justified: the site was updated far faster than Slashdot was, and it featured an eclectic mix of links to articles I wouldn't have found elsewhere. However, after a few months the site's quality seemed to plummet, with the front page being plagued by links to fluff blog posts obviously written solely in the hopes of a Digg link, and by obvious hoaxes such as doctored screenshots of a supposed "GoogleOS" and the story of a teenager claiming to have been hired by Google. It seemed to me that the site must have started with a smaller and more intelligent set of users who selected decent articles, but was shortly inundated with a horde of morons who uncritically accepted the most sensationalist stories. Slashdot's editorial staff may have been slower to update the site than Digg's automatically aggregated votes, and it may not have been au courant with all the latest buzzwords like "emergent" and "collective", but its quality was consistently higher.

Finally, I'm worried that the current popularity of the notion of "collective intelligence" may be linked to the ongoing erosion of trust in expertise. We already have politicians pushing the notion that schools should give "equal time" to science and pseudo-science in biology classes, proposals to predict future terrorist attacks using "the wisdom of the market", and the motto "the surest way to smartness is through massive dumbness" being promoted as a "rule of the new economy"; now we have fanboys who think Wikipedia is better than a university:

"Old World is under attack. The authority of the book, authority of the journalist, authority of the teacher, is under direct assault by Wikipedia and other online efforts," claims the poster, 'Stephen'.

"It should come as no suprise [sic] a journalist and teacher ganged up on Wikipedia. Both have much to loose [sic]. Their claim? Authority. We will see much more of this backlash by the old guard in the future," he continues, confidently.

"The education system its self [sic] will come into question eventually. Universities are formed around libraries and libraries are physical things that require physical campuses. Take away the library, provide full access to every book ever writen [sic] online, imagine the consequences."

(from here)

I'd highly recommend reading all of the Lanier article. What problems do you think "collective intelligence" can really be applied to solve? Do you also think that this fad represents another source of the notion that truth is a matter of consensus rather than evidence?

[Ariphaos]

I'd highly recommend reading all of the Lanier article. What problems do you think "collective intelligence" can really be applied to solve? Do you also think that this fad represents another source of the notion that truth is a matter of consensus rather than evidence?

I don't think it's a fad. In fact I'm pretty sure Wikipedia is here to stay, like it or not. It's not as if the new policy of citing sources is a step in the wrong direction, and honestly, it gives a better rundown on a wide variety of subjects than many textbooks I've read.

I think what we'll see happen is the creation of 'semipublic' projects, that can only be fully edited by people with credentials in a given field, perhaps with the public being allowed to ask for clarification, correct spelling, and so on. I'd honestly prefer that over Wikipedia, but no such place exists, really.

[Talanth]

What problems do you think "collective intelligence" can really be applied to solve? Do you also think that this fad represents another source of the notion that truth is a matter of consensus rather than evidence?

To quote from I've no idea where: "The inteligence of any crowd is the inteligence of its least inteligent member divided by the number of people in the crowd."

But more seriosely I've always been interested in what people think about colective inteligence (even if it is low inteligence). So thanks for the links!

[Talanth]

I think what we'll see happen is the creation of 'semipublic' projects, that can only be fully edited by people with credentials in a given field, perhaps with the public being allowed to ask for clarification, correct spelling, and so on. I'd honestly prefer that over Wikipedia, but no such place exists, really.

http://mathworld.wolfram.com/ is a site like you've described. Only acredited mathemetitions can submit or edit articles. They have several sister sites under the wolfram name that work under the same principle but for each of the sciences.

[Keevan_Colton]

Wikipedia gives you the sum over time of human ignorance on any given topic.

[Molyneux]

Wikipedia gives you the sum over time of human ignorance on any given topic.

Yet it has proven to be of great value to me on various fact-finding searches; it contains a veritable cornucopia of information regarding topics as divers as comic-book characters, biology, the nature of chemical elements, and theology.

I would love to have a similar source for, say, higher-level engineering information - something that would be truly technical, rather than an extremely broad but fairly shallow source - but without Wiki I would have had much more difficulty gaining information than I have.

And I always make it a point to correct any errors, grammatical, spelling or factual, that I find.

[FSTargetDrone]

And I always make it a point to correct any errors, grammatical, spelling or factual, that I find.

I find myself doing the same thing.

[Keevan_Colton]

Get back to me when Wiki embraces the scientific method and drops the NPOV bullshit completely and then we can start talking about what useful place it might occupy in the world.

It really is the sum of human ignorance because if more people subscribe to something stupid/wrong on a topic it is more likely to end up as the "facts" in wikipedia.

[Pezzoni]

I've found Wikipedia to be very useful on many occasions; it can proveide a lot on information on a subject all in one easily accessible place. What is important however, is to then check said information for accuracy and validity.

[Ariphaos]

http://mathworld.wolfram.com/ is a site like you've described. Only acredited mathemetitions can submit or edit articles. They have several sister sites under the wolfram name that work under the same principle but for each of the sciences.

Not entirely. Some of the articles there dive into insane amounts of jargon, and could be explained much better, but there is no easy place to offer commentary. If you want to bring knowledge to the public you must let the public bring its questions.

It'd be neat to set up a wiki where submitters to the main namespace must have some form of accreditation (or other form of recognition), but anyone could converse on the Talk pages.

Get back to me when Wiki embraces the scientific method and drops the NPOV bullshit completely and then we can start talking about what useful place it might occupy in the world.

It really is the sum of human ignorance because if more people subscribe to something stupid/wrong on a topic it is more likely to end up as the "facts" in wikipedia.

The cite sources policy goes nowhere with you, eh?

Rather than trying to correct it, set up or join a viable alternative, you bitch.

[Keevan_Colton]

Frankly it's only a small step up until those other two fucking go it's still absolutely shite. NPOV is the worst fucking idea for something seeking to fill the role of an encyclopedia.

[Keevan_Colton]

Oh and let me add, I have suggested a way to improve it:

Ditch NPOV and embrace the Scientific Method.

...and I dont need to join an alternative, the good people at Britannica do a much better job than I could at providing one.

[Spacebeard]

My problem isn't with anyone using Wikipedia casually as a source of obscure trivia, or as a starting point for later fact-checking: it's with the fanatics who are so enamored of the notion of the "wisdom of the crowd" that they think an encyclopedia with an "anyone can edit" policy can seriously challenge the quality of professionally edited reference works.

There's a serious problem when a supposedly reputable journal is releasing doctored articles to push the notion that the gestalt of constant edits from random anonymous Internet users are equivalent in quality to articles written by experts in their field, and when venture capital money and government grants are being given to fund efforts to "predict" terrorist attacks and natural disasters using betting markets.

Rather than trying to correct it, set up or join a viable alternative, you bitch.

Do you mean a viable alternative like this, or like this?

[Ariphaos]

Britannica is quickly losing comprehensiveness in comparison.

Frankly it's only a small step up until those other two fucking go it's still absolutely shite. NPOV is the worst fucking idea for something seeking to fill the role of an encyclopedia.

I suppose this is a bit about how you use it.

I tend to use Wikipedia to look up articles which are best described as religion, politics and modern culture. The scientific method doesn't readily apply, but the NPOV damned-well does. The NPOV policy has drastically improved the quality of these kinds of articles, in my opinion, and I think it's a good thing (a few gaffes aside, but even Britannica has those).

This tends to leave your scientific method suggestion, which would be valid, but what do they use it on? They have a 'no original research' policy, and its stated purpose is a gathering of knowledge, not the development of it.

[Ariphaos]

My problem isn't with anyone using Wikipedia casually as a source of obscure trivia, or as a starting point for later fact-checking: it's with the fanatics who are so enamored of the notion of the "wisdom of the crowd" that they think an encyclopedia with an "anyone can edit" policy can seriously challenge the quality of professionally edited reference works.

Honestly, after the research projects I had to go through, I have a poor view of encyclopedias in general. I found errors in Britannica, Americana, World Book, Compton's and Encarta (the first set of errors were regarding certain portions of African and European history), but there's nothing I could do about it except mail them a letter that takes an inordinate amount of time that would likely get ignored anyway. It was insanely frustrating, especially when information about the subject at hand is exceedingly sparse.

However, with Wikipedia, after I managed to finally find books that were worth a damn on the subjects in question, I could have entered in the relevant data, cited my sources, and saved other people after me the pain of searching for information about some dead empire nearly no one has ever heard of.

There's a serious problem when a supposedly reputable journal is releasing doctored articles to push the notion that the gestalt of constant edits from random anonymous Internet users are equivalent in quality to articles written by experts in their field, and when venture capital money and government grants are being given to fund efforts to "predict" terrorist attacks and natural disasters using betting markets.

I'm not disagreeing, so much as stating that there is a reason for Wikipedia and it's not going to go away. It fills a rather sore need, and the published, for-pay encyclopedias are not cutting it. I tried six of them and they failed me, three times in a row. Hell, I even ignored Wikipedia they'd left such a bad taste.

It's clear that a lot of professionals see a need for a more professional Wikipedia, and Wolfram is close, but it is not written to the public, and is not written in such a way that say, someone who wants to check up on calculus from a biology entry can easily get there.

Do you mean a viable alternative like this, or like this?

Nope, think something like Wolfram, but one site, that allows for public discourse about a subject. Maybe slightly more open.

Hell, it's a surprise that Brittanica now can tell you, sort of, what an Ogiso is.

[Keevan_Colton]

Britannica wins hands down every single time on the quality front though...and what fucking use is a poor quality reference work?

Wikipedia is a shitty substitute for fucking morons that cant construct a proper web search. Any thing you read in it you must independantly verify yourself to be sure it isnt just bullshit that hasnt been caught yet.

So, here's a hint, why not cut the middle man and actually research in relevant verified sources in the first place.

It's a tool for morons by morons.

I'm frankly sick of the wiki-wanking that prevades the web...yay, an overgrown fucking message board where everyone has mod powers!

[Darth Wong]

Britannica is quickly losing comprehensiveness in comparison.

I'd rather have incomplete but reviewed and researched material than a comprehensive crapshoot.

[Spacebeard]

My problem isn't with anyone using Wikipedia casually as a source of obscure trivia, or as a starting point for later fact-checking: it's with the fanatics who are so enamored of the notion of the "wisdom of the crowd" that they think an encyclopedia with an "anyone can edit" policy can seriously challenge the quality of professionally edited reference works.

Honestly, after the research projects I had to go through, I have a poor view of encyclopedias in general. I found errors in Britannica, Americana, World Book, Compton's and Encarta (the first set of errors were regarding certain portions of African and European history), but there's nothing I could do about it except mail them a letter that takes an inordinate amount of time that would likely get ignored anyway. It was insanely frustrating, especially when information about the subject at hand is exceedingly sparse.

However, with Wikipedia, after I managed to finally find books that were worth a damn on the subjects in question, I could have entered in the relevant data, cited my sources, and saved other people after me the pain of searching for information about some dead empire nearly no one has ever heard of.

And then once you entered it into Wikipedia, any random moron who hadn't read any of those books could flag it for deletion, vandalize it, twist it to fit a "neutral point of view", or insert total fabrications that will remain until someone bothers to remove them. The sum of all of these edits would produce a worse result than your hypothetical original article backed by research, not some mystical "wisdom from the hive mind".

There's a serious problem when a supposedly reputable journal is releasing doctored articles to push the notion that the gestalt of constant edits from random anonymous Internet users are equivalent in quality to articles written by experts in their field, and when venture capital money and government grants are being given to fund efforts to "predict" terrorist attacks and natural disasters using betting markets.

I'm not disagreeing, so much as stating that there is a reason for Wikipedia and it's not going to go away. It fills a rather sore need, and the published, for-pay encyclopedias are not cutting it.

I'm not saying that Wikipedia will or should go away; just that it cannot supercede the quality of well-researched reference works, or make education obsolete, or any of the other things its most fanatical supporters say it will do.

Do you mean a viable alternative like this, or like this?

Nope, think something like Wolfram, but one site, that allows for public discourse about a subject. Maybe slightly more open.

If the Wolfram site privileges the opinions of experts over those of random anonymous users with no accountability, then I have no problem with it. The OP is about people wanking over the notion of the "wisdom of the crowd" or "wisdom of the market", not about the concept of a wiki per se.

[Ariphaos]

And then once you entered it into Wikipedia, any random moron who hadn't read any of those books could flag it for deletion, vandalize it, twist it to fit a "neutral point of view", or insert total fabrications that will remain until someone bothers to remove them. The sum of all of these edits would produce a worse result than your hypothetical original article backed by research, not some mystical "wisdom from the hive mind".

There is merit in the idea, however (and I am perhaps biased since I don't have much of a time investment in the project. The Brian Peppers fiasco was annoying though). Just because it's ridden with flaws doesn't mean you can't get something from it.

I was going to point out this article as being an example of someone putting crap there, but it has been seriously cleaned up since. It was originally the only source you could find on Google on Meganthropus (and was horrible, see the first revisions), but now has some sources that one may actually decide to look up.

I'm not saying that Wikipedia will or should go away; just that it cannot supercede the quality of well-researched reference works, or make education obsolete, or any of the other things its most fanatical supporters say it will do.

I suppose we're not arguing about much, then.

I do think the current educational system is pining for obsolesence, but that's a different debate.

If the Wolfram site privileges the opinions of experts over those of random anonymous users with no accountability, then I have no problem with it. The OP is about people wanking over the notion of the "wisdom of the crowd" or "wisdom of the market", not about the concept of a wiki per se.

When I originally saw the thread title I thought it was about the concept of civilizations as singular beings finding superior solutions through time, trial and error than its individual subcomponents, which is an idea that I find has merit overall and thus, give Wikipedia credit as a kind of semi-successful experiment. :-)

[Mr. Coffee]

Just because it's ridden with flaws doesn't mean you can't get something from it.

My Pop used to say "Son, you can make a sculpture out of a turd all you want. You still just end up with a pile of shit, not a statue".

The problem I have with Wiki is people using it as an "end all/be all" reference. It's good as a starting point, but only IF the current article contains good information when you look it up. But the problem is, Wiki is constantly updated/edited by random people.

Ever play that game in kindergarden called "telephone"? The one where you sat in a circle with a bunch of poeple and the first people would whisper something to the person next to them, that person would then whisper it to the next, and so on. By the time it got back to the original sender the message was usually garbled eight ways to Tuesday.

That's the problem with open editing. After a while errors WILL creep in and the information contained in a given article becomes questionable.

[Surlethe]

I wonder if the distortion in an article could be modeled a priori. Would there be any way to even quantify the accuracy vs. inaccuracy a Wikipedia article (or, generally, any non-fiction article) contains?

[Spacebeard]

And then once you entered it into Wikipedia, any random moron who hadn't read any of those books could flag it for deletion, vandalize it, twist it to fit a "neutral point of view", or insert total fabrications that will remain until someone bothers to remove them. The sum of all of these edits would produce a worse result than your hypothetical original article backed by research, not some mystical "wisdom from the hive mind".

There is merit in the idea, however (and I am perhaps biased since I don't have much of a time investment in the project. The Brian Peppers fiasco was annoying though). Just because it's ridden with flaws doesn't mean you can't get something from it.

No, but the fact that it's riddled with flaws does indicate that the utopian fantasies of its supporters should be seriously questioned. Kevin Kelly says that "the surest way to smartness is through massive dumbness"; applied to Wikipedia, the hope is that the sum of many small, individual, anonymous edits will be a closer approximation of the truth than a few long articles written by accredited experts. As it turns out, there are many worthwhile contributions but they are polluted by vandalism, petty rules-lawyer power-trips, and outright fabrications, making the overall result worse, not better.

I'm not saying that Wikipedia will or should go away; just that it cannot supercede the quality of well-researched reference works, or make education obsolete, or any of the other things its most fanatical supporters say it will do.

I suppose we're not arguing about much, then.

No, we're not; I really shouldn't have mentioned Wikipedia so much in the OP, since the real issue I'm interested in is the underlying idea that an aggregate sample of the opinions of all Wikipedia editors, participants in a futures market, blog authors, or any other "crowd" or "market" can approximate the truth as well as a small number of experts, or predict the future better than a Magic 8-ball or entrail-gazing.

I wonder if the distortion in an article could be modeled a priori. Would there be any way to even quantify the accuracy vs. inaccuracy a Wikipedia article (or, generally, any non-fiction article) contains?

I don't know, but if there is, the interesting question would be how its accuracy changes over time. The hope of wiki advocates is that the sum of many small edits will more closely approximate the truth over time; it seems to me that in many cases it rapidly fluctuates as disputed changes are repeatedly inserted and then reverted, without any real increase in accuracy.

[Surlethe]

I wonder if the distortion in an article could be modeled a priori. Would there be any way to even quantify the accuracy vs. inaccuracy a Wikipedia article (or, generally, any non-fiction article) contains?

I don't know, but if there is, the interesting question would be how its accuracy changes over time. The hope of wiki advocates is that the sum of many small edits will more closely approximate the truth over time; it seems to me that in many cases it rapidly fluctuates as disputed changes are repeatedly inserted and then reverted, without any real increase in accuracy.

And, of course, those fluctuations, while they may approximate accuracy over large time scales, defeat the entire purpose of an encyclopedia, which is continuous accuracy: i.e., at any given time, if you look something up, its article should be accurate.

[Ender]

Britannica is quickly losing comprehensiveness in comparison.

You're right. Why the hell would I want a detailed and accurate source of information dealing with the real world when I can have an arbitrary one that has a multipage entries for pornstars, superheros, and that fucking "O RLY" owl?

[Molyneux]

And then once you entered it into Wikipedia, any random moron who hadn't read any of those books could flag it for deletion, vandalize it, twist it to fit a "neutral point of view", or insert total fabrications that will remain until someone bothers to remove them. The sum of all of these edits would produce a worse result than your hypothetical original article backed by research, not some mystical "wisdom from the hive mind".

There is merit in the idea, however (and I am perhaps biased since I don't have much of a time investment in the project. The Brian Peppers fiasco was annoying though). Just because it's ridden with flaws doesn't mean you can't get something from it.

No, but the fact that it's riddled with flaws does indicate that the utopian fantasies of its supporters should be seriously questioned. Kevin Kelly says that "the surest way to smartness is through massive dumbness"; applied to Wikipedia, the hope is that the sum of many small, individual, anonymous edits will be a closer approximation of the truth than a few long articles written by accredited experts. As it turns out, there are many worthwhile contributions but they are polluted by vandalism, petty rules-lawyer power-trips, and outright fabrications, making the overall result worse, not better.

I'm not saying that Wikipedia will or should go away; just that it cannot supercede the quality of well-researched reference works, or make education obsolete, or any of the other things its most fanatical supporters say it will do.

I suppose we're not arguing about much, then.

No, we're not; I really shouldn't have mentioned Wikipedia so much in the OP, since the real issue I'm interested in is the underlying idea that an aggregate sample of the opinions of all Wikipedia editors, participants in a futures market, blog authors, or any other "crowd" or "market" can approximate the truth as well as a small number of experts, or predict the future better than a Magic 8-ball or entrail-gazing.

I wonder if the distortion in an article could be modeled a priori. Would there be any way to even quantify the accuracy vs. inaccuracy a Wikipedia article (or, generally, any non-fiction article) contains?

I don't know, but if there is, the interesting question would be how its accuracy changes over time. The hope of wiki advocates is that the sum of many small edits will more closely approximate the truth over time; it seems to me that in many cases it rapidly fluctuates as disputed changes are repeatedly inserted and then reverted, without any real increase in accuracy.

So, Wikipedia's accuracy and reliability would greatly improve if there were a means of permanently preventing idiots from vandalizing it?

Well, I do have a baseball bat handy...