• Welcome to the new Internet Infidels Discussion Board, formerly Talk Freethought.

The Dunning-Kruger Effect Debunked

R

rousseau

Contributor
Joined
Jun 23, 2010
Messages
13,471
Here's an interesting article I came across recently. In a nutshell, it explains that a statistical error was made in Dunning and Kruger's original paper, and critiques have since debunked the effect. But the original paper is far more widely cited, so a lot of people believe the effect is real.

economicsfromthetopdown.com

The Dunning-Kruger Effect is Autocorrelation – Economics from the Top Down

Do unskilled people actually underestimate their incompetence?
economicsfromthetopdown.com economicsfromthetopdown.com
 
Could we say Dunning and Kruger are victims of the Dunning-Kruger effect? o_O

Or not...
 
Now I have to find another explanation for Donald Trump's behavior. Ideas? Suggestions?
 
I don't find their argument convincing.

I also find their test description limited--what "skills" test?
 
I read through the comments. Interesting. Lots of disagreement. And why group the results? Plot them on an individual basis.

Loren Pechtel said:
I don't find their argument convincing.

I also find their test description limited--what "skills" test?
Click to expand...

By "their" do you mean DK?
 
Apparently the argument I cited has it's critiques too, after some Googling.
 
Everybody thinks themselves smart until they get asked, "African or European swallow?"

This sounds more like an methodology problem than a statistical problem, but it's no surprise to see someone attempt to prove the obvious and find a way to analyze data that confirms the obvious.

All the graphs are fun to read, but how does a person assess their own skill? I don't know of any arbitrary scale one can use. If someone wants to know their height, any measuring device longer than they are tall will provide a fairly accurate number. There's not really any uncertainty.

Now ask a person how well they can estimate someone else's height. How does a person measure something like that? It has to be expressed as a number in some form. Is there a special DK group of people who think they are very good at quantifying their ability to quantify things that have no reference scale?

I don't know how Dunning and Kruger conducted the experiments, but I'm willing to bet a six pack of Coca Cola in 6&1/2 oz bottles that a group of randomly chosen drivers would assess their ability to drive safely as good to excellent. I'll bet another six pack that an analysis of dash cams and vehicle data recorders would show most, if not all greatly overestimated their ability.

This brings up another problem. These kinds of experiments only can only give valid data for groups of people who are proficient at whatever skill is being considered and it must be the same skill. Ask me how well I can perform a heart bypass operation and I'll gladly concede the patient would be better off living with chest pains. Ask me how well I can properly adjust camber, caster, and toe-in on a 1973 Ford LTD and I will ask if it has radial or bias ply tires. I don't think Dunning and Kruger could assemble very large data sample of heart surgeons or front end mechanics.
 
Bronzeage said:
a group of randomly chosen drivers would assess their ability to drive safely as good to excellent.
Click to expand...
I'm sure the idiot who tried to overtake my 18m (59ft) long articulated bus while it was turning a corner from the only turning lane thought his time-saving idea was a stroke of genius, right up until he arrived at the front of the bus to discover that there wasn't enough space for his car to squeeze through, leaving him blocking one of the straight through lanes on Wickham Terrace (one of the busiest locations in Brisbane) in the middle of the morning rush hour.

The folks stuck behind him, unable to progress through the green light, were quite vocal in their disagreement with his assessment of his genius.
 
rousseau said:
Here's an interesting article I came across recently. In a nutshell, it explains that a statistical error was made in Dunning and Kruger's original paper, and critiques have since debunked the effect. But the original paper is far more widely cited, so a lot of people believe the effect is real.

economicsfromthetopdown.com

The Dunning-Kruger Effect is Autocorrelation – Economics from the Top Down

Do unskilled people actually underestimate their incompetence?
economicsfromthetopdown.com economicsfromthetopdown.com
Click to expand...
It's not a statistical error. It's a methodological error. And it's not really an error (more presentation peculiarity), conclusion is still valid. Dumb people think that they are smarter than they actually are and smart people unaware how dumb the rest of the people are.
Original plot is weirdly organized but it's not correlation plot, it does not have to comply with this auto-correlation thing.
Dunning and Kruger are psychologists, not physicists or even mathematicians.
And effect itself is kinda obvious to say the least.
 
Bronzeage said:
the graphs are fun to read, but how does a person assess their own skill?
Click to expand...
I believe participants were asked to place themselves in a percentile before being tested, to guess their level of proficiency relative to others. Don’t know the subject but I feel certain it was something more general. Then they were tested.
 
T.G.G. Moogly said:
Bronzeage said:
the graphs are fun to read, but how does a person assess their own skill?
Click to expand...
I believe participants were asked to place themselves in a percentile before being tested, to guess their level of proficiency relative to others. Don’t know the subject but I feel certain it was something more general. Then they were tested.
Click to expand...
This means making a judgment about oneself and others, and then putting a number on it. This sounds like being introduced to three people one has never seen before and then asked to predict who will win a game of trivial pursuit.
 
Bronzeage said:
T.G.G. Moogly said:
Bronzeage said:
the graphs are fun to read, but how does a person assess their own skill?
Click to expand...
I believe participants were asked to place themselves in a percentile before being tested, to guess their level of proficiency relative to others. Don’t know the subject but I feel certain it was something more general. Then they were tested.
Click to expand...
This means making a judgment about oneself and others, and then putting a number on it. This sounds like being introduced to three people one has never seen before and then asked to predict who will win a game of trivial pursuit.
Click to expand...
That's essentially correct. One makes a judgement as to where they think they stand knowledge-wise on a given subject. Then everyone takes the test and you find out where you stand. You also find out whether you overestimated your knowledge level relative to your peers or underestimated your knowledge level. The participants were undergrads at university.

My thoughts on DK are that if I know a lot about a given subject I also know that there's lots I don't know. Conversely, if I'm uninformed on a given subject I may not appreciate the degree of my ignorance and fool myself into thinking I'm smarter than I am.
 
T.G.G. Moogly said:
My thoughts on DK are that if I know a lot about a given subject I also know that there's lots I don't know. Conversely, if I'm uninformed on a given subject I may not appreciate the degree of my ignorance and fool m
Click to expand...

Really this.

To me, Dunning-Kruger is a just a label for an excruciatingly common human phenomenon. Also expressed as "A little knowledge is a dangerous thing."

It happens a lot. Somebody reads a Wikipedia article and thinks they understand "Eastern Religions",
or "Evolution"
or "WWII".
They think the Econ 202 class in 1993 makes them an expert on the Federal Treasury Department.
They think that maintaining and repairing the motor in their '73 Chevy means that they can diagnose the overheating problem in their 2008 Toyota.

From marriages to pharmacy to global politics, to climate change, people often wildly overestimate their grasp of a subject.
Tom
 
TomC said:
T.G.G. Moogly said:
My thoughts on DK are that if I know a lot about a given subject I also know that there's lots I don't know. Conversely, if I'm uninformed on a given subject I may not appreciate the degree of my ignorance and fool m
Click to expand...

Really this.

To me, Dunning-Kruger is a just a label for an excruciatingly common human phenomenon. Also expressed as "A little knowledge is a dangerous thing."

It happens a lot. Somebody reads a Wikipedia article and thinks they understand "Eastern Religions",
or "Evolution"
or "WWII".
They think the Econ 202 class in 1993 makes them an expert on the Federal Treasury Department.
They think that maintaining and repairing the motor in their '73 Chevy means that they can diagnose the overheating problem in their 2008 Toyota.

From marriages to pharmacy to global politics, to climate change, people often wildly overestimate their grasp of a subject.
Tom
Click to expand...
Yup--because we comprehend our lack of knowledge only in the areas where we know enough to know we don't know. Things farther out we don't even know they exist so we don't know we don't know them. What would be strange is if there were no DK effect.
 
This article seemed on point to me when I first read it (hence posting), and I think there was a problem in the original study. But it looks like the ensuing academic conversation on Dunning-Kruger is a bit of a wormhole with a lot of facets to it.
 
Last edited:
  • Like
Reactions: WAB
You must log in or register to reply here.
Back
Top Bottom