"The Bell Curve": Twenty Years Later, Q&A With Charles Murray

[Loren Pechtel]

I already told you. We can use statistical and baysein logic to say that practice effects (or lack thereof) have a 95% chance of affecting the IQ by +/- X given a specific result on a test for a particular individual.

We use bayseian reasoning can be used to quantify uncertainty. The more information we know about an individual, the tighter we can make the range or the more confident we can be given a fixed range. That's the best we can do and still provides us very useful information, yet it is not acceptable to you for some reason. Why is that?

Throwing around statistical terms doesn't answer the question.

There is no statistic that tells you if an individual score has been effected by practice. If you think there is, he is an example.

Joe X had a score of 112 on an IQ test. Using any method you like please tell me (with a 95% confidence level) how much this score was effected by practice.

What you are missing is that just because we can't draw conclusions about the individual doesn't mean we can't draw conclusions about the group. Sometimes statisticians even do things specifically to ensure they can't derive data about the individuals.

Consider: You're charged with finding out how many people have stolen money from their workplace.

Obviously you aren't going to get good data if you ask "Have you ever stolen money from your workplace?" So lets make it impossible for the pollster to know. Instead, ask "Please take a coin from your pocket and flip it so only you can see it. If it's heads, answer yes, if it's tails answer truthfully: Have you ever stolen money from your workplace?". Answering yes to such a question carries no stigma because most people that answer yes are ones that got heads on the coin. Suppose 51% of people answer yes. The pollster knows that 2% (subject to the margin of error) of the people polled actually stole, yet he has no idea which ones they are.

 

[Togo]

Throwing around statistical terms doesn't answer the question.

There is no statistic that tells you if an individual score has been effected by practice. If you think there is, he is an example.

Joe X had a score of 112 on an IQ test. Using any method you like please tell me (with a 95% confidence level) how much this score was effected by practice.

What you are missing is that just because we can't draw conclusions about the individual doesn't mean we can't draw conclusions about the group. Sometimes statisticians even do things specifically to ensure they can't derive data about the individuals.

Consider: You're charged with finding out how many people have stolen money from their workplace.

Obviously you aren't going to get good data if you ask "Have you ever stolen money from your workplace?" So lets make it impossible for the pollster to know. Instead, ask "Please take a coin from your pocket and flip it so only you can see it. If it's heads, answer yes, if it's tails answer truthfully: Have you ever stolen money from your workplace?". Answering yes to such a question carries no stigma because most people that answer yes are ones that got heads on the coin. Suppose 51% of people answer yes. The pollster knows that 2% (subject to the margin of error) of the people polled actually stole, yet he has no idea which ones they are.

How many people followed the instructions correctly?

 

[untermensche]

Joe X had a score of 112 on an IQ test. Using any method you like please tell me (with a 95% confidence level) how much this score was effected by practice.

What you are missing is that just because we can't draw conclusions about the individual doesn't mean we can't draw conclusions about the group. Sometimes statisticians even do things specifically to ensure they can't derive data about the individuals.

If the amount of practice for any individual can't be determined by looking at a score the amount of practice of any group can't be determined either.

If you think so, here's a question.

A group of 25,000 people averaged 100 on an IQ test.

How much practice did the group do before the test?

 

[Metaphor]

That's not a test of the validity of the concept.

It most certainly is. If IQ were not related to any outcome whatever; if it did not predict academic success, financial success, even marital success (which it does), I'd say throw it out. It'd be meaningless.

How is that relevant to this discussion?

Because those people want to dismiss IQ as 'meaningless' or 'racist' when they don't know what they're talking about.

It isn't obvious unless you've familiar with such experiments.

Take two US students. Put one in a quiet air-conditioned hall with a No.2 pencil. Put the other on the floor of a grass hut, in the rain, with a goat, and a reed quill, and a paper in a language he's fluent, but not expert in. Do you think there will be a slant in results in favour of one or the other? If so, why?

Is this how you think IQ tests are administered? In the rain with a goat and a reed quill? Is this the absurdity we're meant to believe that explains a one standard deviation gap between Black and White scores?

 

[Metaphor]

For those who appear to believe the practice effect explains everything.

Let's say that every White person who has ever taken an intelligence test has practiced, practiced, practiced, and every Black person ever has not.

There is a known effect of a six-point increase (around a third of a standard deviation) when re-taking the same test (let alone a different test).

Practice effects could explain a third of the gap, at most, given the absurd scenario above where only White and every White practises.

 

[Loren Pechtel]

What you are missing is that just because we can't draw conclusions about the individual doesn't mean we can't draw conclusions about the group. Sometimes statisticians even do things specifically to ensure they can't derive data about the individuals.

If the amount of practice for any individual can't be determined by looking at a score the amount of practice of any group can't be determined either.

If you think so, here's a question.

A group of 25,000 people averaged 100 on an IQ test.

How much practice did the group do before the test?

If you want to figure out the effect of practice you have to administer multiple tests to the same people.

 

[Trausti]

For those who appear to believe the practice effect explains everything.

Oh, it's certainly more than practice. The Bell Curve came out in 1994 and Gould died in 2002. Much has been learned in the time since.

Despite decades of relying on standardized test scores to assess and guide education policy and practice, surprisingly little work has been done to connect these measures of learning with the measures developed over a century of research by cognitive psychologists studying individual differences in cognition. Psychologists now consider cognitive ability (few dare say “intelligence” anymore) to have two primary components: crystallized knowledge and fluid cognitive skills. Crystallized knowledge comprises acquired knowledge such as vocabulary and arithmetic, while fluid skills are the abstract-reasoning capabilities needed to solve novel problems (such as the ability to identify patterns and make extrapolations) independent of how much factual knowledge has been acquired. The terms were coined by the late psychologist Raymond Cattell, who first distinguished two types of intelligence. Cattell noted that one “has the ‘fluid’ quality of being directable at almost any problem,” while the other “is invested in particular areas of crystallized skills which can be upset individually without affecting others.”

Hundreds of studies show that, at any point in time, the two are highly correlated: people with strong fluid cognitive skills are at an advantage when it comes to accumulating the kinds of crystallized knowledge assessed by most standardized tests.

http://educationnext.org/what-effective-schools-do-cognitive-achievement/

Eva Krapohl, joint first author of the study, from the Institute of Psychiatry, Psychology & Neuroscience (IoPPN) at King's, says: "Previous work has already established that educational achievement is heritable. In this study, we wanted to find out why that is. What our study shows is that the heritability of educational achievement is much more than just intelligence -- it is the combination of many traits which are all heritable to different extents.
"It is important to point out that heritability does not mean that anything is set in stone. It simply means that children differ in how easy and enjoyable they find learning and that much of these differences are influenced by genetics."
The researchers found that the heritability of GCSE scores was 62%. Individual traits were between 35% and 58% heritable, with intelligence being the most highly heritable. Together, the nine domains accounted for 75% of the heritability of GCSE scores.

http://www.sciencedaily.com/releases/2014/10/141006152151.htm

 

[Togo]

That's not a test of the validity of the concept.

It most certainly is. If IQ were not related to any outcome whatever; if it did not predict academic success, financial success, even marital success (which it does), I'd say throw it out. It'd be meaningless.

That's great, but you're still talking about the reliability of the test as a predictor, not the validity of the concept. A child's shoe size is a reliable indicator of the distance from the Moon to the Earth, but that doesn't say anything about it's indicator

How is that relevant to this discussion?

Because those people want to dismiss IQ as 'meaningless' or 'racist' when they don't know what they're talking about.

Who precisely are you accusing of not knowing what they're talking about? Or are you just throwing mud randomly in the hope some will stick?

It isn't obvious unless you've familiar with such experiments.

Take two US students. Put one in a quiet air-conditioned hall with a No.2 pencil. Put the other on the floor of a grass hut, in the rain, with a goat, and a reed quill, and a paper in a language he's fluent, but not expert in. Do you think there will be a slant in results in favour of one or the other? If so, why?

Is this how you think IQ tests are administered?

No, it's an illustration of how culture can effect non-verbal reasoning score, and how practice effects as a class can effect more than just the retest score. When you say things like this:

Let's say that every White person who has ever taken an intelligence test has practiced, practiced, practiced, and every Black person ever has not.
There is a known effect of a six-point increase (around a third of a standard deviation) when re-taking the same test (let alone a different test).
Practice effects could explain a third of the gap, at most, given the absurd scenario above where only White and every White practises.

It strongly suggests you're not that familiar with these effects, since you seem to be assuming that practice effects would hit a cealing by retaking the same test. The split I illustrated shows how cultural differences can have a stronger effect. The US student in a grass hut is under no particular hardship in doing the test, the equipment works and he can read the questions, but the entire context of the test is going to damage his score. He's simply not familiar with the set up he is in, the tools he's using and the way the test is described to him. He'll manage, but he'll get a lower score. IQ tests are intended for people who are happy and familiar with a formal western test enviroment. Someone who is less familiar, say because his local school isn't a particularly good place to get work done, or because he's used to less formal settings, will get a lower score. Someone who comes from a more formal enviroment, say an Eastern culture where formality is practiced at home and testing starts very early in life and is taken very seriously, might get a higher score.

People who come up with these issues are not exercising their imaginations - these are well-established problems around experimental design in cognitive psychology, that scientists spend years learning about and trying to control. That's why you can't just decide that because you've taken a few elementry precautions any effect you discover must be something fundamental to the human condition. The science doesn't work that way, and is known not to work that way.

 

[untermensche]

For those who appear to believe the practice effect explains everything.

Let's say that every White person who has ever taken an intelligence test has practiced, practiced, practiced, and every Black person ever has not.

There is a known effect of a six-point increase (around a third of a standard deviation) when re-taking the same test (let alone a different test).

Practice effects could explain a third of the gap, at most, given the absurd scenario above where only White and every White practises.

It is one factor.

And there may be many more.

What is the long term effect on groups subjected to hundreds of years of slavery and then a hundred years of legal discrimination?

How do we even begin to remove this as a factor?

 

[Metaphor]

For those who appear to believe the practice effect explains everything.

Let's say that every White person who has ever taken an intelligence test has practiced, practiced, practiced, and every Black person ever has not.

There is a known effect of a six-point increase (around a third of a standard deviation) when re-taking the same test (let alone a different test).

Practice effects could explain a third of the gap, at most, given the absurd scenario above where only White and every White practises.

It is one factor.

And there may be many more.

What is the long term effect on groups subjected to hundreds of years of slavery and then a hundred years of legal discrimination?

How do we even begin to remove this as a factor?

It depends on what you mean by 'remove'.

You cannot turn back the hands of time. That is certain. To the extent that intelligence, as reflected in IQ scores, can be maximised by environmental changes, we should strive to implement those changes. To the extent that it can't....well, it can't.

 

[Metaphor]

That's great, but you're still talking about the reliability of the test as a predictor, not the validity of the concept. A child's shoe size is a reliable indicator of the distance from the Moon to the Earth, but that doesn't say anything about it's indicator

If the concept of intelligence were nonsense, no test that claimed to measure intelligence would predict anything.

Are you now suggesting that intelligence does not exist?

I don't understand what on earth you mean by the shoe size analogy. Your last sentence is incomprehensible.

Who precisely are you accusing of not knowing what they're talking about? Or are you just throwing mud randomly in the hope some will stick?

Anyone who demonstrates their ignorance of psychometrics in general and intelligence tests in particular.

No, it's an illustration of how culture can effect non-verbal reasoning score, and how practice effects as a class can effect more than just the retest score. When you say things like this:

I'm sorry, what do the test conditions of a particular intelligence test (there are very specific rules about how they are administered) have to do with 'culture'?

It strongly suggests you're not that familiar with these effects, since you seem to be assuming that practice effects would hit a cealing by retaking the same test.

On the WAIS-IV, it does.

The split I illustrated shows how cultural differences can have a stronger effect. The US student in a grass hut is under no particular hardship in doing the test, the equipment works and he can read the questions, but the entire context of the test is going to damage his score. He's simply not familiar with the set up he is in, the tools he's using and the way the test is described to him. He'll manage, but he'll get a lower score. IQ tests are intended for people who are happy and familiar with a formal western test enviroment. Someone who is less familiar, say because his local school isn't a particularly good place to get work done, or because he's used to less formal settings, will get a lower score. Someone who comes from a more formal enviroment, say an Eastern culture where formality is practiced at home and testing starts very early in life and is taken very seriously, might get a higher score.

Are you seriously claiming that the experience of some students in America is equivalent to taking a test in the rain next to their mud hut and a goat?

People who come up with these issues are not exercising their imaginations - these are well-established problems around experimental design in cognitive psychology, that scientists spend years learning about and trying to control. That's why you can't just decide that because you've taken a few elementry precautions any effect you discover must be something fundamental to the human condition. The science doesn't work that way, and is known not to work that way.

Who said anything about 'fundamental' to the human condition? Are you talking about intelligence (which certainly is fundamental), or the Black-White IQ score gap?

 

[Togo]

If the concept of intelligence were nonsense, no test that claimed to measure intelligence would predict anything.

??? Of course it would. That's the entire point of some of the commentary from Jensen that you posted.

Let's take a step back here. One of the central issues with IQ testing is that the evidence for it is correlational. IQ correlates with exam scores, with interview scores, and with various cultural success factors such as income and social class. The battery of tests contained within IQ, when combined, correlated strongly with these criteria. Now there are two possible explanations for this.

The first is that there is a real thing, called g, that IQ tests measure. This actual biochemical value then directly effects the various performance characteristics that are associated with it.

The second is there is no real thing, called, g, that IQ tests measure. Instead g is an abstraction made up of various different measures. Intelligence isn't a thing, it's just the average of the performance on a variety of related cognitive tests.

Through the kinds of correlational studies that IQ researchers use, you can't prove you're looking at one or the other.

So when you ask:

Are you now suggesting that intelligence does not exist?

I'm not sure what you're asking. Jensen and others have been quite vocal in arguing that it doesn't matter whether intelligence exists (is a thing, is reified) or not. It's manifestly not true that intelligence has to be an actual thing, has to actually exist, to be a reliable indicator of future performance. IQ tests could just be an abstract average of a basket of related abilities, function perfectly well as an approximate measure of performance on those abilities, and g, or general intelligence, would not exist at all.

If you have a cloud of points on a graph, and you draw a line of best fit through those points, to show the correlation, that doesn't in itself show that there is a hidden factor that controls all the other results that is represented by the line.

I don't understand what on earth you mean by the shoe size analogy. Your last sentence is incomprehensible.

The last few words should have been 'about whether it's valid.

The point being made is that there are many things in life that are strongly and reliably associated with each other. The shoe size of a child is very strongly correlated with the distance between the Moon and the Earth, because both increase over time. The moon drifts (very slowly) further away from the earth, and child's feet (quite quickly) grow in size. That doesn't mean the two are causally related in any way.

I'm sorry, what do the test conditions of a particular intelligence test (there are very specific rules about how they are administered) have to do with 'culture'?

Culture is just 'the way things are done around here'. Formal testing in an exam hall or similar is not normal for everyone, and people who are used to the cultural set up of a formal test environment may do better than those that aren't.

It strongly suggests you're not that familiar with these effects, since you seem to be assuming that practice effects would hit a cealing by retaking the same test.

On the WAIS-IV, it does.

No, it doesn't. 'Practice effects' is a term referring to wide variety of factors known to influence cognitive testing, from familiarity with the equipment through to boredom with the task, of which immediately retaking the same test is merely one.

People who come up with these issues are not exercising their imaginations - these are well-established problems around experimental design in cognitive psychology, that scientists spend years learning about and trying to control. That's why you can't just decide that because you've taken a few elementary precautions any effect you discover must be something fundamental to the human condition. The science doesn't work that way, and is known not to work that way.

Who said anything about 'fundamental' to the human condition? Are you talking about intelligence (which certainly is fundamental), or the Black-White IQ score gap?

I'm talking about IQ tests, and the claim that they measure something called g (general intelligence). The Bell Curve relies on the idea that IQ testing measures something fundamental to the human condition, whether that is g or something similar to g. IQ testing in general, does not.