"The Bell Curve": Twenty Years Later, Q&A With Charles Murray

[Axulus]

The average height in Japan has risen in the last 60 years.

The genes are the same.

Genes are only part of the story. But how do we know when genes for height are being fully realized?

You could do a study, but in the study to look at the effects of genes you would have to have all subjects experiencing the same environment.

Factors don't need to be 'eliminated' for something else to be a 'key' factor.

That's the scientific method. To study the effects of any variable you have to keep all other variables constant.

It seems to me there's almost nothing you would accept as a genetic difference. Do you think the height difference between men and women is genetic?

Since men and women in the same environments have differences in height you can say more about the genes. The key is controlling the environment to the greatest extent possible.

My contention and it may be the contention of others is that many black Americans don't live in the same environment as many white American's.

Their vision of the society is different. And therefore their reactions to institutions, like school, will be different.

IQ in a way tests your feelings towards your education. It tests how motivated you were to learn in the past. It tests what things you have been motivated to think about.

These things can easily change.

You are failing to understand the statistical techniques that can be employed to tease out the genetic component even when the environments aren't the same. You seem to be saying that the environment must be perfectly controlled and kept perfectly the same when we study two different groups to say anything about the genetic component, which is false.

 

[Metaphor]

Intelligence is not as static as eye colour, but it does not vary wildly either, except downwards as the result of traumatic brain injury or some other illness or event.

It can vary at least an entire standard deviation with just changes to environment.

In a population over generations, not within individuals (except downwards, as previously mentioned).

We are talking about a test that comes with a warning not to allow those taking it to practice taking IQ tests, in case that distorts the results? And for good reason, since practice in assembling your own IQ tests and understanding how the questions are put together can raise your score.

For the WAIS IQ test, the typical test-retest increase in scores (for the exact same test) is 6 points -- or a third of a standard deviation. The 'practice effect' of IQ scores has been absurdly exaggerated by those who wish to reject the validity of intelligence tests, despite the robust psychometric properties displayed by good IQ tests.

If one really believed that IQ tests measure nothing but practice effects, we should see no difference in life outcomes between someone scoring 70 and someone scoring 130. But you would see vast differences in two such people.

 

[Axulus]

But you would see vast differences in two such people.

This can not be emphasized enough. Whatever criticisms one can offer about IQ tests, the fact that the score can be used as a fairly reliable predictive factor for future life outcomes is absolutely key. If it offered no such predictive factor, it would be rightly rejected and tossed out and not discussed again.

Same with any test, such as college admissions tests.

 

[Metaphor]

The average height in Japan has risen in the last 60 years.

Yes; and the average height in Europe has risen in the last 60 years.

The genes are the same.

And the genes in European populations are the same. That doesn't mean that European genes aren't different to Japanese genes, despite the fact that both populations have risen in stature!

Genes are only part of the story. But how do we know when genes for height are being fully realized?

You could do a study, but in the study to look at the effects of genes you would have to have all subjects experiencing the same environment.

No, you wouldn't. If that was necessary, almost all of the medical literature would disappear.

That's the scientific method. To study the effects of any variable you have to keep all other variables constant.

Since we cannot randomly allocate people to races (or any kind of ethnic grouping), there can be no true experiment testing for race or ethnicity. But there are powerful quasi-experimental techniques (including post-hoc statistical corrections, like analysis of covariance) that are powerful tools to analyse cause and effect.

Since men and women in the same environments have differences in height you can say more about the genes. The key is controlling the environment to the greatest extent possible.

Do you think ethnic Asians in America have had the same environment as ethnic Whites in America? If not, what makes you think women have had the same environment as men?

My contention and it may be the contention of others is that many black Americans don't live in the same environment as many white American's.

Their vision of the society is different. And therefore their reactions to institutions, like school, will be different.

IQ in a way tests your feelings towards your education. It tests how motivated you were to learn in the past. It tests what things you have been motivated to think about.

These things can easily change.

I would say, even if what you suspect about IQ measures is true, that things cannot be 'easily' changed, otherwise they would have changed.

 

[untermensche]

You are failing to understand the statistical techniques that can be employed to tease out the genetic component even when the environments aren't the same. You seem to be saying that the environment must be perfectly controlled and kept perfectly the same when we study two different groups to say anything about the genetic component, which is false.

How do you tease out how much a person cared about the IQ test they were taking?

If a certain group on average cares less due to environmental factors, how do you tease this out?

 

[Togo]

Intelligence is not as static as eye colour, but it does not vary wildly either, except downwards as the result of traumatic brain injury or some other illness or event.

It can vary at least an entire standard deviation with just changes to environment.

In a population over generations, not within individuals (except downwards, as previously mentioned).

We are talking about a test that comes with a warning not to allow those taking it to practice taking IQ tests, in case that distorts the results? And for good reason, since practice in assembling your own IQ tests and understanding how the questions are put together can raise your score.

For the WAIS IQ test, the typical test-retest increase in scores (for the exact same test) is 6 points -- or a third of a standard deviation.

That's quite a big difference, and that's only a from a single repeat. Spend a few days working with the tests and you'd expect a bigger increase.

The 'practice effect' of IQ scores has been absurdly exaggerated by those who wish to reject the validity of intelligence tests, despite the robust psychometric properties displayed by good IQ tests.

Probably because they're really, really, obvious if you actually work with the tests, or construct your own, and are entirely inconsistent with some of the claims made about such tests.

If one really believed that IQ tests measure nothing but practice effects,

Why would you need to believe that? Turn the proposition on it's head. If one really believed that IQ measure nothing but inherent differences between individuals, you'd expect to see no practice effects at all. Clearly you do, so clearly IQ differences are not just inherent differences between individuals.

 

[Axulus]

Why would you need to believe that? Turn the proposition on it's head. If one really believed that IQ measure nothing but inherent differences between individuals, you'd expect to see no practice effects at all. Clearly you do, so clearly IQ differences are not just inherent differences between individuals.

The practice effects can be quantified. Do you really think statisticians use ranges and p-values just for the heck of it? The practice effect has been studied and quantified and can not explain all the difference between individuals. In fact, we know almost precisely how much of the difference it can explain.

 

[Togo]

Why would you need to believe that? Turn the proposition on it's head. If one really believed that IQ measure nothing but inherent differences between individuals, you'd expect to see no practice effects at all. Clearly you do, so clearly IQ differences are not just inherent differences between individuals.

The practice effects can be quantified.

Sure, so what? They still exist, which means that IQ tests do not just test inherent differences.

 

[Axulus]

The practice effects can be quantified.

Sure, so what? They still exist, which means that IQ tests do not just test inherent differences.

We can quantify a statistically significant range of inherent differences with an IQ test by controlling for the practice effects and other relevant variables, since the practice effects and other relevant variables have been studied and can be quantified.

 

[untermensche]

Sure, so what? They still exist, which means that IQ tests do not just test inherent differences.

We can quantify a statistically significant range of inherent differences with an IQ test by controlling for the practice effects and other relevant variables, since the practice effects and other relevant variables have been studied and can be quantified.

In any given test taker how do you measure how much of their score is related to the practice effect?

 

[Axulus]

We can quantify a statistically significant range of inherent differences with an IQ test by controlling for the practice effects and other relevant variables, since the practice effects and other relevant variables have been studied and can be quantified.

In any given test taker how do you measure how much of their score is related to the practice effect?

We can quantify it probabilisticly, such as saying that there is a 95% confidence that, in the absence of practice effects, this individual's IQ score falls between 94-106 (or whatever), given a result of 100, and given no further information about this individual.

 

[untermensche]

In any given test taker how do you measure how much of their score is related to the practice effect?

We can quantify it probabilisticly, such as saying that there is a 95% confidence that, in the absence of practice effects, this individual's IQ score falls between 94-106 (or whatever), given a result of 100, and given no further information about this individual.

The person's score either includes practice effects or it doesn't.

Looking at probabilities of whatever does not tell you if an individual score is the result of practice.

How can we tell if any individual score has within it effects from practice?

To answer we would have to know if the person taking the test practiced.

How do we know by looking at a score if the person with that score practiced?

 

[Axulus]

We can quantify it probabilisticly, such as saying that there is a 95% confidence that, in the absence of practice effects, this individual's IQ score falls between 94-106 (or whatever), given a result of 100, and given no further information about this individual.

The person's score either includes practice effects or it doesn't.

Looking at probabilities of whatever does not tell you if an individual score is the result of practice.

How can we tell if any individual score has within it effects from practice?

To answer we would have to know if the person taking the test practiced.

How do we know by looking at a score if the person with that score practiced?

I already told you. We can use statistical and baysein logic to say that practice effects (or lack thereof) have a 95% chance of affecting the IQ by +/- X given a specific result on a test for a particular individual.

We use bayseian reasoning can be used to quantify uncertainty. The more information we know about an individual, the tighter we can make the range or the more confident we can be given a fixed range. That's the best we can do and still provides us very useful information, yet it is not acceptable to you for some reason. Why is that?

 

[Metaphor]

That's quite a big difference,

And the difference between White and Black scores is three times that.

and that's only a from a single repeat. Spend a few days working with the tests and you'd expect a bigger increase.

No; in fact scores do not tend to improve with additional repeats beyond the first of the same test. You could memorise some of the answers to the comprehension questions but you aren't going to improve your digit span backwards score by much.

Probably because they're really, really, obvious if you actually work with the tests, or construct your own, and are entirely inconsistent with some of the claims made about such tests.

What's 'really, really obvious'? That psychometric tests are not perfect? You are correct. They are not perfect.

Why would you need to believe that? Turn the proposition on it's head. If one really believed that IQ measure nothing but inherent differences between individuals, you'd expect to see no practice effects at all. Clearly you do, so clearly IQ differences are not just inherent differences between individuals.

Whoever said they measure nothing but 'inherent' differences?

IQ is positively correlated with all kinds of life outcomes. If there is a difference in IQ between groups, we would automatically expect a difference in life outcomes due to that difference.

 

[Metaphor]

The person's score either includes practice effects or it doesn't.

Looking at probabilities of whatever does not tell you if an individual score is the result of practice.

No score is the 'result' of practice. Most of the variance in IQ scores has nothing to do with practice effects.

How can we tell if any individual score has within it effects from practice?

It doesn't matter. The part of the variance in IQ scores due to practice alone (and therefore unrelated to intelligence) will have the effect of reducing the correlation between IQ scores and life outcomes. But they don't reduce the correlation to zero.

It's manifestly absurd to dismiss IQ testing on the basis that there are quantifiable practice effects.

 

[Togo]

In any given test taker how do you measure how much of their score is related to the practice effect?

We can quantify it probabilisticly, such as saying that there is a 95% confidence that, in the absence of practice effects, this individual's IQ score falls between 94-106 (or whatever), given a result of 100, and given no further information about this individual.

I think you're in danger of misrepresenting the operation of p here, presumably because of the limits of posting on a message board. A confidence level of 95% doesn't mean we can ignore practice effects as being only 5% of the total. It means that we can be sure that such results would only arise from random chance, given the statistical assumptions of data and distribution and independent factors inherent in the test being used, 5% of the time.

In order to be confident that practice effects have been controlled for, we have to identify all such effects and understand their impact and the distribution of that impact, in order to correct for them statistically. That's quite a challenge. This is why, in practice, researchers don't tend to do this, and instead rely on presenting a test that is novel.

This is also why, in practice, researchers tend to try and use the test on matched groups where possible. So you try and match group A with all possible confounding factors as closely as possible to group B. But then you hit some problems, because clearly you can't match on a factor that you're actually wanting to test. You can't match on educational attainment, for example, if you want to make a point about correlation between IQ score and educational attainment. Similarly, you can't match on income if you want to compare the effects of IQ on income. IQ researchers have to be very careful in how they construct their field tests, because they face formidable challenges in constructing a meaningful comparison.

Then there is the mechanics of the test itself, IQ tests work by measuring conformity. You take the subject's answers, you compare them to the correct answers determined in advance, and work out the score based on how often one conforms to the other. Or as one critic rather unfairly put it, your score is based on how well your answer matches that of a psychology professor from New York. Now obviously steps are taken to control this. You use a multiple choice format so that personal expression is eliminated. You use questions where there is an unambiguously correct answer. You try and avoid questions of knowledge or questions that assume knowledge, and so on. Nonetheless, if you examine one of these tests carefully, you're going to find at least a few questions where you don't agree with the answer, or at least find the answer ambiguous. This effect is greater the further you are from the assumptions and way of thinking of the person setting the test. So again, IQ researchers have to take a great deal of care to balance out cultural differences so that they can be confident that they are getting a test that's broadly valid across different cultural groups.

But remember what we were discussing earlier? You can't control for what you're trying to measure. So when you look at data such as that presented in The Bell Curve, you're looking at tests that have deliberately not been balanced to produce equivalent results across different ethnic groups, and the book claims the differences that emerge as demonstrating something genetic and inherent about those groups. The problem that plagues studies that purport to show consistent differences between such groups is demonstrating that cultural factors have been eliminated, when the usual methods for doing so, correcting for differences between ethnic groups, are not available.

In short (too late), IQ testing is not a simple or straightforward business, and researchers need to be very, very, careful in interpreting the results. I'm not seeing that level of care in The Bell Curve, which is one reason why I suggest that they go beyond the science, speculating on possible sources for differences, rather than actually demonstrating through scientific study what those sources are.

 

[Metaphor]

Then there is the mechanics of the test itself, IQ tests work by measuring conformity. You take the subject's answers, you compare them to the correct answers determined in advance, and work out the score based on how often one conforms to the other. Or as one critic rather unfairly put it, your score is based on how well your answer matches that of a psychology professor from New York. Now obviously steps are taken to control this. You use a multiple choice format so that personal expression is eliminated. You use questions where there is an unambiguously correct answer. You try and avoid questions of knowledge or questions that assume knowledge, and so on. Nonetheless, if you examine one of these tests carefully, you're going to find at least a few questions where you don't agree with the answer, or at least find the answer ambiguous. This effect is greater the further you are from the assumptions and way of thinking of the person setting the test. So again, IQ researchers have to take a great deal of care to balance out cultural differences so that they can be confident that they are getting a test that's broadly valid across different cultural groups.

Well, the tests are valid across different cultural groups. Even if there were a whole range of other effects, a Black person with an IQ of 130 is going to be doing much better on life outcomes than a Black person with an IQ of 70.

The cultural chauvinism of any test, to the extent that it exists, tend to be grossly exaggerated by people wholly ignorant of psychometrics. Indeed, some clearly desperate people call IQ tests nothing but a score of conformity to White culture.

"Cultural conformity" may have limited purchase as a concept when talking about some questions in some of the verbal subscales of WAIS-IV. Precisely how matrix reasoning or letter-number sequencing, on the other hand, are either ambiguous or culture-dependent, I cannot fathom.

 

[Togo]

Then there is the mechanics of the test itself, IQ tests work by measuring conformity. You take the subject's answers, you compare them to the correct answers determined in advance, and work out the score based on how often one conforms to the other. Or as one critic rather unfairly put it, your score is based on how well your answer matches that of a psychology professor from New York. Now obviously steps are taken to control this. You use a multiple choice format so that personal expression is eliminated. You use questions where there is an unambiguously correct answer. You try and avoid questions of knowledge or questions that assume knowledge, and so on. Nonetheless, if you examine one of these tests carefully, you're going to find at least a few questions where you don't agree with the answer, or at least find the answer ambiguous. This effect is greater the further you are from the assumptions and way of thinking of the person setting the test. So again, IQ researchers have to take a great deal of care to balance out cultural differences so that they can be confident that they are getting a test that's broadly valid across different cultural groups.

Well, the tests are valid across different cultural groups. Even if there were a whole range of other effects, a Black person with an IQ of 130 is going to be doing much better on life outcomes than a Black person with an IQ of 70.

That's not a test of the validity of the concept.

The cultural chauvinism of any test, to the extent that it exists, tend to be grossly exaggerated by people wholly ignorant of psychometrics.

How is that relevant to this discussion?

"Cultural conformity" may have limited purchase as a concept when talking about some questions in some of the verbal subscales of WAIS-IV. Precisely how matrix reasoning or letter-number sequencing, on the other hand, are either ambiguous or culture-dependent, I cannot fathom.

It isn't obvious unless you've familiar with such experiments.

Take two US students. Put one in a quiet air-conditioned hall with a No.2 pencil. Put the other on the floor of a grass hut, in the rain, with a goat, and a reed quill, and a paper in a language he's fluent, but not expert in. Do you think there will be a slant in results in favour of one or the other? If so, why?

 

[untermensche]

The person's score either includes practice effects or it doesn't.

Looking at probabilities of whatever does not tell you if an individual score is the result of practice.

How can we tell if any individual score has within it effects from practice?

To answer we would have to know if the person taking the test practiced.

How do we know by looking at a score if the person with that score practiced?

I already told you. We can use statistical and baysein logic to say that practice effects (or lack thereof) have a 95% chance of affecting the IQ by +/- X given a specific result on a test for a particular individual.

We use bayseian reasoning can be used to quantify uncertainty. The more information we know about an individual, the tighter we can make the range or the more confident we can be given a fixed range. That's the best we can do and still provides us very useful information, yet it is not acceptable to you for some reason. Why is that?

Throwing around statistical terms doesn't answer the question.

There is no statistic that tells you if an individual score has been effected by practice. If you think there is, he is an example.

Joe X had a score of 112 on an IQ test. Using any method you like please tell me (with a 95% confidence level) how much this score was effected by practice.

 

[Loren Pechtel]

The average height in Japan has risen in the last 60 years.

The genes are the same.

Genes are only part of the story. But how do we know when genes for height are being fully realized?

Japan?? I've seen this happen in China *MUCH* faster.

When I first went to China I could see over the heads of basically everyone. In less than a decade that was no longer true, while I'm still about the tallest around there are plenty of young adults whose hair is above my eye level. This can be traced back to those who grew up after China switched from the socialist path to the capitalist one. People were eating better, they grew taller.

My contention and it may be the contention of others is that many black Americans don't live in the same environment as many white American's.

Their vision of the society is different. And therefore their reactions to institutions, like school, will be different.

IQ in a way tests your feelings towards your education. It tests how motivated you were to learn in the past. It tests what things you have been motivated to think about.

These things can easily change.

In other words, many don't develop the intelligence they are born with. I definitely agree--I have seen no indication that there is any genetic component to intelligence. The issue is cultural.