Standard Tests And Bias

[Loren Pechtel]

HS GPA is a better predictor than the SAT is the result from the study. Nothing you or LP or Trausti or Auxulus rebutted that fact. No one said anything about using only HS GPA as the criterion.

You're still ignoring the fact they took into consideration the school the GPA came from.

Also, since it's being proposed to get rid of the SAT/ACT it does amount to using only the HS GPA as the criterion as everything else being used is basically BS.

I don't believe any of us are saying that GPA has no predictive value.

 

[laughing dog]

HS GPA is a better predictor than the SAT is the result from the study. Nothing you or LP or Trausti or Auxulus rebutted that fact. No one said anything about using only HS GPA as the criterion.

You're still ignoring the fact they took into consideration the school the GPA came from.

I am not ignoring it.

Also, since it's being proposed to get rid of the SAT/ACT it does amount to using only the HS GPA as the criterion as everything else being used is basically BS.

Only if HS GPA is used. To my knowledge, no institution is relying only on HS GPA.

 

[Trausti]

I am not ignoring it.

Also, since it's being proposed to get rid of the SAT/ACT it does amount to using only the HS GPA as the criterion as everything else being used is basically BS.

Only if HS GPA is used. To my knowledge, no institution is relying only on HS GPA.

Standardized tests are a proxy for IQ. So it’d not be surprising that those with high SAT/ACT scores also had high GPA. But GPA is a less reliable indicator as it’s dependent on some subjective teacher grading and relation to high school peers.

 

[Loren Pechtel]

I am not ignoring it.

Also, since it's being proposed to get rid of the SAT/ACT it does amount to using only the HS GPA as the criterion as everything else being used is basically BS.

Only if HS GPA is used. To my knowledge, no institution is relying only on HS GPA.

Standardized tests are a proxy for IQ. So it’d not be surprising that those with high SAT/ACT scores also had high GPA. But GPA is a less reliable indicator as it’s dependent on some subjective teacher grading and relation to high school peers.

What the study found is that amongst those with adequate ACT scores GPA was more predictive of success. Note that that means the students with a high GPA from a shitty school were basically excluded and thus no conclusions can be drawn about how useful it is without first filtering with with SAT/ACT.

 

[laughing dog]

Standardized tests are a proxy for IQ. So it’d not be surprising that those with high SAT/ACT scores also had high GPA. But GPA is a less reliable indicator as it’s dependent on some subjective teacher grading and relation to high school peers.

What the study found is that amongst those with adequate ACT scores GPA was more predictive of success. Note that that means the students with a high GPA from a shitty school were basically excluded and thus no conclusions can be drawn about how useful it is without first filtering with with SAT/ACT.

Since you not read the report, you don't know what the study basically found. The study's goal was to look at which was a better predictor of college success - HS GPA or SAT/ACT. Naturally, it only included students who went to college. Any student who do not go to college should not have included. Neither you nor the authors had the information on why a student did not go to college. Your focus on high GPAs from shitty schools who did not go to college makes the assumption they applied but were not admitted to college because of their SAT/ACT score. Now, I wonder why you keep repeating in illogical point over and over when you do not mention high GPAs from good schools who did not go to college or any other HS GPA range from any type of school who did not go to college.

 

[Lumpenproletariat]

Without testing there is no learning or teaching.

And if it cannot be tested, then nothing was learned or taught. If you know someone learned it, it's because you tested them somehow.

The tests themselves should be tested, or all the test items should be scrutinized to ensure that they perform their (culturally neutral) function of objectively measuring each student's mastery of the subject matter.

the problem is that this is impossible - the only thing a test can ever measure is your mastery of taking the test, but it's almost completely unrelated to mastery of the subject that the test is about.

No, mastery of the test is about the same as mastery of the subject matter. Knowing how to answer the test questions is the same as knowing the subject matter. There's no difference between knowing something and knowing how to answer questions about it. You can't claim to know something if you can't answer questions about it. The better you know it, the better you can answer questions about it.

You can't give an example of someone doing well on the tests but not knowing the subject matter, unless it's a bad test which could be made better. Also you can't give an example of knowing the subject matter but not being able to answer questions about it. To have learned it, one must have more than only a degree or an accumulation of classroom hours. Whereas ability to answer questions about it is demonstration of one's knowledge of the subject.

the problem is that people are stupid enough to think that one's ability to master a test is the same thing as one's ability to master the thing the test is about, . . .

They are the same, as long as the test items are really about the subject matter and not something artificial or superficial to the subject matter. The tests can be improved so as to focus better on the intended subject matter, so they can perform better at measuring the student's knowledge. So the flaws of the test can be reduced, and we can rate the test, or test the test itself, to improve it.

. . . and that has led to a huge number of problems in US culture when it comes to notions of intelligence and career preparedness.

What "problems"? You can't name them without tests you're using in order to make that judgment.

If you really believe those problems exist and want to correct them, then you want to improve the tests to make them perform better at measuring the knowledge of the students. But without improving the tests, you are proliferating the problems caused by bad tests. Also increasing the good tests helps correct those problems. You have no way to correct those problems without more and better tests.

the population needs to understand that testing only tests one's ability to test, and if that's the data point we're going to use to judge people fine, but stop pretending like taking a test is analogous to understanding the material.

But "understanding" the subject matter is the same as doing well on tests, or being able to, or able to answer questions about it, or to perform what the subject matter is about. The phrase "understanding the material" is meaningless if it doesn't mean being able to perform well on a test of one's knowledge of the subject matter.

You can't give an example of "mastery of the subject" or "understanding" it or "ability to master" it which is separate from one's ability to answer questions about it or perform well on the test about it.

If you dismiss the testing as unimportant, then you also dismiss the subject matter as unimportant. You can't claim the subject matter is important, or any learning of it, or teaching it, if the testing of it is not important. If there's nothing there needing to be tested, then there's no subject matter needing to be learned, and nothing to be educated about, also no "problems in US culture" needing to be fixed.

Either we improve the tests, and test everything that matters, or -- nothing matters and nothing needs to be learned and no "problems" exist and no "education" is needed.

To say tests don't matter is to say truth and knowledge doesn't matter and that nothing is worth learning because nothing matters.

 

[Elixir]

No, mastery of the test is about the same as mastery of the subject matter. Knowing how to answer the test questions is the same as knowing the subject matter.

I vehemently disagree.
In fact, I'm a veritable poster child for doing well on tests in subjects about which I know little or nothing.
To give a very oversimplified example, take a multiple choice math question supposedly testing a student's ability to perform long division:

4671/27 = x

x=
a) 213
b) 169
c) 417
d) 173

It would take me about 3 seconds to know that the answer was "d" even though I sucked at long division.
I'd immediately see it had to be a or d because the last digit is 1, which can only be the product of 7x3. So the answer has to end in 3.
Since 2x4=8, "a" is too big a number, so it has to be "d".

That is a GROSS oversimplification. But believe me - many test answers can be divined by means other than by knowing the subject matter.

 

[Lumpenproletariat]

No, mastery of the test is about the same as mastery of the subject matter. Knowing how to answer the test questions is the same as knowing the subject matter.

I vehemently disagree.
In fact, I'm a veritable poster child for doing well on tests in subjects about which I know little or nothing.
To give a very oversimplified example, take a multiple choice math question supposedly testing a student's ability to perform long division:

4671/27 = x

x=
a) 213
b) 169
c) 417
d) 173

It would take me about 3 seconds to know that the answer was "d" even though I sucked at long division.
I'd immediately see it had to be a or d because the last digit is 1, which can only be the product of 7x3. So the answer has to end in 3.
Since 2x4=8, "a" is too big a number, so it has to be "d".

That is a GROSS oversimplification. But believe me - many test answers can be divined by means other than by knowing the subject matter.

You obviously used your knowledge of the subject matter to arrive at the answer in this case.

You have to give a better example than this, or you're only proving my point.

Of course you can also be suggesting an IMPROVEMENT in the testing, which would be to have the student simply provide the correct answer, rather than choosing from among 4 multiple choice answers. But that also makes my point, that the solution is to IMPROVE the tests, not de-emphasize testing.

(Another improvement would be to have ALL multiple choice answers end in 3, also increase the number of choices.)

So you do not disagree vehemently, but are emphatically proving my point with this example.

 

[ZiprHead]

No, mastery of the test is about the same as mastery of the subject matter. Knowing how to answer the test questions is the same as knowing the subject matter.

I vehemently disagree.
In fact, I'm a veritable poster child for doing well on tests in subjects about which I know little or nothing.
To give a very oversimplified example, take a multiple choice math question supposedly testing a student's ability to perform long division:

4671/27 = x

x=
a) 213
b) 169
c) 417
d) 173

It would take me about 3 seconds to know that the answer was "d" even though I sucked at long division.
I'd immediately see it had to be a or d because the last digit is 1, which can only be the product of 7x3. So the answer has to end in 3.
Since 2x4=8, "a" is too big a number, so it has to be "d".

That is a GROSS oversimplification. But believe me - many test answers can be divined by means other than by knowing the subject matter.

You obviously used your knowledge of the subject matter to arrive at the answer in this case.

You have to give a better example than this, or you're only proving my point.

I haven't had to do long division in forty years. I have absolutely no idea how to do it today, calculators and e-spreadsheets have replaced my skills. Yet I could still figure out the answer to the question. The question would have no value in determining my skill at long division.

 

[Lumpenproletariat]

It would take me about 3 seconds to know that the answer was "d" even though I sucked at long division.
I'd immediately see it had to be a or d because the last digit is 1, which can only be the product of 7x3. So the answer has to end in 3.
Since 2x4=8, "a" is too big a number, so it has to be "d".

That is a GROSS oversimplification. But believe me - many test answers can be divined by means other than by knowing the subject matter.

You obviously used your knowledge of the subject matter to arrive at the answer in this case.

You have to give a better example than this, or you're only proving my point.

I haven't had to do long division in forty years. I have absolutely no idea how to do it today, calculators and e-spreadsheets have replaced my skills. Yet I could still figure out the answer to the question.

That proves you have more skill than others who could not answer it, which is the point of the testing.

The question would have no value in determining my skill at long division.

It would separate you from others who could not answer it correctly. It would determine your level of math ability compared to them. There's something dishonest, and phony, about insisting that this ability has no value and is not worth testing.

If this ability has no value in society, for any purpose, then what knowledge does have any value? Why should anything be learned or taught? Why should there be any education at all, if the ability to do math problems has no value?

This is not an argument against testing, but against any education or learning.

 

[ZiprHead]

I haven't had to do long division in forty years. I have absolutely no idea how to do it today, calculators and e-spreadsheets have replaced my skills. Yet I could still figure out the answer to the question.

That proves you have more skill than others who could not answer it, which is the point of the testing.

The question would have no value in determining my skill at long division.

It would separate you from others who could not answer it correctly. It would determine your level of math ability compared to them. There's something dishonest, and phony, about insisting that this ability has no value and is not worth testing.

If this ability has no value in society, for any purpose, than what knowledge does have any value? Why should anything be learned or taught? Why should there be any education at all, if the ability to do math problems has no value?

You can only say that by misrepresenting the point of the test.

 

[laughing dog]

Learning occurs with or without tests. Verification of learning requires some observation or evidence of the learning.

 

[Lumpenproletariat]

That proves you have more skill than others who could not answer it, which is the point of the testing.

The question would have no value in determining my skill at long division.

It would separate you from others who could not answer it correctly. It would determine your level of math ability compared to them. There's something dishonest, and phony, about insisting that this ability has no value and is not worth testing.

If this ability has no value in society, for any purpose, then what knowledge does have any value? Why should anything be learned or taught? Why should there be any education at all, if the ability to do math problems has no value?

You can only say that by misrepresenting the point of the test.

I.e., you have a conspiracy theory about the sinister motives of someone who wants testing. If you don't want to tell us this theory, then we can only assume it's a case of paranoia on your part. Testing of students has been going on for centuries. It's not true that this has been something sinister.

Everyone knows there are flaws in testing. Whatever the flaws, even if someone has hidden secret sinister motives, no response makes any sense other than to improve the tests and increase them, to reduce the flaws.

 

[ZiprHead]

You can only say that by misrepresenting the point of the test.

I.e., you have a conspiracy theory about the sinister motives of someone who wants testing. If you don't want to tell us this theory, then we can only assume it's a case of paranoia on your part. Testing of students has been going on for centuries. It's not true that this has been something sinister.

 

[Jarhyn]

So, I'm good at testing. Like, stupid good at it. Give me a multiple choice test with as many as six answers, and I will assure you I will get less than 1/2 of the questions wrong even if I don't actually know the answers.

If the test is for a single chapter on a subject, this number will be down as low as 1/3.

This is because most tests ask questions in a particular way: "which answer is the result of X+Y?", Wherein some basic knowledge will throw out two answers right away, one is going to be close "but probably wrong", and the remaining two allow for a test of one to true the other.

Of course if I really can't figure it out, there are other methodologies: in math tests, there is generally another test question most likely that invokes the principles for solution. Once I find it, I just go back and apply the principles that question assumes. Like "ax2 + bx + c has how many real roots?" And then later "dx2+ex+f has a real root 3, what is it's remaining root?" And one is imaginary and the other answers don't check.

I could do this without knowing the quadratic equation, or anything else actually math related other than how to "guess and check"

All that test does is test how good I am at fooling tests.

I don't really think that "knowing the signs of a quadratic form imply the number of solutions that may be found" is really what the teacher wants to ensure I know. They want to know I can operate The Quadratic Equation and that I understand numbers with an imaginary part.

 

[Lumpenproletariat]

So, I'm good at testing. Like, stupid good at it. Give me a multiple choice test with as many as six answers, and I will assure you I will get less than 1/2 of the questions wrong even if I don't actually know the answers.

If the test is for a single chapter on a subject, this number will be down as low as 1/3.

This is because most tests ask questions in a particular way: "which answer is the result of X+Y?", Wherein some basic knowledge will throw out two answers right away, one is going to be close "but probably wrong", and the remaining two allow for a test of one to true the other.

Of course if I really can't figure it out, there are other methodologies: in math tests, there is generally another test question most likely that invokes the principles for solution. Once I find it, I just go back and apply the principles that question assumes. Like "ax2 + bx + c has how many real roots?" And then later "dx2+ex+f has a real root 3, what is it's remaining root?" And one is imaginary and the other answers don't check.

I could do this without knowing the quadratic equation, or anything else actually math related other than how to "guess and check"

All that test does is test how good I am at fooling tests.

I don't really think that "knowing the signs of a quadratic form imply the number of solutions that may be found" is really what the teacher wants to ensure I know. They want to know I can operate The Quadratic Equation and that I understand numbers with an imaginary part.

All this just confirms further that what we need are improvements in the tests and increased use of good testing. And there's plenty of good testing already, in addition to some testing which is flawed.

We need more test-inspectors and test-improvement workers. And fewer of Trump's steel workers and auto workers and factory workers doing artificial "jobs" they're not needed for and being paid more than they're worth. There are even homeless people who could do needed test-improvement work which is not being done because we waste so much resources artificially putting people into factories and other "jobs" we don't need them for.

No one can name a case where the students' ability to perform well on tests or give correct answers is not a good demonstration of their knowledge of the subject matter (except some occasional examples where some flawed items on the test could be improved).

No case can be named where tests need to be reduced or eliminated. All the facts show that
more and better testing is the solution to whatever is failing in education and learning and advancement of knowledge.

 

[Metaphor]

No, mastery of the test is about the same as mastery of the subject matter. Knowing how to answer the test questions is the same as knowing the subject matter.

I vehemently disagree.
In fact, I'm a veritable poster child for doing well on tests in subjects about which I know little or nothing.
To give a very oversimplified example, take a multiple choice math question supposedly testing a student's ability to perform long division:

4671/27 = x

x=
a) 213
b) 169
c) 417
d) 173

It would take me about 3 seconds to know that the answer was "d" even though I sucked at long division.
I'd immediately see it had to be a or d because the last digit is 1, which can only be the product of 7x3. So the answer has to end in 3.
Since 2x4=8, "a" is too big a number, so it has to be "d".

That is a GROSS oversimplification. But believe me - many test answers can be divined by means other than by knowing the subject matter.

So, your knowledge of division caused you to immediately get the correct answer in a division question.

Shocking.

 

[Lumpenproletariat]

the solutions:

There should be more testing, not less.

The tests themselves should be tested, or all the test items should be scrutinized to ensure that they perform their (culturally neutral) function of objectively measuring each student's mastery of the subject matter. And those creating the tests have to be corrected for their bad decisions in shaping the test items.

Education opportunities ideally should be open to all seeking to enter, with no limits to the number of students admitted, so there's no need to exclude anyone, by testing or any other barrier. And all the students in a program should be tested and advanced further according to their performance on the tests, with no sympathy to anyone performing poorly, other than to drive them to improve their performance on future tests.

Improving the integrity of the tests is the correct solution to anything wrong in the testing system, while reducing the testing is always the wrong solution.

Which is why, when fattening hogs, the farmers don't waste too much precious time feeding them, they just weigh them as often and as accurately as possible.

translation: they feed them in addition to testing them.

But how do they know to feed them? or what to feed them? It was learned, through testing (over thousands of years), that livestock is made better by proper feeding. Without testing they would never have known this. Without the final product, which must be tested and testable, it's impossible to know how to better produce it.

Good test results are not (or at least, should not be) the objective of education.

They are inseparable from the objective. The objective and the good test results are virtually the same. When they are not the same, it's only because the tests need to be improved. And in some cases a good test can be difficult, but the only solution to that is to keep trying to improve the tests, never to reduce the tests.

The tests have to try to duplicate the real life situations the educated one faces later, for which s/he was educated. When these are different than the learning environment, the testing process is made more difficult. Sometimes "on-the-job training" is the best way to learn, so that the testing and actual application of one's education mix together. In which case the "testing" is still essential. There's never a case where testing is separate from the objective of the learning.

The objective is for people to know stuff and/or have skills at the end of the process, that they didn't have at the beginning.

But that has to mean that they can be tested and do well on the tests. If they cannot, then the objective -- for people to know stuff etc. -- is not achieved.

Tests can give, at best, a very crude idea of the knowledge and/or skills that a person has at the end of the process, . . .

How do you know that? You can't claim this without applying your own test to verify that the person has the knowledge/skills, so you can judge the other test as giving only a "crude" idea of what was learned. If you're able to judge someone's knowledge enough that you can judge the test of it as being "crude," then you must have used your own test which is even better and is not "crude," which means you know how the test could be improved.

Tests can give, at best, a very crude idea of the knowledge and/or skills . . . but tell you nothing about whether the education was responsible for imparting those skills or knowledge.

If that's true, then you're saying the education has no useful function and should be done away with, because there's no reason to believe it does any good, which it does not if testing it tells us "nothing about whether the education was responsible for imparting those skills or knowledge." Why have any education if there's no reason to believe it makes any difference? How do you know it makes any difference without testing it?

And testing is rarely at its best - typically they measure . . .

"nothing is perfect" -- but usually it can be improved. The solution always is to improve the testing, not reduce it because it's not perfect or "at its best."

. . . typically they measure a tiny subset of the subject matter, and assume that a student who knows that stuff on test day will know the entire subject equally well in a year's time.

solution: test more than just a tiny subset. And test the student more than only once a year -- like maybe once a week. Always more and better testing is the answer.

Neither assumption (scope nor duration) is likely to be true.

The assumption is true that a little testing is better than no testing at all. And still more testing is better than less.

So here again we have further corroboration that more and better testing is always the answer. No one yet has made any case or given any example to show otherwise.

 

[Elixir]

No, mastery of the test is about the same as mastery of the subject matter. Knowing how to answer the test questions is the same as knowing the subject matter.

I vehemently disagree.
In fact, I'm a veritable poster child for doing well on tests in subjects about which I know little or nothing.
To give a very oversimplified example, take a multiple choice math question supposedly testing a student's ability to perform long division:

4671/27 = x

x=
a) 213
b) 169
c) 417
d) 173

It would take me about 3 seconds to know that the answer was "d" even though I sucked at long division.
I'd immediately see it had to be a or d because the last digit is 1, which can only be the product of 7x3. So the answer has to end in 3.
Since 2x4=8, "a" is too big a number, so it has to be "d".

That is a GROSS oversimplification. But believe me - many test answers can be divined by means other than by knowing the subject matter.

So, your knowledge of division caused you to immediately get the correct answer in a division question.

Shocking.

But it was supposed to test ability to do long division, not to pick the right answer (using multiplication and elimination). Point proven.
Jarhyn explained it more elaborately (better):

So, I'm good at testing. Like, stupid good at it. Give me a multiple choice test with as many as six answers, and I will assure you I will get less than 1/2 of the questions wrong even if I don't actually know the answers.

If the test is for a single chapter on a subject, this number will be down as low as 1/3.

This is because most tests ask questions in a particular way: "which answer is the result of X+Y?", Wherein some basic knowledge will throw out two answers right away, one is going to be close "but probably wrong", and the remaining two allow for a test of one to true the other.

Of course if I really can't figure it out, there are other methodologies: in math tests, there is generally another test question most likely that invokes the principles for solution. Once I find it, I just go back and apply the principles that question assumes. Like "ax2 + bx + c has how many real roots?" And then later "dx2+ex+f has a real root 3, what is it's remaining root?" And one is imaginary and the other answers don't check.

I could do this without knowing the quadratic equation, or anything else actually math related other than how to "guess and check"

All that test does is test how good I am at fooling tests.

I don't really think that "knowing the signs of a quadratic form imply the number of solutions that may be found" is really what the teacher wants to ensure I know. They want to know I can operate The Quadratic Equation and that I understand numbers with an imaginary part.

I even recall geometry test that asked for the distance from x to y (no multiple choice) involving a curve and some planar coordinates... at sea with applying the formula for the parabola, I drew it roughly and measured the distance, which was really close to a rational number of units so I used that number. It was correct, and took a fraction of the time that calculating it would have taken, even if I knew what I was doing.
Test-taking is a skill (or call it a talent) that can have a lot - or almost nothing - to do with the actual subject matter that the test designer had in mind.

 

[Lumpenproletariat]

Education opportunities ideally should be open to all seeking to enter, with no limits to the number of students admitted, so there's no need to exclude anyone, by testing or any other barrier. And all the students in a program should be tested and advanced further according to their performance on the tests, with no sympathy to anyone performing poorly, other than to drive them to improve their performance on future tests.

Only if educational opportunities exist at all levels.

They should exist at all levels -- ideally. If it's not yet totally practical, then extend the opportunities as much as possible -- to more levels.

It should be possible for anyone to enroll into a rocket science program. Maybe not today, but eventually. The same program could be open to a 5-year-old (prodigy?), and also to an 80-year-old geezer. They would be tested and routed to wherever their appropriate learning level is, regardless of all the other differences. And this should be open to EVERYONE to apply and be accepted, and then be routed to where they can best learn at their level.

The narrower the range of abilities of the students in a class the better off the students are.

OK, but there should be a rocket-science "class" to everyone, based on their aptitude. So, no one should be excluded from the program. Certain ones who are poor performers at first might improve, based on the testing. So any of them could advance if they show improvement, based on the testing.

Also, there's a place here for some of the more advanced students to participate in teaching the less advanced ones, so there needn't be total segregation of the more able from the less able.