• Welcome to the new Internet Infidels Discussion Board, formerly Talk Freethought.

The Reformation: Can Social Scientists Save Themselves?

NobleSavage said:
This is a really good read:

http://www.psmag.com/navigation/hea...entists-save-themselves-human-behavior-78858/
Click to expand...

From what I read it extends to all science. Scientific progress depends on peers volunteering (mostly), repeating, testing, material produced. If almost no one is interested in an area of endeavor, for instance, stuff will stand until one tests, fails to validate, or questions results generated. Social science has other risks too. There's buck in promoting viewpoints on everything from weight issues to sex issues. So there's a lot of temptation for the unqualified, the unethical, the self promoting, to pump something out and reap bucks because so many are eager for anything that will spark success.

Still, there's little need to react to such as that beatles thingie since its so meaningless and easily countered. What is the real challenge is stuff that is faked in an area that's hot that hard to debunk for whatever reason. Good ethics and good recruitment of the guardians, the editors, reviewers, the interested scientists, should do the job.
 
Paragraph 11:
Unlike Sokal’s attack, the current critique of experimental social science is coming mainly from the inside.
Click to expand...

Self correction right there. Therefore, to answer the title of the thread, "Can Social Scientists Save Themselves?", they already are. Salvation is not to salvage your status quo (to save your drinking habit) but to come to terms with your situation and turn around (to come around sober, fighting your previous habits).

Corruption exists because of positive and negative reinforcement, the temptation of easy ways out and reachable prizes of fame and fortune; because we're not theological entities, but animals. We know this thanks to psychological experimentation (B.F.Skinner, anyone?). Science works, even among imperfect, bumbling humans.
 
from the article said:
Simmons and his colleagues duly reported their adjustment in their mock paper—but they left out any mention of all the other factors that they had tried and discarded. This was all, Simmons emphasizes, within the bounds of what is considered fair play in most psychology departments and journals.
Click to expand...

The article makes some good points, but the above part is bullshit. It is not at all considered "fair play" within psychology to conduct multiple analysis and report only some without making a correction to the criterion p-value used to determine "significance". It is considered fraud and this is highly emphasized in the multiple statistics courses that all psychology grad students take. In fact, psych grad students receive more training in this and related stats issues than scientists in most hard sciences. Journals regularly require researchers to make alpha corrections to results to account for the inflated Type I errors that occur with multiple comparisons.
Do some researchers fail to make these corrections, sure, as they do in other sciences.

The only reason it is a bigger issue in psychology is that psychology inherently has to deal with smaller effect sizes because every human behavior is so complexly determined by countless interacting factors that experimentally manipulating a single factor with the kind of weak manipulations you can use in a lab in 60 minutes is likely to lead to only small effects, even if the variable is very much a causal factor in the behavior and has important real world impact.
For example, even if TV violence increases aggression and really matters when dealing with thousands of hours of exposure over decades, a 30 minute exposure in a lab will have very small impact.
P-Values are a function of the size of the effect and the size of the sample. Psychology experiments have a pragmatic limit on sample size because they are sampling humans and not chemicals or bacteria. This means that even when causal effects are very much real, they will often be just around the p = .05 level. If you run 30 people in each condition and the effect is at the .07 level, then you run some more people. IF the effect was fake, this will usually make the p value go up, but if the effect is real it will go down. IF you have a p level below .05, you don't run more people because that takes a lot of time and money. This is why there are a cluster of reported p values just below the .05 level. That isn't an indicator of fraud, it is just what you would expect if people are studying small effects and can only afford to have the size of the sample they need to get small effects below a p value of .05.

One thing that will help is doing away entirely with p-values and null hypothesis testing. Bayesian approaches present an alternative to these things and since Bayesian models are also used as idealized forms of psychological reasoning, it is mostly Psychologists who trying to lead the way in pushing for the use of Bayesian analyses to test hypothesis and choose between competing theories. Not only will get around the failure to correct p-values for the number of analyses done, but it will solve the file drawer problem.

BTW, on the issue of fraud, studies have shown that Medicine is the field with the most research fraud.
 
doubtingt said:
from the article said:
Simmons and his colleagues duly reported their adjustment in their mock paper—but they left out any mention of all the other factors that they had tried and discarded. This was all, Simmons emphasizes, within the bounds of what is considered fair play in most psychology departments and journals.
Click to expand...

The article makes some good points, but the above part is bullshit. It is not at all considered "fair play" within psychology to conduct multiple analysis and report only some without making a correction to the criterion p-value used to determine "significance". It is considered fraud and this is highly emphasized in the multiple statistics courses that all psychology grad students take. In fact, psych grad students receive more training in this and related stats issues than scientists in most hard sciences. Journals regularly require researchers to make alpha corrections to results to account for the inflated Type I errors that occur with multiple comparisons.
Click to expand...

Certainly wouldn't have gotten away with it at my old lab. We also had a special course on the right and wrong way to interpret multivariate correlations, to get around the issue of slicing a set of data until you find something significant.

doubtingt said:
The only reason it is a bigger issue in psychology is that psychology inherently has to deal with smaller effect sizes because every human behavior is so complexly determined by countless interacting factors that experimentally manipulating a single factor with the kind of weak manipulations you can use in a lab in 60 minutes is likely to lead to only small effects, even if the variable is very much a causal factor in the behavior and has important real world impact.
Click to expand...

Or to put it another way, you can't just take the same person and run hundreds of iterations of tests on him, like you could on a physical science set-up.
 
You must log in or register to reply here.
Back
Top Bottom