Tests of Astrology

[lpetrich]

Astrology is the claim that the celestial bodies control our destinies and other earthly events to much greater degree and much finer granularity than is recognized in modern science. Not surprisingly, astrology and astronomy were not distinguished very clearly until recent centuries.

However, astrology has a problem with making predictions, problems that astrologers themselves recognize. They like to say "astra inclinant, non necessitant" or "astra inclinant, sed non necessitant" or "astra inclinant, sed non obligant" -- Latin for "the stars incline, but they do not compel".

But there is a major field of applied mathematics for testing for the presence of influences that incline without compelling. Statistics. So do any astrological predictions pass statistical tests?

I must first rebut a common criticism of astrology: lack of known mechanism. While we have learned a lot, there are still many things that we poorly understand or not understand at all. So there could be some unknown mechanism that causes astrological effects. Many things that are now well-understood started off in that kind of state, but they were first accepted on empirical grounds, from passing observational and experimental tests.

So can any astrological predictions pass empirical tests?

 

[Shadowy Man]

So can any astrological predictions pass empirical tests?

there’s no reason to believe they can.

 

[lpetrich]

Paranormal Borderlands of Science - Google Books - "Best of Skeptical Inquirer" - Prometheus Books

One of its chapters is An Empirical Test of Popular Astrology by Ralph W. Bastedo in Fall 1978

RWB investigated Sun signs, those divisions of the year from around the 21st of each month to around the 21st of the next month. Why the 21st? That's when the equinoxes and solstices are.

He then considered leadership ability, political position (left-right), subjectively-assessed intelligence, IQ, and belief in astrology, testing for correlations with Sun signs.

• Leadership: n = 33, X = 38.012, P = 0.25
• Politics: n = 88, X = 84.274, P = 0.59
• Intellgence: n = 33, X = 35.371, P = 0.36
  [*}IQ: n = 44, X = 48.199, P = 0.31
• Belief in astrology: n = 33, X = 33.043, P = 0.47

n = number of degrees of freedom, X is the chi-squared value, and P is the probability of a random distribution producing a X value greater than the observed value.

So there is zero correlation between Sun sign and these features.

 

[lpetrich]

Double-Blind Test of Astrology by Shawn Carlson in 1983, submitted to Nature magazine

Two double blind tests were made of the thesis that astrological "natal charts" could be used to describe accurately personality traits of test subjects. In the first test, we attempted to determine whether a person could recognize his own personality when described by an astrologer through a "natal chart interpretation". In the second test, we attempted to determine whether astrologers could correctly match a person's natal chart to the results of a well known and scientifically accepted personality test (the California Personality Inventory or CPI). Care was taken to make sure that the procedures satisified both scientists and astrologers. Subjects' recognition of their own natal chart interpretations was poor, but we draw no conclusion from this first test because the same subjects failed to recognize their own CPI profiles as well. The abilities of astrologers to match natal charts to CPIs was not significantly different from that predicted by the "scientific" bypothesis (i.e. their choices were no better than random), a result which strongly refutes the astrological thesis.

It got this response:
Appraisal of Shawn Carlson’s Renowned Astrology Tests by Suthbert Ertel, Journal of Scientific Exploration, Vol. 23, No. 2, pp. 125–137, 2009

Shawn Carlson’s 1985 study, published in Nature, which ended with a devastating verdict of astrology, is scrutinized. The design of Carlson’s study violated the demands of fairness and its mode of analysis ignored common norms of statistics. The study’s piecemeal analysis of sub-samples avoided testing the totals for astrological effects, as did the neglect of test power, effect size, and sample size. Nevertheless, a correct reanalysis of Carlson’s two astrological tests reveals that astrologers matched profiles of the California Personality Inventory to natal charts better than expected by chance with marginal significance (three-way forced choice, p = .054), and that a positive result was replicable by a different assessment method (10-point rating, p = .04). The results are regarded as insufficient to deem astrology as empirically verified, but they are sufficient to regard Carlson’s negative verdict on astrology as untenable

 

[Elixir]

So can any astrological predictions pass empirical tests?

Are you a Libra?

 

[Cheerful Charlie]

Astrology is the claim that the celestial bodies control our destinies and other earthly events to much greater degree and much finer granularity than is recognized in modern science. Not surprisingly, astrology and astronomy were not distinguished very clearly until recent centuries.

However, astrology has a problem with making predictions, problems that astrologers themselves recognize. They like to say "astra inclinant, non necessitant" or "astra inclinant, sed non necessitant" or "astra inclinant, sed non obligant" -- Latin for "the stars incline, but they do not compel".

But there is a major field of applied mathematics for testing for the presence of influences that incline without compelling. Statistics. So do any astrological predictions pass statistical tests?

I must first rebut a common criticism of astrology: lack of known mechanism. While we have learned a lot, there are still many things that we poorly understand or not understand at all. So there could be some unknown mechanism that causes astrological effects. Many things that are now well-understood started off in that kind of state, but they were first accepted on empirical grounds, from passing observational and experimental tests.

So can any astrological predictions pass empirical tests?

Many years ago, a French researcher had claimed he could prove astrology worked Michael Gauquelin claimed that it could be demonstrated great sports champions prominently showed effects of Mars in their charts. This set off an investigation of the claims by The Humanist Magazine. And that lead to the founding of the Committee For Scientific Investigation Of Claims Of The Paranormal. CSICOP. Pursuing Gauquelin's claims, these claims soon collapsed on careful investigation. For example Gauquelin claimed there were large numbers of great Italian Aviators. But would not accept examination of charts of Heisenberg trophy Winners. In the end, these Mars Effect claims collapsed totally. The effect relied on sloppy definitions of Great Sports Champions, and careful cherry picking of said Great Sports Champions. along the way, other observations were noted. The ephermises of astronomers were plagiarized by astrologers for preparing charts, But for some years, these astrologer's books contained numerous errors but no astrologer ever noticed, that made no difference to their claims. Astrology was thoroughly tested for some years by CSICOP.the astrologers couldn;t show any evidence astrology did anything. Google astrology Mars Effect for more.

 

[bilby]

I must first rebut a common criticism of astrology: lack of known mechanism. While we have learned a lot, there are still many things that we poorly understand or not understand at all. So there could be some unknown mechanism that causes astrological effects. Many things that are now well-understood started off in that kind of state, but they were first accepted on empirical grounds, from passing observational and experimental tests.

This is weak sauce.

There cannot be any unknown mechanism, unless we are wildly wrong about literally all of physics; And we are not - we checked.

The possible ways in which planets could affect humans are few. The nuclear forces are too short-ranged to have any effect; Electromagnetic effects are basically limited to being able to see the planets; And gravitational and tidal influences are ridiculously tiny.

An A380 passing overhead causes far larger gravitational effects* on a human than the planet Jupiter does.

Anyone who casts a horoscope, and who takes careful note of the positions of the planets, but not of the positions of local air traffic, is at best delusional.

The absence of mechanism is a critical flaw in astrological claims. It might reasonably be ignored in order to look at other ways in which astrology fails, but it cannot be so readily dismissed as a gross failure in its own right.









* and for that matter, it likely has greater effects in all four of the known forces, albeit these are immeasurably small for both Jupiter and the wide-bodied jet. It would be very brave indeed to posit an unknown influence from Jupiter that either is not subject to the inverse square law, or is emitted so much more by Jupiter than by an aircraft, as to swamp inverse square considerations - while still remaining undetected.

 

[Elixir]

An A380 passing overhead causes far larger gravitational effects* on a human than the planet Jupiter does.

At least it’s not a Boeing product, so you probably won’t even get hit by a falling door.

 

[Politesse]

As a Scorpio, I am inclined to be skeptical.

 

[lpetrich]

A Scientific Inquiry Into the Validity of Astrology by John H. McGrew and Richard M. McFall, Journal of Scientific Exploration. Vol. 4, No. 1, pp. 75-83, 1990

Six expert astrologers independently attempted to match 23 astrological birth charts to the corresponding case files of 4 male and 19 female volunteers. Case files contained information on the volunteers' life histories, full - face and profile photographs, and test profiles from the Strong -Campbell Vocational Interest Blank and the Cattell 16-P.F. Personality Inventory. Astrologers did no better than chance or than a nonastrologer control subject at matching the birth charts to the personal data; this result was independent of astrologers' confidence ratings for their predicted matches. Astrologers also failed to agree with one another's predictions.

 

[Loren Pechtel]

Paranormal Borderlands of Science - Google Books - "Best of Skeptical Inquirer" - Prometheus Books

One of its chapters is An Empirical Test of Popular Astrology by Ralph W. Bastedo in Fall 1978

RWB investigated Sun signs, those divisions of the year from around the 21st of each month to around the 21st of the next month. Why the 21st? That's when the equinoxes and solstices are.

He then considered leadership ability, political position (left-right), subjectively-assessed intelligence, IQ, and belief in astrology, testing for correlations with Sun signs.

• Leadership: n = 33, X = 38.012, P = 0.25
• Politics: n = 88, X = 84.274, P = 0.59
• Intellgence: n = 33, X = 35.371, P = 0.36
  [*}IQ: n = 44, X = 48.199, P = 0.31
• Belief in astrology: n = 33, X = 33.043, P = 0.47

n = number of degrees of freedom, X is the chi-squared value, and P is the probability of a random distribution producing a X value greater than the observed value.

So there is zero correlation between Sun sign and these features.

I would expect there to be some difference, but not at a level that can be detected with an n that small.

Spoiler

School works on years, but kids are born all through the year. There are differences between those who were born just before the cutoff from those who are born just after and obviously that point is extremely well correlated with anything else determined by the year.

 

[lpetrich]

Spencer Greenberg on X: "Does astrology work? We tested the ability of 152 astrologers to see if they could demonstrate genuine astrological skill. ..." / X

Can astrological sun signs (or zodiac signs) predict facts about people's lives? We tested it.

We started by measuring 37 life outcomes for a total of 308 people, along with their astrological sun signs. The outcomes we measured included things like:

• Levels of satisfaction with one’s (i) social, (ii) romantic, (iii) work, and (iv) general life circumstances.
• Political alignment (on a progressive to conservative spectrum)
• Religiosity
• Amount that one hosts or plans events for friends, family, or coworkers
• Whether one has received public recognition or awards for achievement
• Time spent volunteering in one’s community in the last year
• Number of distinct crimes for which one has received conviction
• The presence of suicidal ideation in the last 10 years

A typical astrological claim and how it would be tested in this research:

For instance, according to some sources Libras are the most social of the zodiac signs and Capricorn the least social. If this is really true, then we'd expect zodiac signs to be predictive of life outcomes such as how satisfied people are with their friendships, how often people host events, and how many close friends they report having.

The researchers found correlations between Sun signs and these outcomes, and they found zero ability to predict those outcomes from those signs. They tried assigning every subject a randomly-chosen sign, and they found exactly one significant correlation, clearly a false positive.

As a comparison, they found the Big Five personality factors for their subjects. They found significant correlations for 22 out of 37, with an overall average correlation of 0.23. However, Sun signs gave zero.

 

[lpetrich]

Fixed earlier list:

• Leadership: n = 33, X = 38.012, P = 0.25
• Politics: n = 88, X = 84.274, P = 0.59
• Intellgence: n = 33, X = 35.371, P = 0.36
• IQ: n = 44, X = 48.199, P = 0.31
• Belief in astrology: n = 33, X = 33.043, P = 0.47

n = number of degrees of freedom, X is the chi-squared value, and P is the probability of a random distribution producing a X value greater than the observed value.

 

[lpetrich]

....

• Leadership: n = 33, X = 38.012, P = 0.25
• Politics: n = 88, X = 84.274, P = 0.59
• Intellgence: n = 33, X = 35.371, P = 0.36
• IQ: n = 44, X = 48.199, P = 0.31
• Belief in astrology: n = 33, X = 33.043, P = 0.47

n = number of degrees of freedom, X is the chi-squared value, and P is the probability of a random distribution producing a X value greater than the observed value.

So there is zero correlation between Sun sign and these features.

I would expect there to be some difference, but not at a level that can be detected with an n that small.

n is not the sample size, but the number of components of the data that are unfitted by one's model.

I will explain. Imagine that something varies over two discrete parameters i and j: x_ij One wants to model it as the product of two vectors: a_i*b_j For na of the a's and nb of the b's, the number of model parameters is

(na-1) + (nb-1) + 1 = na + nb - 1

since there is only one overall scaling parameter. The total number of x values is na*nb, and their difference is the number of degrees of freedom:

na*nb - (na + nb - 1) = (na-1)*(nb-1)

 

[lpetrich]

The authors then did an ANOVA test, an analysis-of-variance test, looking for evidence that the amount of observed variation is very improbable if there is no correlation. That probability is the p-value, and with their data, they expected to find two p values less than 0.05. They found only one.

The authors conceded that it would be hard to rule out very weak effects.

This inability of our method to identify small effects could also explain why our study didn't pick up on effects that previous studies have documented about the date of birth influencing outcomes like athletic success (due to being either one of the youngest on the team or one of the oldest on the team depending on whether you are born just before or just after an arbitrary cutoff). Such effects may well exist and may be especially important for special populations (such as elite athletes), but if, when averaged across the adult population, these effects are small, our method would not have been likely to identify them.

They also noted that the online dating site OKCupid discovered that partner compatibility has essentially zero variation by the sign of each partner.

 

[lpetrich]

Can astrologers use astrological charts to understand people's character and lives? Our new study put astrologers to the test
Some astrology believers claimed that only look at Sun signs was far too simplistic.

Sun signs alone are insufficient to yield good results. You need to take into account all planets and houses. Otherwise you're compressing all of humanity into only 12 dimensions (sun signs).

"We tested astrology with sun signs" is like saying "we tested the DSM with buzzfeed".

Did you speak with a single professional astrologer before jumping into this?

Yes it's taboo but Astrology is the mother of science, plenty to learn if you'd allowed yourself.

I mean yeah if you're only using their sun signs it's not going to yield accurate results. if you take into account the houses, degrees, planetary positions, etc. it's going to be much more complex and accurate. surprise, surprise, we are unique beings!

Author André Ferretti and his team then recruited 6 astrologers to help them design better tests.

If it's true that a person's natal astrological chart contains lots of information about their character or life, then it stands to reason that astrologers should be able to match people to their charts at a rate that is at least moderately better than random chance. If they can do that, then that would provide substantial evidence that astrology really works!

 

[lpetrich]

Their summary:

• We tested and analyzed the ability of 152 astrologers to accurately match people to their natal charts. For our primary analyses, we excluded anyone who reported no prior astrology experience as well as anyone who believed they would not do better than random guessing at the task of matching people to their astrological charts.
• The 152 astrologers largely believed that they were capable of doing this task with accuracy well above chance. Whereas a random guesser would, on average, only correctly answer 2.4 questions out of 12, astrologers with the least experience thought they had correctly answered 5 charts after completing the study tasks, and those with the most astrology expertise believed they had gotten 10 right.
• Despite their high-degree of confidence in their performance, astrologers as a group performed no better than chance - that is, their distribution of results closely resembled what you'd see if they had all been guessing at random, and the number of charts they matched correctly, on average, was not statistically significantly different than random guessing either.
• Not a single astrologer got more than 5 out of 12 answers correct - even though, after completing the task, more than half of astrologers believed they had gotten more than 5 answers correct.
• More experience with astrology had no statistically significant association with better performance, and the astrologers with the most experience didn't do any better than the rest.
• If astrologers as a group had been able to do even slightly better than chance, getting at least 22% of questions right on average (compared to random guessing, which would yield 20% correct), this study design would have supported the conclusion that astrology works. But, as it turned out, astrologers in the study performed in a manner statistically indistinguishable from random guessing.
• Despite astrologers' belief that they were performing well on the task, there was little agreement among astrologers about which natal chart belonged to each study subject. The astrologers who reported the greatest expertise had the highest level of agreement, but they still only agreed with each other 28% of the time - whereas if they had been selecting charts at random they would have agreed 20% of the time.

Zero success there also.

 

[lpetrich]

How to do research:

1. Make the claim precise
2. Choose a measurement
3. Design a study
4. Run the study and analyze the result

The astrologers were presented with many details about real people and were asked to pick out that person's chart from 5 charts, with the other 4 being randomly generated. They performed as if they were guessing at random.

 

[Swammerdami]

Instead of personal horoscopes, some astrologers were intrigued by the 25,800-year orbit (one of the Milankovitch cycles) of the  Vernal point and allegedly related long-term trends in human civilization. The vernal point is now moving toward the constellation of Aquarius: Some considered this a happy fact back in the days when optimism was in vogue.

. . . An A380 passing overhead causes far larger gravitational effects* on a human than the planet Jupiter does. . .

Anyone who casts a horoscope, and who takes careful note of the positions of the planets, but not of the positions of local air traffic, is at best delusional.

The point is that any effects need not be due to gravitation specifically. While astrology can be rejected, various unexpected results in astronomy and particle physics call into question our "certainty" about physical laws. (Recall the 19th-century physicists who thought there were few discoveries yet to be made!)

But thanks for the optimistic news that Jupiter's mass is too small to cause chaos. Earth's stable orbit may have been prerequisite to the development of advanced life. (How much more massive could Jupiter be and still leave the solar system stable? What about exoplanets? Are stable orbits the rule or the exception?)

 

[Bronzeage]

All the inconstancies of conventional astrology can be explained by quantum astrology.