I think there is a lot of confusion about the following question:
Given a set of models, how can we figure out the extent to which those models explain reality?
Let's assume that we have a number of models, including the MBTI, Big 5, Enneagram, Socionics and Astrology, and let's try to succinctly answer the question I posed.
The answer to this question is, in broad strokes, Science. Science can help us determine the extent to which these models reliably explain variance in reality.
The best way we presently have of doing this with personality inventories is a statistic known as test-retest reliability. Ideally, this involves giving someone the exact same inventory decades apart. Given the basic assumption that personality is largely but not entirely innate, the person should respond with almost the same answers decades apart.
Out of the models I mentioned, I am presently aware of test-retest reliability metrics for the MBTI and Big 5, and they are remarkably high as compared to other research in the field of psychology.
There is one thing to mention here, which is that there is a potential criticism that simply taking the inventory causes the person to reinforce those traits in themselves over the intervening test-retest days, months, years or decades, which inflates the test-retest reliability metric. This is conceivable. However, practically speaking, the first time people take a personality inventory they don't really recognize that there is a pattern to the questions they are answering. Additional evidence that simply taking an inventory does not cause you to permanently reinforce those traits in yourself outside an explicit attempt to do so (such as visiting this forum every day) is that I have never seen a test-retest reliability statistic for astrology. If astrologers could produce a reproducible test-retest reliability statistic they would be ALL OVER THAT. It would be scientific evidence for astrology.
With regards to test-retest reliability statistic itself, there are a number of important factors of the experiment. The first is the power of the experiment, which has to do with the number of subjects. The second is the effect size. If you give an astrology inventory to every human being decades apart it will almost certainly be statistically significant and with high power, however, it will probably have a truly tiny effect size, because it's completely made up.
The MBTI test-retest reliability metric is large (corresponding to the effect size) and has high power. They have had sample sizes on the order of 100,000 subjects. Compare that to studies in psychology which are often done with the bare minimum of around 20 subjects and thus have low power (and inflated effect sizes).
Here it is from Consulting Psychologists Press, this year:
No, we haven’t been “duped” by the world’s most popular personality assessment
You read that right: 57% - 81% of people who retake the MBTI at varying intervals end up retesting as the exact same type. That's amazing!.Originally Posted by CPP
If you believe in a personality theory that hasn't published this statistic, you should demand that they tell you why, and seriously reconsider whether that system is, to use philosopher Harry Frankfurt's meaning of the term, bullshit. In reality, it's probably the case that the enneagram and Socionics could produce a small but powerful test-retest reliability metric because they are high dimensional rotations of the Big 5 and MBTI, which are strongly correlated with eachother. However, without that statistic, we have no clue how much confidence to put in them, and so we should definitely be asking the people who are pimping those inventories why, given how strongly they believe in it, they don't take the extra step of running the experiment.
And lastly here's a meta-analysis that doesn't come from the creators of the test:
Capraro, R.M. & Capraro, M.M. (2002). Myers-Briggs Type Indicator Score Reliability Across: Studies a Meta-Analytic Reliability Generalization Study. Educational and Psychological Measurement
Originally Posted by Capraro