User Tag List

123 Last

Results 1 to 10 of 38

  1. #1
    Senior Member
    Join Date
    Aug 2013
    MBTI
    ISTP
    Posts
    353

    Default The Scientific State of the MBTI

    I think there is a lot of confusion about the following question:

    Given a set of models, how can we figure out the extent to which those models explain reality?

    Let's assume that we have a number of models, including the MBTI, Big 5, Enneagram, Socionics and Astrology, and let's try to succinctly answer the question I posed.

    The answer to this question is, in broad strokes, Science. Science can help us determine the extent to which these models reliably explain variance in reality.

    The best way we presently have of doing this with personality inventories is a statistic known as test-retest reliability. Ideally, this involves giving someone the exact same inventory decades apart. Given the basic assumption that personality is largely but not entirely innate, the person should respond with almost the same answers decades apart.

    Out of the models I mentioned, I am presently aware of test-retest reliability metrics for the MBTI and Big 5, and they are remarkably high as compared to other research in the field of psychology.

    There is one thing to mention here, which is that there is a potential criticism that simply taking the inventory causes the person to reinforce those traits in themselves over the intervening test-retest days, months, years or decades, which inflates the test-retest reliability metric. This is conceivable. However, practically speaking, the first time people take a personality inventory they don't really recognize that there is a pattern to the questions they are answering. Additional evidence that simply taking an inventory does not cause you to permanently reinforce those traits in yourself outside an explicit attempt to do so (such as visiting this forum every day) is that I have never seen a test-retest reliability statistic for astrology. If astrologers could produce a reproducible test-retest reliability statistic they would be ALL OVER THAT. It would be scientific evidence for astrology.

    With regards to test-retest reliability statistic itself, there are a number of important factors of the experiment. The first is the power of the experiment, which has to do with the number of subjects. The second is the effect size. If you give an astrology inventory to every human being decades apart it will almost certainly be statistically significant and with high power, however, it will probably have a truly tiny effect size, because it's completely made up.

    The MBTI test-retest reliability metric is large (corresponding to the effect size) and has high power. They have had sample sizes on the order of 100,000 subjects. Compare that to studies in psychology which are often done with the bare minimum of around 20 subjects and thus have low power (and inflated effect sizes).

    Here it is from Consulting Psychologists Press, this year:

    No, we haven’t been “duped” by the world’s most popular personality assessment

    Quote Originally Posted by CPP
    On a related note, Krznaric also references supposed “…low ‘test-retest reliability…’” In actuality, the test-retest correlations for the most recent version of the Myers-Briggs are in the range of .57 to .81, which is considered quite good for psychometric assessments. It’s worth pointing out that instances where people receive different results typically occur when they have a low PCI along a certain category. For example, if results initially show a slight preference for Extraversion, that individual might at a later time show a very slight preference for Introversion. It is rare for someone with a clear or very clear preference to show conflicting results from a subsequent assessment.
    You read that right: 57% - 81% of people who retake the MBTI at varying intervals end up retesting as the exact same type. That's amazing!.

    If you believe in a personality theory that hasn't published this statistic, you should demand that they tell you why, and seriously reconsider whether that system is, to use philosopher Harry Frankfurt's meaning of the term, bullshit. In reality, it's probably the case that the enneagram and Socionics could produce a small but powerful test-retest reliability metric because they are high dimensional rotations of the Big 5 and MBTI, which are strongly correlated with eachother. However, without that statistic, we have no clue how much confidence to put in them, and so we should definitely be asking the people who are pimping those inventories why, given how strongly they believe in it, they don't take the extra step of running the experiment.

    And lastly here's a meta-analysis that doesn't come from the creators of the test:

    Capraro, R.M. & Capraro, M.M. (2002). Myers-Briggs Type Indicator Score Reliability Across: Studies a Meta-Analytic Reliability Generalization Study. Educational and Psychological Measurement

    Quote Originally Posted by Capraro
    The Myers-Briggs Type Indicator (MBTI) was submitted to a descriptive reliability generalization (RG) analysis to characterize the variability of measurement error in MBTI scores across administrations. In general, the MBTI and its scales yielded scores with strong internal consistency and test-retest reliability estimates, although variation was observed.

  2. #2
    Senior Member reckful's Avatar
    Join Date
    Jul 2013
    MBTI
    INTJ
    Enneagram
    5
    Posts
    559

    Default

    On the one hand, I agree with you that the MBTI's test-retest stats are respectable, that the MBTI is respectable overall by the soft-scientific standards applicable to personality typologies, and that the MBTI is tapping into four of the Big Five personality dimensions.

    On the other hand, if I'm correct in assuming that you're one of those cognitive function renegades who consider themselves S's even though they come out N on the official MBTI instrument, I find it a little ironic that you, in particular, have posted an OP raving about the MBTI's reliability.

    ----------------------------------------------------------------

    "Well..." said funtensity, "but... the test mistypes me consistently! That's the point!" And with that he scowled, turned abruptly and shuffled back to the spartan but funtense burrow where he did all his best introverted thinking.

  3. #3
    Senior Member
    Join Date
    Aug 2013
    MBTI
    ISTP
    Posts
    353

    Default

    Not sure what the relevancy is but I tested very clear on all four dimensions, and, FWIW, I also went to a weeklong CAPT qualifying course - 10 years ago.

  4. #4
    Senior Member reckful's Avatar
    Join Date
    Jul 2013
    MBTI
    INTJ
    Enneagram
    5
    Posts
    559

    Default

    Quote Originally Posted by funtensity View Post
    Not sure what the relevancy is but I tested very clear on all four dimensions, and, FWIW, I also went to a weeklong CAPT qualifying course - 10 years ago.
    What types have you found yourself attracted to?

    Quote Originally Posted by funtensity View Post
    INTJ - good communication. good activity partners. some of my best memories. would consider getting back together. also most of my friends in general.
    INFJ - my longest relationship and also, separately, my longest opposite sex friendship. endlessly kind, understanding, reliable and warm.
    ENFP - have a crush on almost every one I meet. dated one, she left me on a whim for an impractical dream. they are fun to toss around ideas with.
    INFP - wants to spend all her time with me. endlessly sweet. and yes, idealistic.
    What are your favorite types?

    Quote Originally Posted by funtensity View Post
    For dating, *NF*, especially INFJ, ENFP, INFP. For friends, *NT*, especialy INTJ, ENTP, ENTJ. For recreational activities, i.e., volleyball, ISTP. Dat reaction time.
    It's good to know you at least schedule some recreational time with your fellow S's, to give yourself an occasional break from all that S/N interaction.

  5. #5
    Senior Member
    Join Date
    Aug 2013
    MBTI
    ISTP
    Posts
    353

    Default

    Ok... that's bordering on creepy. But yes, I'm addicted to information and so have been in academia for a long time. Surprise surprise, I'm surrounded by Ns. That doesn't mean I am one.

  6. #6
    Senior Member reckful's Avatar
    Join Date
    Jul 2013
    MBTI
    INTJ
    Enneagram
    5
    Posts
    559

    Default

    Quote Originally Posted by funtensity View Post
    I identify with MBTI ISTP and Jung's description of introverted thinking. I am much more like Kant or Nietzsche than Einstein. The kind of extraverted thinking mental imagery that Einstein used to see how gravity warps spacetime is not my first nature.

    The MBTI has very strong correlations with the Big 5 and both have very strong test-retest reliability. Thus, I must ask on what basis Socionics has been validated as an "evolved" version of the MBTI.
    Quote Originally Posted by funtensity View Post
    Let's assume that we have a number of models, including the MBTI, Big 5, Enneagram, Socionics and Astrology, and let's try to succinctly answer the question I posed.

    The answer to this question is, in broad strokes, Science. Science can help us determine the extent to which these models reliably explain variance in reality.

    The best way we presently have of doing this with personality inventories is a statistic known as test-retest reliability. ...

    Out of the models I mentioned, I am presently aware of test-retest reliability metrics for the MBTI and Big 5, and they are remarkably high as compared to other research in the field of psychology. ...

    If you believe in a personality theory that hasn't published this statistic, you should demand that they tell you why, and seriously reconsider whether that system is, to use philosopher Harry Frankfurt's meaning of the term, bullshit.
    It sounds like you're seriously confused. The "MBTI" that has "strong correlations" with the Big Five and respectable "test-retest reliability" is the four dichotomies, not "introverted thinking" or "extraverted thinking" or any of the other so-called "cognitive functions."

    The MBTI dichotomies, which substantially line up with four of the Big Five dimensions, now have decades of studies in support of their validity and reliability, while the "cognitive functions" — which James Reynierse (in the 2009 article linked below) refers to as a "category mistake" — have barely been studied. And the reason they've barely been studied is that, unlike the dichotomies, they've never been taken seriously by any significant number of academic psychologists. Going all the way back to 1985, the MBTI Manual described or referred to somewhere in the neighborhood of 1,500 MBTI studies and, as I understand it, not one of the many study-based correlations reported in the manual were framed in terms of the functions. And many more dichotomy-based studies have been done in the years since. The third edition of the MBTI Manual was published in 1998 and, as Reynierse notes in the linked article, it cited a grand total of eight studies involving "type dynamics" (i.e., the functions model) — which Reynierse summarizes as "six studies that failed, one with a questionable interpretation, and one where contradictory evidence was offered as support."

    Dario Nardi's one of the leading cognitive functions guys (as I'm sure you know), and his test is arguably the most-linked-to cognitive functions test — but, as further discussed in this INTJforum post, INTJs typically get high Ni scores and high Ne scores (with Ni not substantially favored over Ne), and high Te scores and high Ti scores (with Te not substantially favored over Ti), when they take Nardi's test.

    When it comes to the functions, forget retest reliability. As I understand it, there isn't a single function-based test on or off the internet on which, e.g., INTJs reliably get high Ni and Te scores and low Ti and Ne scores and INTPs reliably get high Ti and Ne scores and low Ni and Te scores — never mind scoring the third and fourth functions in a way that matches the standard functions model. Far from racking up impressive test-retest stats, the cognitive functions haven't even made it out of the starting gates.

    And what functions model should a good test be matching, anyway? Myers acknowledged that the majority of Jung scholars believed (rightly, IMHO) that Jung's model for a Ti-dom with an S auxiliary was Ti-Si-Ne-Fe. Myers' model was Ti-Se-Ne-Fe — although, as explained in my linked INTJforum post (below), Myers, despite some lip service to the contrary, essentially abandoned the functions for the dichotomies. Harold Grant's model — followed by Berens and Nardi and most of the other modern functions theorists — was Ti-Se-Ni-Fe.

    In any case, regardless of which model any functions fan wants to choose, there's never been a respectable test for any of them.

    You say, "If you believe in a personality theory that hasn't published [its test-retest statistics,] you should demand that they tell you why, and seriously reconsider whether that system is, to use philosopher Harry Frankfurt's meaning of the term, bullshit." So... OK, funtenstity, why do you keep talking about the "cognitive functions" in post after post after post?

    If you're interested, you can find out quite a bit more about the place of the functions (or lack thereof) in the MBTI's history — and the tremendous gap between the dichotomies and the functions in terms of scientific respectability — in this long INTJforum post.

    Links in INTJforum posts don't work if you're not a member, so here are replacements for the two links in that post:


  7. #7
    Banned
    Join Date
    Mar 2012
    MBTI
    INTJ
    Enneagram
    5w6 sx/so
    Socionics
    Will
    Posts
    5,927

    Default

    Any kind of test relies on the idea of the same or similar results being applied in the majority of contexts universally, based on the idea that what was repeated in the past will prove true into the future.

    It's easy to measure things by evidence as we see them, because then we don't have to think. But that doesn't mean we're seeing the evidence correctly.

  8. #8
    Senior Member
    Join Date
    Aug 2013
    MBTI
    ISTP
    Posts
    353

    Default

    Quote Originally Posted by reckful View Post
    It sounds like you're seriously confused. The "MBTI" that has "strong correlations" with the Big Five and respectable "test-retest reliability" is the four dichotomies, not "introverted thinking" or "extraverted thinking" or any of the other so-called "cognitive functions."
    Reckful, it doesn't matter how you unpack the MBTI when comparing it to the Big 5. It still explains a large amount of the variance in the Big 5, and vice versa. This means they are high dimensional rotations of each other.

    With regards to how to unpack the MBTI, we can only trust the system of cognitive functions that the MBTI uses, because their test is the only one that we have test-retest reliability statistics for.

    Easy peasy.

    RaptorWizard, that's really not enough substance to go on.

  9. #9
    Senior Member Jaguar's Avatar
    Join Date
    May 2007
    Posts
    12,409

    Default

    Quote Originally Posted by funtensity View Post
    I also went to a weeklong CAPT qualifying course - 10 years ago.
    I figured as much. Many of your posts read like an advertisement for the MBTI.

  10. #10
    Senior Member
    Join Date
    Aug 2013
    MBTI
    ISTP
    Posts
    353

    Default

    Quote Originally Posted by Jaguar View Post
    I figured as much. Many of your posts read like an advertisement for the MBTI.
    Ad Hominem: Failure to refute the original claim.

Similar Threads

  1. The MBTI types of MLB stars.
    By ferunandesu in forum Arts & Entertainment
    Replies: 13
    Last Post: 07-05-2008, 04:53 PM
  2. Scientific evidence for the MBTI assumptions
    By Nails in forum Myers-Briggs and Jungian Cognitive Functions
    Replies: 19
    Last Post: 12-17-2007, 12:13 PM
  3. What's the MBTI of..
    By Alfa Prime in forum Popular Culture and Type
    Replies: 3
    Last Post: 11-06-2007, 08:59 PM
  4. Tone of the MBTI forum...
    By Athenian200 in forum Myers-Briggs and Jungian Cognitive Functions
    Replies: 12
    Last Post: 09-30-2007, 02:24 PM
  5. What is the point of the MBTI?
    By Dufresne in forum Myers-Briggs and Jungian Cognitive Functions
    Replies: 15
    Last Post: 05-31-2007, 04:37 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
Single Sign On provided by vBSSO