Bias in self-scored tests like the MBTI

[Ghost of the dead horse]

Personality test results come directly from our answers to the test questions. How do we then answer the test questions? Probably much like any other questions. Something immediately comes to our mind when a question is presented, and then we may proceed on it consciously.

Now, "biases" mean that something shady is happening in the "immediate" part. There's things like question substitution, where we replace a difficult question with another.

For example, "Am I an organized person?" might become "Did I put my pencils to the drawer?" for someone. Another person might replace it with "Do I appear in meetings on time?"

Then there's phenomena like availability heuristic. Some situations come more readily to the mind than the others. These easily thought-of situations form the base group for comparison in our counscious effort of answering the question. Maybe we've experienced those situations a lot, or maybe we've read a lot about them.

For example, "Do you appreciate art?" might become "Do I appreciate art more than my spouse who never goes to the art exhibitions?" for someone, and "Am I annoyed by those art students at my local café" for someone else. In both situations, the answer is further adjusted by the people in question.

This is sad, as a personality test would be more valuable if it measured more than what kind of company the person keeps. For that we would require multiple different persons to score us, or the use of less error-prone questions in the questionaire.

If a test promises to tell you how extroverted you are and the test asks YOU how extroverted you are, the test isn't doing it's job. The test maker should go through the trouble to find out questions that are somewhat reliable on a population level, not only on my or your circle of acquintances.

 

[five sounds]

This is why I take test results with a grain of salt. It's hard to tell someone they are subject to error in this way, but honestly I believe we all are.

 

[Doctor Cringelord]

Our emotional states also color how we perceive, interpret and respond to the questions at any given time.

 

[SearchingforPeace]

The tests are only as good as we are actually self-aware. It is very easy to test wrong if we do not adequately know ourselves.

So, for me, it took awhile for me to test accurately, in part because it took awhile to actually see myself the way I really am.

 

[Doctor Cringelord]

I could take it today and test as an ESTP if I was in the right mood. It's pointless.

 

[Frosty]

Why I refuse to type, or one of the reasons at least. Write more later, this is a good topic.

 

[cascadeco]

Well yeah, there's bias. Of course it's going to reflect how we currently perceive ourselves. Maybe we're perceiving/assessing accurately, maybe we're not.

There's also the element of, if you don't have a good gauge of the spectrum of possibilities, and real world experience with various types, you may not know where exactly you fall on a particular trait or statement. So say you're a person who thinks you're logical - in fact you can be very capable of it. But until or unless you meet people who live and breathe that parameter, you may be wrong in thinking you have a preference for it.

Also I feel many tests don't emphasize enough the PREFERENCE aspect -- seems like a lot are worded more 'I'm capable of this / I can do this/I'm good at this' -- which doesn't really work for determining types. People can become skilled at things that they may not actually prefer or like.

 

[Bush]

It turns out that I'm corroborating everything [MENTION=988]Santtu[/MENTION] has said, but maybe providing another perspective.

If a test is done right, each of the questions actually does account for confounding factors like culture or emotional state. Each will have been rigorously examined to make damn sure that it measures what it's supposed to measure, and any limitations on when it's expected to be accurate should be made clear to those who administer it.

But even that's not everything. There could be a problem with abstraction. Questions that are too abstract aren't meaningful.

A question such as "I am an extroverted person." maps one-to-one with "I/E" -- you answer 'no,' and you're an I; 'yes,' and you're an E. Of course it measures what it's supposed to measure -- it's short circuited that way. Verily, we have stated that 1 = 1. It's way too easy to be lazy, using abstraction to quickly and effortlessly produce 'good' results.

They're also difficult for subjects to answer. What "I am an extroverted person." is, is way too open to interpretation. You begin to ask yourself more specific questions, guessing at whether, by and large, they can help you decide whether or not you are an extroverted person. The test should take that guesswork out and ask specifics for you. (If some specific question isn't applicable -- perhaps you don't have a wife in the first place, or you don't even have pencils or a drawer -- then there's always the "N/A" option. That works in many cases.)

But the official MBTI demands that you tell it whether "I hurry to pick up the phone before anyone else does." .. in an era where we all have individual cellphones.

 

[Doctor Cringelord]

We don't all have individual cellphones. Well, I don't. Probably the only person in the civilized world without one, but that's a different topic.

The phone one could be changed to "I prefer to let calls go to voicemail first" or something similar

 

[Jaq]

Of course there is bias in self scored tests like the MBTI. That is why I see the MBTI as mere a conversational piece. Though the MBTI, Enneagram, Socionics and other personality assessment tests I find to be quite interesting conversational pieces.

 

[ChocolateMoose123]

The tests are only as good as we are actually self-aware. It is very easy to test wrong if we do not adequately know ourselves.

So, for me, it took awhile for me to test accurately, in part because it took awhile to actually see myself the way I really am.

When I was younger, 12 or13, I would test Ni dom (INTJ) as frequenty as Ti dom. At that time in my life, I was under enormous stress. I was testifying in court against my father and dealing with a lot.

So much so that the skin on my palms peeled off (extreme stress reaction). I attribute that to Ti/Ni loop.

 

[HongDou]

[MENTION=10082]Starry[/MENTION] maybe this all factors into the phenomenon we discussed in your thread.

For example, "Do you appreciate art?" might become "Do I appreciate art more than my spouse who never goes to the art exhibitions?" for someone, and "Am I annoyed by those art students at my local café" for someone else. In both situations, the answer is further adjusted by the people in question.

Yeah, it's a role vs. identity issue. I wonder if this is more of an extrovert problem because of the focus on what's going down on the outside. I know a friend who IDs now as ENTP 7w8 but only after scoring 7w8 on an Enneagram test while IDing as INTP which is a rarer score that can facilitate examining oneself under a new perspective. Turns out she agreed with INTP because she's more misanthropic and "likes to have fun on her own" which doesn't necessarily discount extroversion. So I explained aspects of Get Things Going vs Behind The Scenes and leading with Ne vs Ti and she ended up agreeing more with ENTP. So what led her to the conclusion that she was introverted was the fact that she's not inherently a people-lover or outgoing as one would assume extroverts might be, basically over-identifying with her social role and not identifying enough with her actual preferences...which were in reality fairly extroverted preferences.

I'm sure with the ESFJs identifying as NFP scenario that's starting to be discussed as of late that is partly why they could lean towards NFP. Even if they don't seek to fulfill certain social roles I'm sure most ESFJs are aware of how they fit into a group, what they bring, how their presence uniquely influences others, etc. If an ESFJ is bringing "quirky ideas" to the social circle and not as much "mothering" or "a friendly touch of responsibility" or whatever else their bios make them out to have, it makes sense they might identify and test more as NFP as an NFP's social role is described as someone who is more of a dreamer or inspirer of their group.

Even though it's basically stating the obvious, I'm sure MBTI scores more accurately for those who are more self-aware. I know I used to feel shitty about the fact that my my self-image wasn't accurately reflected in my social role because it made me feel like I wasn't being true to myself and others didn't 100% understand me, but the bottom line is you can never fully portray your true self to the world because in the world you're on a spectrum compared to other people. So even if your position in a group conflicts with your actual preferences it doesn't make your preferences any less valid.

So I think the issue is that the world can never fully comprehend someone's identity and existence. And people might reconcile this idea by either feeling internal conflict and misunderstanding or identifying too much with how the world perceives them...or probably a mix of both a lot of the time as well.

 

[hjgbujhghg]

I agree that the test can be inaccurate, because people might live in delusions about who they are. There's a difference between how I perceive myself and how others might perceive me and I am aware of that. Also the mood and the overall psychological state is very important in MBTI.

Yet I have to say I can barely score anything else than an INFP. I usually score INFJ on JCF tests and INFP on MBTI tests.

 

[Zen_alpha]

I think a better test will be a test that put you in situation and according to those choices you will get the result. However, to make that possible, we need to find a way to put the choices without showing any hint which function/letter it represent. Each choice will put more new choices. For example, if tester A choose choice 1, then he will get 4 more choices that is different than tester B.
Example:-

Tester A choose choice 1
1- X
2- Y
3- Z
4- W
Tester B choose choice 2
1- U
2- I
3- O
4- K

The idea is based on case studies, but with choices.

 

[pinkgraffiti]

This is why, IMO it's all about the cognitive functions and not about the % of letters in a random question test. I think the first few times I took the test it was giving me ENFJ/ENFP because I had around 50% judging or so. I mean, what is judging anyway? That test was actually detecting my Te, not that I am a "J". Also, how can a test give you " ENFJ or ENFP"? All the functions are different between those two types, not just a "letter"...!!
So, I read a lot on cognitive functions and that made everything clear. So I wouldnt trust anyone else to actually know their MBTItype if they havent read about cognitive functions AND analysed themselves. Honestly and deeply.

 

[Ghost of the dead horse]

The need for self-awareness is often mentioned in the replies. I do appreciate the search process to understand the people around you, yourself, and your mbti type.

There are other types of tests and surveys. Marketing research surveys ask how many times a person goes to the movies, how many times to sports events, etc.. and they cluster the respondents accordingly. They define the groups by external means. They find people who think highly of art and culture, yet rarely go to any such events to corroborate it. They find people who appreciate culture and go to events, those who don't appreciate and don't go, etc. And then they find the demographics of all these groups.

This kind of research is pretty interesting as well, as they have a factual part, opinion part and a variety of background statistics related to their combinations. I'd like to see something like that done with personality types. Your informed opinions combined in a relevant manner with some hard data. Someone's result might then be that they have a personality type XYZW in the traditional way, and the type OPQR when measured completely externally. Maybe there could be an explanation of the differences between self-assed and the externally validated results. I would buy that.

 

[Ghost of the dead horse]

If there were an MBTI letter I'd be wrong about myself, it would be P/J. If this were so, it would be due to my appreciation of J in the last decade. I was wrong about my T/F 20 years ago, when I felt especially liberated due to some significant changes in my life.

Interpreting this event in terms of availability heuristic, 20 years ago my base group for comparison would have been the inhibited life I lived before. Being more liberated felt more like F to me, making me evaluate myself as an F before realizing the extent of T/F things people actually do in the world.

I don't know if it's typical of people to exaggarate the changes they go through. If they did, this "self-awareness" would include understanding the impact of latest significant life-changes. My theory would be that the people overestimate the impact of life changing events and developments. How is it?

 

[miss fortune]

yeah... most people have enough trouble knowing themselves in the first place... or they hold an idealized version of themselves in their head... you never really know what you seem like to other people from the outside unless you obsessively ask them about yourself and then you lose all of your friends pretty quickly because you look like some sort of self obsessed monster

not to mention crappy questions and that it doesn't take into account that people can have interests that don't fall into "Sensor" or "Thinker" or whatever categories as well

I often test as an INTP when I'm being perfectly honest and I'd make a pretty shit INTP according to everyone on here... I just happen to value peace and quiet in my free time because I don't get that the 44 hours or so that I spend at work and the contrast pleases me... I also enjoy talking about ideas and going to the art museum by myself because I like things that are interesting and I've never seen why stuff like that should be an inherently "iNtuitive" thing I guess I'm saying that the questions are crappy and biased as well because they don't take into account that not everyone falls into a cut and dried "everyone who is x does y" category

not to mention conditional circumstances for things and so on and so forth... so many problems with the test that I have no faith in it really because I don't feel like it judges things accurately

 

[magpie]

yeah... most people have enough trouble knowing themselves in the first place... or they hold an idealized version of themselves in their head... you never really know what you seem like to other people from the outside unless you obsessively ask them about yourself and then you lose all of your friends pretty quickly because you look like some sort of self obsessed monster

This is a tangent but when I was doing an acting program we had to ask random people (I chose to ask strangers in a coffee shop) what their first impression of us was. To see how we came off for casting purposes but it works just to get a general sense of yourself (though most people are going to give positive answers to be nice). I got mysterious, regal, and of course, tall. I don't know if that really influences my test taking though. But if you're curious what other people think of you it works to ask random strangers you're never going to see again. Less awkward in the long run. (Though obviously less in depth answers. But first impressions matter too.)

 

[miss fortune]

This is a tangent but when I was doing an acting program we had to ask random people (I chose to ask strangers in a coffee shop) what their first impression of us was. To see how we came off for casting purposes but it works just to get a general sense of yourself (though most people are going to give positive answers to be nice). I got mysterious, regal, and of course, tall. I don't know if that really influences my test taking though. But if you're curious what other people think of you it works to ask random strangers you're never going to see again. Less awkward in the long run. (Though obviously less in depth answers. But first impressions matter too.)

that could be an interesting little experiment

I know that when customers tried to describe me to coworkers the description was almost always "short, dark hair, funny and speaks spanish" ... how does height play into people's descriptions so much?!