Bias in self-scored tests like the MBTI

[Jaguar]

For example, "Am I an organized person?" might become "Did I put my pencils to the drawer?" for someone. Another person might replace it with "Do I appear in meetings on time?"

For example, "Do you appreciate art?" might become "Do I appreciate art more than my spouse who never goes to the art exhibitions?" for someone, and "Am I annoyed by those art students at my local café" for someone else. In both situations, the answer is further adjusted by the people in question.

I agree with your first example, but not the second. Asking if I appreciate art is crystal clear: Yes, I appreciate art. It's all over my house and I go to galleries all the time. It has nothing to do with anyone except me. The same would be true if asked "Do you appreciate rap music?" My answer would be, "Hell no." It doesn't involve anyone except me.

 

[EJCC]

I totally agree with the OP.

That's why it's so important for tests to be sneaky. Sneaky enough for the answers to not obviously link to one or another result.

A good example of how this can work, and not work, is the original set of Sorting Hat questions on "Pottermore", vs. the new set. The first set was predictable as hell -- four answers to (pretty much) every question, with obvious links to each house. And everyone could test as whichever house they wanted to be in. The new set has more abstract questions -- some with ten answers, some with two. You end up getting results that you wouldn't expect, that only make sense when you reflect on it.

This should be what we strive for with personality tests overall. Outsmarting the test-takers.

 

[Bush]

I totally agree with the OP.

That's why it's so important for tests to be sneaky. Sneaky enough for the answers to not obviously link to one or another result.

Exactly.

Those who already know quite a bit about MBTI wonder what items such as "I brood a lot." have to do with the MBTI. Well, they don't have anything to do with the MBTI; they're control questions, often included with the intent of being sneaky. More than likely, they have zero effect on the final scoring. By design.

 

[Jaguar]

Sometimes you can see bias or inconsistent value = in the code:

1.
<INPUT TYPE=RADIO NAME=Q1 VALUE="ff">(A) It is more important to seek consensus in decision-making than to make decisions quickly. <br>
<INPUT TYPE=RADIO NAME=Q1 VALUE="tt">(B) Making the correct decision is more important than trying to pander to everybody's concerns. <br>
<P>
2.
<br>
<INPUT TYPE=RADIO NAME=Q2 VALUE="nn">(A) Don't waste time endlessly gathering facts -
learn how to shift interpretive frames. <br>
<INPUT TYPE=RADIO NAME=Q2 VALUE="ss">(B) Don't waste your time imagining unrealistic possibilities - learn how to use statistics to assess the viability of a strategy.
<br>
<P>
3. <br>
<INPUT TYPE=RADIO NAME=Q3 VALUE="tt">(A) Isn't one better off learning how to detect significant causal connections than trying to be sensitive to mood swings?
<br>
<INPUT TYPE=RADIO NAME=Q3 VALUE="ss">(B) Isn't one better off learning how to
make accurate observations than trying to honor mere hunches?<br>

<p>
4. <br>
<INPUT TYPE=RADIO NAME=Q4 VALUE="nn">(A) It is more important to be able to appreciate the profound meaning of a central metaphor than to notice what color shirt someone is wearing.<br>
<INPUT TYPE=RADIO NAME=Q4 VALUE="ff">(B) Being skilled in empathic listening is more important than knowing how to decide whether a syllogism is valid.<br>
<P>
<br>
<P>
5. I am more intrigued by <br>
<INPUT TYPE=RADIO NAME=Q5 VALUE="n">(A) puzzles and anomalies<br>
<INPUT TYPE=RADIO NAME=Q5 VALUE="s">(B) patterns<br>
<INPUT TYPE=RADIO NAME=Q5 VALUE="t">(C) causal connections<br>
<p>
6. When I am in a leadership position I often find I am best as <br>
<INPUT TYPE=RADIO NAME=Q6 VALUE="f">(A) a behind-the-scenes catalyst<br>
<INPUT TYPE=RADIO NAME=Q6 VALUE="n">(B) a visionary<br>
<INPUT TYPE=RADIO NAME=Q6 VALUE="s">(C) a good steward<br>
<p>
7. I would least like being called<br>
<INPUT TYPE=RADIO NAME=Q7 VALUE="s">(A) an idealist<br>
<INPUT TYPE=RADIO NAME=Q7 VALUE="f">(B) a logical positivist<br>
<INPUT TYPE=RADIO NAME=Q7 VALUE="t">(C) a humanist<br>
<p>
8. When I am conversing with someone, I get most annoyed when they speak in a way that is<br>
<INPUT TYPE=RADIO NAME=Q8 VALUE="s">(A) mystifying<br>
<INPUT TYPE=RADIO NAME=Q8 VALUE="n">(B) desultory<br>
<INPUT TYPE=RADIO NAME=Q8 VALUE="f">(C) manipulative<br>
<INPUT TYPE=RADIO NAME=Q8 VALUE="t">(D) sappy<br>
<p>
9. I'd say that I'm most often concerned with <br>
<INPUT TYPE=RADIO NAME=Q9 VALUE="f">(A) what ought to be<br>
<INPUT TYPE=RADIO NAME=Q9 VALUE="s">(B) what is<br>
<INPUT TYPE=RADIO NAME=Q9 VALUE="n">(C) what could be<br>
<INPUT TYPE=RADIO NAME=Q9 VALUE="t">(D) what will happen in certain circumstances<br>
<p>
10. I am most likely to feel <br>
<INPUT TYPE=RADIO NAME=Q10 VALUE="t">(A) uneasy about disruptive emotion<br>
<INPUT TYPE=RADIO NAME=Q10 VALUE="s">(B) disoriented by too many choices<br>
<INPUT TYPE=RADIO NAME=Q10 VALUE="n">(C) oppressed by accidental contingencies<br>
<INPUT TYPE=RADIO NAME=Q10 VALUE="f">(D) imposed upon by impersonal rules<br>
<p>
11. I would miss it most if at my workplace there were an absence of
<br>
<INPUT TYPE=RADIO NAME=Q11 VALUE="t">(A) clear lines of authority <br>
<INPUT TYPE=RADIO NAME=Q11 VALUE="n">(B) individual autonomy<br>
<INPUT TYPE=RADIO NAME=Q11 VALUE="f">(C) teamwork<br>
<p>
12. I am least offended when I am labeled a <br>
<INPUT TYPE=RADIO NAME=Q12 VALUE="t">(A) rationalist<br>
<INPUT TYPE=RADIO NAME=Q12 VALUE="s">(B) empiricist<br>
<INPUT TYPE=RADIO NAME=Q12 VALUE="f">(C) humanist<br>
<INPUT TYPE=RADIO NAME=Q12 VALUE="n">(D) romantic<br>
<p>
13. Although I don't like it, I must admit that I can sometimes get <br>
<INPUT TYPE=RADIO NAME=Q13 VALUE="t">(A) moody and sentimental<br>
<INPUT TYPE=RADIO NAME=Q13 VALUE="f">(B) stubborn and trite<br>
<INPUT TYPE=RADIO NAME=Q13 VALUE="n">(C) too detailed and pedantic<br>
<INPUT TYPE=RADIO NAME=Q13 VALUE="s">(D) superstitious and somewhat flakey<br>
<p>
14. Although I may not brag about it, I can sometimes be<br>

<INPUT TYPE=RADIO NAME=Q14 VALUE="s">(A) perfectly precise and thorough <br>
<INPUT TYPE=RADIO NAME=Q14 VALUE="f">(B) really compassionate<br>
<p>
15. People with whom I am not compatible might misconstrue my actions as <br>
<INPUT TYPE=RADIO NAME=Q15 VALUE="t">(A) mean-spirited<br>
<INPUT TYPE=RADIO NAME=Q15 VALUE="f">(B) a bleeding-heart<br>
<INPUT TYPE=RADIO NAME=Q15 VALUE="s">(C) uptight<br>
<INPUT TYPE=RADIO NAME=Q15 VALUE="n">(D) mystifying<br>
<p>
16. In most situations, the bottom line for me is <br>
<INPUT TYPE=RADIO NAME=Q16 VALUE="t">(A) what is true<br>
<INPUT TYPE=RADIO NAME=Q16 VALUE="f">(B) what is of real value<br>
<INPUT TYPE=RADIO NAME=Q16 VALUE="n">(C) what is meaningful <br>
<p>
17. At this point in my life I probably need to learn how to be more<br>
<INPUT TYPE=RADIO NAME=Q17 VALUE="t">(A) loving <br>
<INPUT TYPE=RADIO NAME=Q17 VALUE="f">(B) productive<br>
<INPUT TYPE=RADIO NAME=Q17 VALUE="s">(C) creative<br>
<INPUT TYPE=RADIO NAME=Q17 VALUE="n">(D) successful<br>
<p>
18. I least trust people who insist on being<br>
<INPUT TYPE=RADIO NAME=Q18 VALUE="t">(A) ethical<br>
<INPUT TYPE=RADIO NAME=Q18 VALUE="n">(B) factual<br>
<INPUT TYPE=RADIO NAME=Q18 VALUE="s">(C) profound<br>
<INPUT TYPE=RADIO NAME=Q18 VALUE="f">(D) logical
<p>
19. I never seem to impress others as being very<br>
<INPUT TYPE=RADIO NAME=Q19 VALUE="t">(A) caring<br>
<INPUT TYPE=RADIO NAME=Q19 VALUE="n">(B) observant<br>
<INPUT TYPE=RADIO NAME=Q19 VALUE="f">(C) persuasive<br>
<INPUT TYPE=RADIO NAME=Q19 VALUE="s">(D) insightful<br>
<p>
20. I most admire people who <br>
<INPUT TYPE=RADIO NAME=Q20 VALUE="t">(A) can get things done<br>
<INPUT TYPE=RADIO NAME=Q20 VALUE="n">(B) are profound<br>
<INPUT TYPE=RADIO NAME=Q20 VALUE="f">(C) are warm and kind<br>
<INPUT TYPE=RADIO NAME=Q20 VALUE="s">(D) are stable and successful<br>
<p>
21. I don't like to admit it, but sometimes I regret that I <br>
<INPUT TYPE=RADIO NAME=Q21 VALUE="n">(A) have paid too little attention to material things<br>
<INPUT TYPE=RADIO NAME=Q21 VALUE="s">(B) don't have a more meaningful life<br>
<INPUT TYPE=RADIO NAME=Q21 VALUE="f">(C) am not more assertive<br>
<INPUT TYPE=RADIO NAME=Q21 VALUE="t">(D) don't have close friends <br>
<p>
22. The bottom line is that reality is, in essence <br>
<INPUT TYPE=RADIO NAME=Q22 VALUE="f">(A) consensual<br>
<INPUT TYPE=RADIO NAME=Q22 VALUE="n">(B) paradoxical<br>
<INPUT TYPE=RADIO NAME=Q22 VALUE="s">(C) complex and predictable<br>
<p>
23. Generally, I prefer to<br>
<INPUT TYPE=RADIO NAME=Q23 VALUE="t">(A) deduce (draw conclusions) <br>
<INPUT TYPE=RADIO NAME=Q23 VALUE="s">(B) induce (generalize from particulars)<br>
<p>
24. In groups to which I have contributed most successfully, I have functioned as
the group's <br>
<INPUT TYPE=RADIO NAME=Q24 VALUE="f">(A) heart<br>
<INPUT TYPE=RADIO NAME=Q24 VALUE="t">(B) head<br>
<INPUT TYPE=RADIO NAME=Q24 VALUE="n">(C) soul or spirit<br>
<INPUT TYPE=RADIO NAME=Q24 VALUE="s">(D) backbone<br>
<p>
25. I'd rather <br>
<INPUT TYPE=RADIO NAME=Q25 VALUE="s">(A) classify<br>
<INPUT TYPE=RADIO NAME=Q25 VALUE="t">(B) analyze<br>
<INPUT TYPE=RADIO NAME=Q25 VALUE="n">(C) brainstorm<br>
<INPUT TYPE=RADIO NAME=Q25 VALUE="f">(D) connect<br>
<p>
26. Its a shame that others aren't able to be more <br>
<INPUT TYPE=RADIO NAME=Q26 VALUE="n">(A) playful<br>
<INPUT TYPE=RADIO NAME=Q26 VALUE="f">(B) considerate<br>
<p>
27. In life I look for what is <br>
<INPUT TYPE=RADIO NAME=Q27 VALUE="n">(A) meaningful<br>
<INPUT TYPE=RADIO NAME=Q27 VALUE="t">(B) true<br>
<INPUT TYPE=RADIO NAME=Q27 VALUE="f">(C) good<br>
<INPUT TYPE=RADIO NAME=Q27 VALUE="s">(D) real<br>
<p>
28. I most often seem drawn toward that which is <br>
<INPUT TYPE=RADIO NAME=Q28 VALUE="s">(A) elegant<br>
<INPUT TYPE=RADIO NAME=Q28 VALUE="n">(B) ineffable<br>
<INPUT TYPE=RADIO NAME=Q28 VALUE="t">(C) elite<br>
<INPUT TYPE=RADIO NAME=Q28 VALUE="f">(D) egalitarian<br>
<p>
29. In the past I have usually been more interested in <br>
<INPUT TYPE=RADIO NAME=Q29 VALUE="t">(A) effectiveness<br>
<INPUT TYPE=RADIO NAME=Q29 VALUE="n">(B) innovation<br>
<p>
30. Although I wouldn't brag about it, I am most proud of those occasions on which I have been<br>
<INPUT TYPE=RADIO NAME=Q30 VALUE="n">(A) profound<br>
<INPUT TYPE=RADIO NAME=Q30 VALUE="t">(B) brilliant<br>
<INPUT TYPE=RADIO NAME=Q30 VALUE="s">(C) exceptionally precise and thorough. <br>
<p>
31. People see me as <br>
<INPUT TYPE=RADIO NAME=Q31 VALUE="s">(A) efficient<br>
<INPUT TYPE=RADIO NAME=Q31 VALUE="f">(B) collaborative<br>
<INPUT TYPE=RADIO NAME=Q31 VALUE="t">(C) influential<br>
<p>
32. I have <br>
<INPUT TYPE=RADIO NAME=Q32 VALUE="s">(A) astute observational powers<br>
<INPUT TYPE=RADIO NAME=Q32 VALUE="t">(B) sound judgment<br>
<INPUT TYPE=RADIO NAME=Q32 VALUE="f">(C) a warm and kindly nature <br>
<INPUT TYPE=RADIO NAME=Q32 VALUE="n">(D) penetrating insight<br>
<p>
33. I feel most uncomfortable when I am not allowed to express my<br>
<INPUT TYPE=RADIO NAME=Q33 VALUE="n">(A) inquisitiveness<br>
<INPUT TYPE=RADIO NAME=Q33 VALUE="t">(B) decisiveness<br>
<INPUT TYPE=RADIO NAME=Q33 VALUE="s">(C) competencies<br>
<INPUT TYPE=RADIO NAME=Q33 VALUE="f">(D) concern for others<br>

 

[Ghost of the dead horse]

I agree with your first example, but not the second. Asking if I appreciate art is crystal clear: Yes, I appreciate art. It's all over my house and I go to galleries all the time. It has nothing to do with anyone except me. The same would be true if asked "Do you appreciate rap music?" My answer would be, "Hell no." It doesn't involve anyone except me.

You have a large sample size on the subject matter - art, and appreciation of it. There are some things we just know intrinsically, but for most of our knowledge, we need some outside data. Would you be capable of forming equally independent views of the subject if you had less experience on it? Before people have good experience on something, they first have limited experience, and that limited experience becomes their bias. For many people, that bias would include being overly reliant on opinions of people close to you.