We built an innovative personality test, would love your feedback!

[Peter Deadpan]

I just wanted to know why you were hesitant to share your report?

Because it's an unflattering result, especially for someone of my general type. It reveals that I'm a static Ne dom and a sometimes insensitive/selfish person (the vagueness is a reflection of my general lack of connection or dedication to those immediately around me, specifically professionally).

Basically, it means my fire has dimmed, and I am still working to reignite it.

 

[FireBeard]

Because it's an unflattering result, especially for someone of my general type. It reveals that I'm a static Ne dom and a sometimes insensitive/selfish person (the vagueness is a reflection of my general lack of connection or dedication to those immediately around me, specifically professionally).

Basically, it means my fire has dimmed, and I am still working to reignite it.

Fair point.

 

[Turi]

A few things - this is not the 'new standard' in personality assessments - to claim such a thing, you need research behind it, and you're in the beta stage as it says in the link.
Therefore, this is not "redefining" anything at the moment, and so such a claim is misleading.

The idea is also not unique - on the website, it's claimed this is the "world's first image based personality tool" - have you heard of Buzzfeed?
I could link about a million "image based personality tools" - it's literally a selection on a number of free quiz-creator platforms.
It's also quite lengthy, though that's not an issue for me as I'm one of those guys who prefers to take his time with everything to ensure accuracy - I understand others have difficulty with this and prefer to be less than accurate in their decisions.

That said, I liked the artwork, and if we're speaking from a pure Big 5 perspective - I'm unaware of other tests that do this same thing, as the others I know of are MBTI-based, of "cognitive functions" based - though, it's likely they're out there - I've never searched for them.

I found the selections to be clear and intuitive - easy to understand, however there was one I simply disagreed with as it's basically just two selections representing the same result - and that is, one picture is of a guy happy that there is sunshine, and the other is a guy upset that there is rain.

This feels like it's the same option, just 'rephrased' - it would be more accurate to have one as perhaps upset that it is raining, and another happy that is raining - or something of the sort. As it is, there is no meaning here that I can discern - what's the point of that one particular selection?

 

[FireBeard]

A few things - this is not the 'new standard' in personality assessments - to claim such a thing, you need research behind it, and you're in the beta stage as it says in the link.
Therefore, this is not "redefining" anything at the moment, and so such a claim is misleading.

The idea is also not unique - on the website, it's claimed this is the "world's first image based personality tool" - have you heard of Buzzfeed?
I could link about a million "image based personality tools" - it's literally a selection on a number of free quiz-creator platforms.
It's also quite lengthy, though that's not an issue for me as I'm one of those guys who prefers to take his time with everything to ensure accuracy - I understand others have difficulty with this and prefer to be less than accurate in their decisions.

That said, I liked the artwork, and if we're speaking from a pure Big 5 perspective - I'm unaware of other tests that do this same thing, as the others I know of are MBTI-based, of "cognitive functions" based - though, it's likely they're out there - I've never searched for them.

I found the selections to be clear and intuitive - easy to understand, however there was one I simply disagreed with as it's basically just two selections representing the same result - and that is, one picture is of a guy happy that there is sunshine, and the other is a guy upset that there is rain.

This feels like it's the same option, just 'rephrased' - it would be more accurate to have one as perhaps upset that it is raining, and another happy that is raining - or something of the sort. As it is, there is no meaning here that I can discern - what's the point of that one particular selection?

Hi Turi thank you for your response. I guess I haven't released any of the psychometric work I am currently doing as I am gathering the data. I'm hopeful to be able to publish something soon.

I don't consider Buzzfeed or any other type of 'quiz' like that to be scientific. Our work was specifically developed on the Big 5. I am a psychometrician and take my work very seriously.

The claims on the site are aspirational at this stage, but the stats I have are indicating that the assessment is working. There are two other similar ventures out there Redbull Wingfinder and VisualDna, both of which use words and photographs. No assessment to my knowledge uses an animated style framework to assess personality. Perhaps, I need to be more careful in how it is written, I don't mean offence as I was trying to write some marketing copy. A more apt description might be that it is the 'first animated personality' tool.

With that particular image I was hoping to try to convey a sunny disposition vs a moody or grey disposition. That item will probably be dropped after analysis if it is performing badly. This is our first release and we're trying to grapple with how to be optimise these items so that they are understandable. So yes, that item had everything to do with mood and the weather was meant to be representative of the person's disposition.

Thank you for your feedback and the time you took to take the assessment. I'm not sure I'll be able to reduce it from 75 items down to have enough power, reliability and accuracy, but I will try once I do my analyses.

btw if you are interested in the findings I'm happy to publish them here once I have completed stats, I just require more participants right now to be able to dissect the data.

 

[Turi]

[MENTION=37312]FireBeard[/MENTION] - regarding the wording - not everyone is going to be as pedantic about "the truth" as I am - it got to page three, after all.

With that specific image - again, both instances depict the same thing:

• Sunny and happy.
• Rainy and upset.

It's the same end-result, they're both attributing happiness to sunshine, and therefore unhappiness to "not" sunshine.

 

[FireBeard]

Yeah the weather in that sense is not literal. It's meant to be metaphors for mood, but I can also see how it is confusing. I'll pay particular attention to that one to see if it is related to a personality factor or predictive, even then I might rework it.

 

[Abcdenfp]

impulsive persona
i like the concept , maybe doesnt need to be so long and also some of the pictures its not clear what is different in each senario but i like it!!
also the persona names could be tweaked

 

[Luminous]

also the persona names could be tweaked

Yes, many of them sound rather insulting.

 

[FireBeard]

Yes, many of them sound rather insulting.

Hi [MENTION=33711]Abcdenfp[/MENTION] and [MENTION=35566]Luminous[/MENTION] the reporting is based on the theoretical work of Dr. John A. Johnson and I did really consider whether they were appropriate or not. Some of them do seem insensitive, so I might look at how to adapt it to be less offensive, but it is really hard to find replacement words that describe a particular pattern of behaviour. I've tried to soften them as much as possible.

Man it's really awesome to hear this feedback!

 

[Stigmata]

Satisfied Persona
Unassuming Unexcitable Placid Tranquil
Satisfied Personas feel content with things as they are. They see little point in getting involved in a rat-race to struggle to the top of the heap. They see stability and security as more important than getting ahead and are likely to be content with a respectable job, earning enough money to make a living and living a 'regular' life. They are described by others with terms such as even tempered, responsible, unexcitable, and unassuming.

 

[Abcdenfp]

Hi [MENTION=33711]Abcdenfp[/MENTION] and [MENTION=35566]Luminous[/MENTION] the reporting is based on the theoretical work of Dr. John A. Johnson and I did really consider whether they were appropriate or not. Some of them do seem insensitive, so I might look at how to adapt it to be less offensive, but it is really hard to find replacement words that describe a particular pattern of behaviour. I've tried to soften them as much as possible. Man it's really awesome to hear this feedback!

i have thought to myself many times what a rich source of information Typeology central is for someone looking to expand on any type of personality concepts. Its wonderful to see someone interested in using it for accuracy when it comes to this kind of testing in any kind of application

 

[Bush]

I like the approach. It's more immersive and so has the potential to be more relateable. Images speak loudly. You're also in a real sense providing scenarios, which hardly any test does -- most tests just roll with "Are you a social person?" or "Do you like books?"

Some of the individual images could be tweaked for more "intuitiveness" though -- if I'm to choose which picture I relate to, I need an understanding of what the pictures are about. ("Am.. am I stealing this guy's bike in this picture..?") The exception is if the interpretation is part of how you're evaluating someone (see: Rorschach's inkblots), but I don't think that's what you're going for here.

Openness
Your score on Openness to Experience is average, indicating you enjoy tradition but are willing to try new things. Your thinking is neither simple nor complex. You tend to balance pragmatism vs exploring new ways or 'outside of the box' thinking.

Conscientiousness
Your score on Conscientiousness is average. This means you are reasonably reliable, organized, and self-controlled.

Extraversion
Your score on Extraversion is average, indicating you are neither completely subdued nor a jovial chatterbox. You enjoy time with others but also time alone.

Agreeableness
Your level of Agreeableness is average, indicating you tend to balance concern for the needs of others and your own.

Neuroticism
Your score on Neuroticism is average, indicating that your level of emotional reactivity is typical. Stressful and frustrating situations are somewhat upsetting to you, but you are generally able to get over these feelings, move on and cope.​

(I sense a common theme here)

Pleasant Persona
Generous Pleasant Tolerant Peaceful Flexible
Pleasant Personas are full of positive emotions and free from negative emotions. They are almost universally liked. They are described by others as confident, cheerful, relaxed, tolerant, composed, calm, good natured, warm, trustworthy, empathic, cooperative, and down-to-earth.​

This is a result that I could see being true. For a typology to be useful IMO, there have to be types in the roster that scream "yeah this is definitely not me," lest you get that dreaded Forer effect. That's not a criticism, that's just how I roll.

I could see the career aspects working well when you collect more data -- an optional online survey afterward asking for career, age, etc is an easy way to get there. (If I have to plug through it in order to see my results, I'll probably just X out the tab and roll my eyes.)

Giving a link to a description of all of your types, and maybe a "Does this fit you? If not, consider [this] or [this]..." if results are close, would be useful as well. (Plus it's a CYA accountability measure in the case where someone would say "no this got me completely wrong")


In short, you've got a good thing here; at least a good solid framework on which you can build something pretty cool with a bit of tweaking and data collection.

I don't consider Buzzfeed or any other type of 'quiz' like that to be scientific.

Even if not, participants probably won't take that into account -- or know what you mean, or differentiate your tests from 'unscientific' ones -- when they see the claim that you're the first. I know that I didn't. You might be able to claim the first 'something,' though. Even if it's "first Big Five-based Image quiz" or something.

 

[FireBeard]

I like the approach. It's more immersive and so has the potential to be more relateable. Images speak loudly. You're also in a real sense providing scenarios, which hardly any test does -- most tests just roll with "Are you a social person?" or "Do you like books?"

Some of the individual images could be tweaked for more "intuitiveness" though -- if I'm to choose which picture I relate to, I need an understanding of what the pictures are about. ("Am.. am I stealing this guy's bike in this picture..?") The exception is if the interpretation is part of how you're evaluating someone (see: Rorschach's inkblots), but I don't think that's what you're going for here.

Openness
Your score on Openness to Experience is average, indicating you enjoy tradition but are willing to try new things. Your thinking is neither simple nor complex. You tend to balance pragmatism vs exploring new ways or 'outside of the box' thinking.

Conscientiousness
Your score on Conscientiousness is average. This means you are reasonably reliable, organized, and self-controlled.

Extraversion
Your score on Extraversion is average, indicating you are neither completely subdued nor a jovial chatterbox. You enjoy time with others but also time alone.

Agreeableness
Your level of Agreeableness is average, indicating you tend to balance concern for the needs of others and your own.

Neuroticism
Your score on Neuroticism is average, indicating that your level of emotional reactivity is typical. Stressful and frustrating situations are somewhat upsetting to you, but you are generally able to get over these feelings, move on and cope.​

(I sense a common theme here)

Pleasant Persona
Generous Pleasant Tolerant Peaceful Flexible
Pleasant Personas are full of positive emotions and free from negative emotions. They are almost universally liked. They are described by others as confident, cheerful, relaxed, tolerant, composed, calm, good natured, warm, trustworthy, empathic, cooperative, and down-to-earth.​

This is a result that I could see being true. For a typology to be useful IMO, there have to be types in the roster that scream "yeah this is definitely not me," lest you get that dreaded Forer effect. That's not a criticism, that's just how I roll.

I could see the career aspects working well when you collect more data -- an optional online survey afterward asking for career, age, etc is an easy way to get there. (If I have to plug through it in order to see my results, I'll probably just X out the tab and roll my eyes.)

Giving a link to a description of all of your types, and maybe a "Does this fit you? If not, consider [this] or [this]..." if results are close, would be useful as well. (Plus it's a CYA accountability measure in the case where someone would say "no this got me completely wrong")


In short, you've got a good thing here; at least a good solid framework on which you can build something pretty cool with a bit of tweaking and data collection.

Even if not, participants probably won't take that into account -- or know what you mean, or differentiate your tests from 'unscientific' ones -- when they see the claim that you're the first. I know that I didn't. You might be able to claim the first 'something,' though. Even if it's "first Big Five-based Image quiz" or something.

How to respond This is such detailed feedback!

We will definitely tweak the individual images once the stats reveal which ones are working and which ones are not. For me it's to hard to step outside of my mind right now and understand which ones do really work and which ones do not. Or ideally how some people are misinterpreting them compared to others. I wanted to run a study where basically all the image pairs are presented and then people with a brief sentence describe what is happening in that scenario. Based on the consensus and similarity of descriptions the items would then be kept, so that bad items where there are many different interpretations are then dropped. That's definitely the next step in item development and would fix the problem you raise nicely!

I like your idea of letting people self identify which 'persona' they fit more if they disagree with the result because at the moment we only allow people to say whether the report is accurate or not. One of the other issues that aligns with this is that the scores are not being transformed as the results haven't been normed yet. Once I do the work there's a likelihood that the results could change and become more representative. At the moment the 2 most extreme traits from the big 5 determine the persona type. This is based on the work on Dr. John. A Johnson which is available on github. We essentially plugged in our methodology and extended some of his reporting capability which is in the public domain. There's around 48 different personas on the roster. Fine tuning the scoring will definitely make it more worthwhile. So I really appreciate your suggestion here as I haven't considered that self report type identification to see whether the persons perception of themselves matches what they received. I mean there's obvious short-comings with this in that sometimes people see themselves as something they're not, but it would be interesting nonetheless!

We collect the demographic information up front and most people are filling it in. One very interesting fact is that people choose to complete the measure 4:1 anonymously rather than logging in through social media. To me this is very fascinating!

Again thank you so much, I have noted your ideas on improvement and I really thank you for your constructive feedback it's honestly super useful!

I can't wait to share some of the statistical findings with everyone on this thread. I'll probably have enough data in the next week to start a principal component analysis, item analysis and basic demographic stuff. Right now the means and spread of scores are very much in line with other validated 5 factor model measures of personality. The other thing I need to invest time in is getting people to complete another Big 5 measure like the 50 item IPIP and correlate their result across both measures.

I might ask for that in future here. People have honestly really been nothing more than helpful on this forum with their suggestions and feedback!!!

 

[stupidgirl]

Humble Personas are peace-loving, somewhat timid, and seek social acceptance by going along with what others want. These individuals are described by others with terms such as calm, agreeable, cooperative, composed, warm, preserving, and submissive.

Openness (41)
Your score on Openness to Experience is average, indicating you enjoy tradition but are willing to try new things. Your thinking is neither simple nor complex. You tend to balance pragmatism vs exploring new ways or 'outside of the box' thinking.

Conscientiousness (47)
Your score on Conscientiousness is average. This means you are reasonably reliable, organized, and self-controlled

Extraversion (37)
Your score on Extraversion is average, indicating you are neither completely subdued nor a jovial chatterbox. You enjoy time with others but also time alone.

Agreeableness (66)
Your level of Agreeableness is average, indicating you tend to balance concern for the needs of others and your own.

Neuroticism (57)
Your score on Neuroticism is average, indicating that your level of emotional reactivity is typical. Stressful and frustrating situations are somewhat upsetting to you, but you are generally able to get over these feelings, move on and cope.

 

[FireBeard]

Humble Personas are peace-loving, somewhat timid, and seek social acceptance by going along with what others want. These individuals are described by others with terms such as calm, agreeable, cooperative, composed, warm, preserving, and submissive.

Openness (41)
Your score on Openness to Experience is average, indicating you enjoy tradition but are willing to try new things. Your thinking is neither simple nor complex. You tend to balance pragmatism vs exploring new ways or 'outside of the box' thinking.

Conscientiousness (47)
Your score on Conscientiousness is average. This means you are reasonably reliable, organized, and self-controlled

Extraversion (37)
Your score on Extraversion is average, indicating you are neither completely subdued nor a jovial chatterbox. You enjoy time with others but also time alone.

Agreeableness (66)
Your level of Agreeableness is average, indicating you tend to balance concern for the needs of others and your own.

Neuroticism (57)
Your score on Neuroticism is average, indicating that your level of emotional reactivity is typical. Stressful and frustrating situations are somewhat upsetting to you, but you are generally able to get over these feelings, move on and cope.

Thank you for completing the test.

 

[Obfuscate]

Hi Obfuscate, interesting that you have quotes in your signature and you got the bookworm persona. I'd be really interested in your feedback, but in the simplest terms thank you for doing the assessment!

simple, okay... overall it is pretty good for what it is... i was annoyed that the generalized summary of "bookworm" called me rule abiding when my responses on the law based pictures would indicate the opposite (speeding, etc)... i know that descriptions like that use very broad strokes, but it was still annoying... most any other complaint i might have would be on a similar vein... i recall there was one pair that i saw little to no difference in... i would need to retake it to know which... maybe i'll do that and show it to you... the percentages were a bit higher or lower than my estimation, but the conclusion seemed fine...

 

[FireBeard]

simple, okay... overall it is pretty good for what it is... i was annoyed that the generalized summary of "bookworm" called me rule abiding when my responses on the law based pictures would indicate the opposite (speeding, etc)... i know that descriptions like that use very broad strokes, but it was still annoying... most any other complaint i might have would be on a similar vein... i recall there was one pair that i saw little to no difference in... i would need to retake it to know which... maybe i'll do that and show it to you... the percentages were a bit higher or lower than my estimation, but the conclusion seemed fine...

Sure I guess one thing to understand is that it's based on a range of choices. Also from the overall higher order factor of Conscientiousness and Neuroticism, you have subfacets that touch on orderliness and compliance. Some of these images are classified under subfacets that contribute to a higher order personality dimension/factor.

One of the really cool things that I want to start looking at is whether each of the items should be weighted the same, or whether specific decisions or combinations of decisions should contribute more to higher order factors. This can get complex quite quickly, but I can understand your frustration in trying to guess the scoring mechanism. The reason there are a larger amount of items is to help deal with some of the error that individual choices might throw up.

Thanks for that feedback, I'll definitely look into it and try to deal with some of those images that specifically have just the happy smile vs sad face. One that comes to mind is the lady walking through the park. On mobile this is probably very hard to notice!

edit: I forgot to include a visual chart of what I was referrencing https://www.mindtools.com/media/Diagrams/BigFivePersonalityTraits_v2.jpg

 

[Norexan]

 

[FireBeard]

Thank you Norexan, would you say that's in line with your other test scores?

 

[Norexan]

Thank you Norexan, would you say that's in line with your other test scores?

Conscientious and Openess should swap place but yeah Ambivert RCOEI