I would just like to take an opportunity to thank everyone for participating in providing us with some feedback. We had a number of people from this forum jump in and try out the assessment. I just wanted to sort of round if off by summarising some of the key themes. So here goes:
Image pair clarity and understanding - some of the images are down right confusing for some people. Others are hard to spot on a small screen. We will definitely look at these and enhance some of those more puzzling and confusing pairs.
Overestimation and accuracy - so far the report has been rated as 85% (rating of accurate or not accurate based on close to 400 responses) accurate based on the participants that have taken the time to provide feedback. However, just like with the above point it's very clear that certain items are not working and that specific personality factors need a bit more nuance especially agreeableness and neuroticism. We also need to create a transformation on the scores and use percentile ranking, so there will be an analysis to address these issues.
Length and fatigue - some people found the assessment a little long and tiring. We're looking to change this in two ways a) remove items that aren't contributing to the personality model by conducting a factor analysis and shortening the assessment to keep highly predictive items b) changing the design back to the original format where animations would be kicked off on hover or single tap on mobile so that you have an opportunity to study the stills first and then look at animations if necessary. We're guessing that the animation is introducing cognitive load that is really making you thinking about what choice to endorse. We're going to try to ease that a little.
Boredom - while many found the quiz engaging and have rated it very positively, some people reported the assessment to be boring. We will be adding more item types (e.g. card sorting task and a scaled type approach where there's more than 2 images to choose from, think Likert scale but with images) to enhance the dynamics of the assessment as well as robustness of the measure
Dreaded facebook login - ok we use this as integration point to be able to allow people to connect and compare, but it's clear that most people don't like facebook. That's cool
We'll be looking to allow for different types of account creations (e.g. other social media integrations or simple account setup) to help people get the most out of their report, compare with their friends and share their results
Report harshness - some commented on the fact that some of the personas are negative. Parts of our report are based on modified works of Dr. John A. Johnson's. Dr. Johnson's is outstanding, but probably not suited for a social sharing type scenario. They need to be updated for the context we're looking to use them in. We'll try to definitely refine the language so that they are more positive sounding. This may not be possible in all cases but we're committed to making it better. If you have any specific ideas about this please do not hesitate to message me here and I'll definitely look at incorporating your ideas.
Countries - comparison with countries was raised to be silly. Well we're not seeing any difference. Maybe most people are alike, but we're happy to try to see if we can add like a famous people type comparison group of fictional and non-fictional characters based on sourced information. Not sure on this one but just an idea.
Overestimation and accuracy - so far the report has been rated as 85% (rating of accurate or not accurate based on close to 400 responses) accurate based on the participants that have taken the time to provide feedback. However, just like with the above point it's very clear that certain items are not working and that specific personality factors need a bit more nuance especially agreeableness and neuroticism. We also need to create a transformation on the scores and use percentile ranking, so there will be an analysis to address these issues.
Length and fatigue - some people found the assessment a little long and tiring. We're looking to change this in two ways a) remove items that aren't contributing to the personality model by conducting a factor analysis and shortening the assessment to keep highly predictive items b) changing the design back to the original format where animations would be kicked off on hover or single tap on mobile so that you have an opportunity to study the stills first and then look at animations if necessary. We're guessing that the animation is introducing cognitive load that is really making you thinking about what choice to endorse. We're going to try to ease that a little.
Boredom - while many found the quiz engaging and have rated it very positively, some people reported the assessment to be boring. We will be adding more item types (e.g. card sorting task and a scaled type approach where there's more than 2 images to choose from, think Likert scale but with images) to enhance the dynamics of the assessment as well as robustness of the measure
Dreaded facebook login - ok we use this as integration point to be able to allow people to connect and compare, but it's clear that most people don't like facebook. That's cool
Report harshness - some commented on the fact that some of the personas are negative. Parts of our report are based on modified works of Dr. John A. Johnson's. Dr. Johnson's is outstanding, but probably not suited for a social sharing type scenario. They need to be updated for the context we're looking to use them in. We'll try to definitely refine the language so that they are more positive sounding. This may not be possible in all cases but we're committed to making it better. If you have any specific ideas about this please do not hesitate to message me here and I'll definitely look at incorporating your ideas.
Countries - comparison with countries was raised to be silly. Well we're not seeing any difference. Maybe most people are alike, but we're happy to try to see if we can add like a famous people type comparison group of fictional and non-fictional characters based on sourced information. Not sure on this one but just an idea.