One in a trillion unique personality profiles

Recently, one of Master's customers asked the following question: “What influence does the Barnum effect have in connection with your tests?”. This is not an easy question to answer and perhaps there is not only one answer. Nevertheless, the answer is of obvious relevance to our products and our field of work. In this insight article, we focus on the Barnum effect in connection with OPTO.

Section Overview:

1) Understanding the Barnum Effect

2) The Presence of Barnum Statements in OPTO Score Reports

3) Conclusion

Understanding the Barnum effect

In 1949 Bertram R. Forer, a university professor, initiated the scientific exploration of the Barnum effect. Forer administered a personality test to his students and one week later gave all students the same vague and ambiguous positive feedback on their personality using statements taken from a newsstand astrology book. Forer’s statements were great examples of Barnum statements and below we show the first three of them:

• You have a great need for other people to like and admire you.
• You have a tendency to be critical of yourself.
• You have a great deal of unused capacity which you have not turned to your advantage.

The students were then asked to rate the feedback on their personality on an accuracy scale going from 0 (poor accuracy) to 5 (perfect accuracy). The mean accuracy score was surprisingly high (4.3). This psychological phenomenon of people accepting vague positive personality profiles as their own has since been extensively researched and replicated and is now referred to as the Barnum effect (commemorating an American circus director from the late 19th century who made extensive use of the effect in his shows).

Dickson & Kelly (1985) review a lot of the early research on the Barnum effect and summarize their findings in the following way:

• General statements, like Barnum statements, apply to people in general. So perhaps they are accepted as fitting descriptions because they do fit?
• The vague wordings of Barnum statements allow people to project their interpretations into them and this might be why they accept them.
• Barnum statements involving socially desirable traits are accepted as more accurate.
• If people are told the Barnum statements were made for them, the statements are accepted as more accurate.

The Presence of Barnum Statements in OPTO Score Reports

The basic building blocks of the OPTO reports: Master’s personality test OPTO consists of 154 statements which a test-taker must either agree or disagree with at varying degrees (7-point Likert scale). An OPTO statement might read “I always double-check my work to ensure perfect quality” or “I take tasks too lightly” and the test-taker must choose a level of agreement. From a person’s answers to the 154 statements, an algorithm draws up an OPTO profile with scores on 20 Aspects and 8 Dimensions. The statements, Aspects, and Dimensions are the basic building blocks of the OPTO reports. These building blocks are used to generate a score-specific written personality profile, which then, together with STEN-scores for Aspects or Dimensions, makes up the personal OPTO report.

Are the 154 statements Barnum statements? Without the option of agreeing or disagreeing the statements might be Barnum statements. Recall though, that the Barnum effect entails test-takers accepting Barnum statements as reflections of their personality. At this basic level of measurement, test-takers are explicitly asked to not simply accept the statements but to actively accept or reject the statements. Therefore, these basic building blocks of OPTO are made in a way that makes them less likely to be Barnum statements and therefore oppose the Barnum effect. Furthermore, test-takers actually do vary their responses to the statements across the 7 options in the Likert scale which is the opposite of what would occur with Barnum statements.

Are the texts describing Aspect and Dimension scores Barnum statements? Through statistical analysis, the 154 statements are segregated into 20 different Aspects (on average 7 statements pr. Aspect), and the Aspects are further segregated into 8 Dimensions (on average 19 statements pr. Dimension). The creation of the Aspects and Dimensions is based on objective numerical analysis and the statements collected within an Aspect or a Dimension belong together because the level of agreement within the statements follows each other numerically. For each Aspect and Dimension, the test-taker will get a so-called STEN-score ranging from 1-10. STEN-scores are based on norms and therefore relay the degree to which the test-taker agrees with the 19 constituent statements relative to how other test-takers agree. But what about the text in the OPTO reports describing the 20 aspect scores and the 8 Dimension scores?

For each Aspect and Dimension, the score-text is generated in four nuances corresponding to scores of 1-3, 4-5, 6-7, and 8-10. A given STEN-score on a specific Aspect or Dimension will therefore result in one of four possible descriptions of what the score means in terms of personality. Let’s look at the four nuances in the Aspect named Risk Taking as a concrete example:

1) Will go a long way to avoid risk. Is less willing to take chances. Is a more cautious person.

2) Takes risks only when needed. Is willing to take some chances. Is cautions of not being reckless.

3) Willingly takes risks. Is enterprising. Is bold.

4) Is highly enterprising. Is adventurous. Is very bold.

 Are these texts Barnum statements? They are written in a neutral to positive tone because there is no wrong personality profile in the OPTO (personality is only evaluated as a relative fit to the job task or as a starting point for chosen development). To a certain degree, each of these four nuances of written personality descriptions is like Barnum statements (positively toned personality descriptions set in vague general terms). They must be because approx. one in four test-takers must be able to see themselves in them. But still, even at this level of detail, the four text descriptors of each Aspect or Dimension are not Barnum statements. They are derived through Pilot studies and through hundreds of working hours where psychologists have carefully worded the nuances of each of the four generic text descriptors according to the selection of underlying statements constituting each Aspect and Dimension. This ensures that they accurately reflect and cover four different levels of disagreement or agreement with the constituent statements. In this sense, they are not so general and are therefore not Barnum statements.

What about the full personality description collecting all the nuances in OPTO’s Expanded Score Report (presenting information at the Aspect level)? Because there are 20 Aspects with four possible nuances there is a possibility of a staggering 1 trillion unique written personality profiles with this report (4^20). OPTO’s shorter Score Report presents information at the Dimensional level. Because there are 8 Dimensions with four possible nuances there is a smaller, but still impressive possibility of 65.536 unique written personality profiles (4^8). Clearly one trillion descriptions cannot all be Barnum statements.

Conclusion

What influence does the Barnum effect have in connection with the OPTO? according to the logic and arguments presented above, only very little.

The 154 statements are not Barnum statements because test-takers are asked to actively accept or reject them and in doing so test-takers vary their responses to the statements across the 7 options in the Likert scale.

The written personality descriptions in four possible nuances pr. Aspect or Dimension are also not Barnum statements. But they might be called quasi-Barnum statements because they have similarities with Barnum statements. They are deliberately written in a neutral/positive tone and made sufficiently vague so that approx. one in four test-takers can see themselves in them. Nevertheless, they are directly derived from a given individual's disagreement or agreement with the constituent statements, and in this sense, they cannot be said to be Barnum statements.

Finally, the present analysis has provided us with a new and exciting way of looking at the OPTO: Every time a test-taker completes an OPTO the complete written personality profile from the Expanded Score Report is just one in a trillion unique possible profiles.

References:

Dickson, D. H., & Kelly, I. W. (1985). The ‘Barnum Effect’ in Personality Assessment: A Review of the Literature. Psychological Reports, 57(2), 367–382.

Forer, B. R. (1949). The fallacy of personal validation: A classroom demonstration of gullibility. The Journal of Abnormal and Social Psychology, 44(1), 118.