The importance of choosing the right norm

Maintaining high quality of our solutions involves updating norms based on the newest data reflecting the local population of the norm. But what is a norm? Why is it so important when we look at test results? And why can scores change when a norm is updated? This article will go into depth answering these questions.

What is a Norm?

A norm in psychometrical testing serves as a baseline for comparing an individual's test results to those of a larger group. Unlike absolute scores, psychological measurements often focus on relative scores, where the significance of a score is understood in comparison to others.

Consider a test designed to measure extroversion. If an individual scores 15 on this test, what does that indicate? Is the person extroverted? We cannot say!

The raw score alone doesn't provide a clear answer. To interpret this score, we need a norm group—a collection of individuals who have also taken the test. By comparing the individual's score to this group, we can derive meaningful insights.

For instance, if a norm group of at least 300 people has an average score of 11, a score of 15 suggests that the individual is more extroverted than the average person in that group. In Master's tools, raw scores are converted into standardized scales such as the STEN (Standard Ten) scale, using the mean and standard deviation of the norm group. This conversion facilitates the comparison of an individual's score to others using the same norm.

For a norm group to be representative, it must include a sufficient number of responses from a sample that reflects the target population's demographics, such as gender, age, and educational level. According to the European Federation of Psychologists' Associations (EFPA), a norm group with over 1,000 representative responses and less than 10 years old is considered excellent for high-stakes decisions like recruitment (Evers et al, 2013). To ensure representativeness, norm groups are weighted by population demographics. For example, if the working population in Denmark is 52% women, but only 49% of test responses come from women, their responses are given more weight to match the actual demographic distribution.

Why Are Norms Important?

Norms are essential for ensuring that an individual's responses are compared to a representative group, thereby mitigating unconscious discrimination. For example, comparing a man to a group consisting solely of women or comparing lower-income individuals only to high-income individuals can lead to biased results.

While test providers may not have local norms for every country, maintaining high-quality solutions requires a diversity of local norms and a truly international norm that includes a large, representative population. External agencies like the British Society of Psychology (BPS) and Det Norske Veritas (DNV) emphasize the importance of high-quality norms when reviewing and accrediting psychometrical solutions like Master's.

Why can scores change?

Several factors can impact and potentially change a candidate's score:

1. Norm Group selection: The interpretation of a score is always relative to the chosen norm group. For example, a raw score of 15 might be above average in a Finnish norm group, resulting in a STEN score of 6 and labelling the individual as extroverted. However, the same score might be below average in a Spanish norm group, resulting in a STEN score of 5 and labelling the individual as a little bit less extroverted. This discrepancy arises because the baseline for comparison changes with the norm group.

2. Norm Updates: When norms are updated using the most recent data to provide an accurate picture of the norm group, the scores might change slightly. Over time, demographic changes and significant life experiences can shift the average responses of a population. Regular updates, ensuring norms are never more than 10 years old, are crucial for maintaining accuracy (Evers et al, 2013).

3. Algorithmic improvements: The algorithms used to convert raw scores into standardized scales are based on complex models that are periodically reviewed and updated to reflect the latest research. 

In conclusion, the process of updating and ensuring the high quality of norms is ongoing. At Master International, the psychology team is dedicated to this process, proud of the diverse norms we offer, and committed to future updates. 

How to choose the right norm

When assessing candidate the choice of norms can significantly impact the candidate's result, and test users must therefore consider who is relevant to use for comparing candidates to. 

For instance, if a recruiter is looking for a a new team player to join a Spanish workplace, using a Spanish norm is relevant. This ensures that the candidate's traits, such as extroversion, are evaluated in the context of Spanish cultural norms, which may differ from other cultures. This also means, that the norm should not necessarily be chosen based on the culture of the candidate, but based on the culture of the workplace.

By choosing the right norm and keeping it up-to-date, we ensure fair and accurate psychological measurements, ultimately enhancing the quality of recruitment and other high-stakes decisions.

References:

Evers, A., Muñiz, J., Hagemeister, C. Høstmælingen, A., Lindley, P., and Sjöberg, A. (2013) EFPA Review model for the description and evaluation of psychological and educational tests. Version 4.2.6

Woods, R. T. and Clare, L. (edit.) (2015) Handbook of the Clinical Psychology of Ageing. John Wiley & Sons.