Understanding Test Results
Updated
by
Amy Thomas
I administered two tests and they are providing conflicting results. Why?
When you administer two or more tests to candidates, occasionally a candidate will score well on one test and not as well on another test. While these results appear to be conflicting, they are simply measuring different things.
For example, a candidate could receive a high score on the WPP but simultaneously receive a low score on the CCAT. The Workplace Productivity Profile (WPP) is a personality assessment that is used to help predict whether an individual will be a conscientious, productive and reliable employee. The CCAT, however, measures cognitive aptitude, which includes logic and reasoning ability. Because these are separate abilities, candidates may score well on one and not the other.
Ideally you would want your candidate to score well on both assessments, because together the two tests assess two important but different factors that make for a successful employee in that position.
What does the icon with an exclamation point on a red triangle mean?
The icon with an exclamation point on a red triangle means that the test may be invalid. We flag invalid results to warn you when a test result may not be reliable based on the candidate's response patterns. Our system may flag a test as invalid for a number of reasons. Some personality tests may flag a result as invalid if the test taker is answering questions inconsistently, or if they seem to be answering too positively or too negatively about themselves. Some of our aptitude tests may be flagged as invalid if the test taker flies through the test too quickly, indicating that they may have seen the test before.
What can I do if my candidate has an invalid result in an assessment?
An invalid consistency score indicates that the candidate has responded to questions in an inconsistent manner (e.g., giving different responses to similarly worded questions). Whilst it is difficult to identify the exact cause for this, common scenarios include:
- The candidate has misunderstood the questions
- Being distracted when completing the assessment
- Not paying attention of responding carelessly (e.g., in a random fashion)
In any case, this means their results are not likely to be an accurate representation of their personality or attitudes and therefore should be interpreted with caution. We recommend reaching out to the candidate to let them know that you were unable to generate a valid assessment result for them, and ask if there was anything about their testing environment or experience that may have contributed to this. If the candidate is happy to do so, we would further recommend inviting them to recomplete the assessment and reminding them to complete the assessment in a distraction free environment and to read each question carefully. In the majority of cases, candidates will receive a valid result on their second attempt.
What is the difference between percentage and percentile?
A percentage refers to the amount out of a total. For example, if a candidate answers 30 out of 40 questions correctly, the percentage correct is 75%. The candidate answered 75% of the total questions correctly.
Percentile, on the other hand, refers to how a candidate scored in comparison to other people. A candidate may answer 75% of the questions correctly, but his or her percentile ranking may be 88th percentile. What this means is that the candidate scored better on the test than 88% of the other people who have taken the test.
Why are some of my results missing a percentile ranking?
Occasionally you might notice that some of your results do not show percentile rankings. For the EPP, for example, the job needs to be associated with a baseline that includes the EPP job match in order for a percentile ranking to automatically appear on the score report.
What does a low score on the WPP mean?
The WPP is a behavioral risk assessment that measures an individual's ability to follow rules. We generally recommend administering the WPP for positions for which rule adherence is particularly important, which includes jobs with a lot of routine and repetition.
We don't recommend the WPP for managerial or supervisory positions because these jobs require more abstract problem solving and outside-the-box thinking. A low score on the WPP is not an indication that someone is unethical or untrustworthy. The WPP is most useful for screening out low-scoring candidates applying for non-managerial roles.
We generally recommend that you look for both medium and high scores when administering the WPP to candidates.
How are the score ranges for different positions established?
The score ranges are established based on the test scores of a representative sample of people who perform well in those positions. The score ranges are simply suggestions that you can use to guide your decisions when setting your own baselines.
Why did my top performer do badly on an aptitude test?
There are a lot of reasons a top performer might not perform well on a test. Often when employees are asked to take an aptitude test at work, they may be interrupted or distracted by their regular job duties, which negatively impacts their scores. Another reason that top performers sometimes don't do well on tests is that they may be outliers. Outliers can happen within large samples of data even though the data is predictive of success overall. This is why we recommend looking at tests as one factor contributing to the hiring decision, as opposed to the sole deciding factor.
Can I send my candidate a copy of their score report?
How you decide to respond to candidates requesting their results or additional feedback about their performance, is up to you. We advise against sending candidates the Summary and Score reports, as these contain information to help you with your recruitment decision and some of the details may be confusing to candidates.
We recommend implementing a consistent approach to providing candidates with feedback. You may choose to withhold candidates' results completely or only provide the Workplace Insights Report. Another approach for candidates who reach out to request their results might be to provide them with the results category they achieved (e.g. average, above average etc), rather than an exact percentile score for each assessment.
Whatever process you decide to adopt, we recommend also advising your candidates that assessments form just one part of the overall selection process and that you will be reviewing multiple sources of information when making a decision.
