The candidate left the room feeling they had kept up the pace: the multitasking section went well, working memory too, and the verbal feedback spoke of a result well above average. Three weeks later the answer is negative and explains nothing. The misreading is almost always the same: a score is read as a mark, when it is only a position within a group, set against a profile written before the test and cut into thresholds that do not compensate for one another.
A score is not a mark, it is a position
A selection battery produces no absolute mark. It produces a raw score — number of correct answers, mean response time, deviation from the instruction, number of omissions — that nobody uses as it stands. That raw figure is converted into a normed score: a percentile, a rating on a ten-class scale, a standard score centred on a conventional mean. The conversion says one thing only, and that is already a great deal: where this candidate stands relative to a reference group. Change the group and the raw figure does not move, while the reported result changes completely.
Three norm sets are commonly in circulation. The first compares the candidate with the general adult population, and it flatters just about anyone who has turned up for an aviation selection. The second compares them with everyone who has already sat that same selection: the pool is self-selected, often prepared, and the same raw score becomes median there. The third compares them with pilots already qualified and flying the line, a group more tightly grouped towards the top still. Verbal feedback announcing “above average” without naming the norm therefore says very little.
Two parameters sit behind the norm and sometimes weigh as much as it does: the age of the norming — a norm built long ago on a different medium, describing a group that no longer exists — and the conditions of administration: language of the test, equipment, order of the subtests. The battery has to be re-validated whenever one of these changes, an operation that is invisible to the candidate.
A psychometric test doesn’t give a simple pass/fail grade. It compares your performance to a group of other people, and that comparison can change completely depending on who that group is. A “good” score in one group might be average in another.
An A320 pilot candidate scores 85 on a psychomotor test. Against the general population, this puts them in the 95th percentile. But against other candidates who have taken the same test (often prepared candidates), their percentile drops to 65th. Against active pilots, it might only be 40th. The raw score is the same, but the interpretation changes completely.
The target is a band, not a ceiling
Before the first test session, somebody wrote down what the job requires. This job analysis lists dimensions — processing information under time pressure, working memory, resistance to interference, but also rule awareness, cooperation, tolerance of uncertainty — and attaches to each of them not a maximum to be reached, but an expected range. This is the point most candidates miss, because it contradicts their entire school experience, where higher is always better.
On the cognitive dimensions the relationship remains roughly monotonic: fast and accurate beats slow and accurate. On the behavioural dimensions it no longer is. High assertiveness supports decision-making up to the point where it silences the other pilot’s contribution. A high concern for detail serves procedural rigour up to the point where it locks the crew onto a minor anomaly while the flight path drifts. Each of these scales has a bottom and a top, and the top is disqualifying for the very human-factors reasons that led to the writing of the crew competency frameworks.
Personality questionnaires additionally carry control scales: consistency of answers across equivalent items, tendency to present oneself in a favourable light, systematic extreme responses. These do not measure a quality, they measure how credible the rest of the protocol is. A perfectly smooth profile, high everywhere, matching the idea people have of the ideal pilot, is the most ordinary signal that the questionnaire was filled in as an exercise in communication.
Aviation selection isn’t about being the best at everything. It’s about being in the right range on all important skills. Too much of a “good” trait (like being overly assertive) can be just as bad as too little.
A candidate for a Boeing 737 first officer position scores 95% on assertiveness but only 40% on workload management. While their assertiveness score is impressive, their low workload management score means they wouldn’t be selected, as they might dominate the cockpit but fail to manage critical flight tasks under pressure.
The trap. Answering the personality questionnaire “the way a good pilot would answer” produces precisely the profile that the validity scales are designed to spot: too favourable, too consistent, without a rough edge. The application is then set aside not on one dimension but on the credibility of the whole, and that reason is not put right by sitting the battery again in the same way.
Adding up or clearing the bar: two arithmetics, two verdicts
Once the scores have been normed, they have to be aggregated. Three logics coexist and do not give the same answer on the same application. The compensatory model computes a weighted sum: a very high peak buys back a dip elsewhere, and this is the one candidates spontaneously imagine. The conjunctive model sets a threshold on every dimension judged critical and requires all of them to be cleared: no peak buys back anything. The successive-hurdles model places the tests in series, each one eliminating.
In aviation selection, the arrangement most often encountered combines the first two logics: a few dimensions handled as hard thresholds because they condition the safety of training — multitasking, ability to divide attention, holding up under load — and the rest aggregated in compensatory fashion to rank the remaining candidates. This architecture explains the letter that makes no sense: the candidate was ruled out by a threshold, on a dimension they did not know was critical, even though their overall average placed them very high.
| Model | Rule applied | What a very high peak contributes | What a weak point costs | Usual use |
|---|---|---|---|---|
| Compensatory | Weighted sum of the dimensions, compared with a total | It raises the total and can carry the application through | It is absorbed by the rest if it is not too pronounced | Final ranking among candidates already eligible |
| Conjunctive | One threshold per critical dimension, all to be cleared | Nothing beyond the threshold of its own dimension | It eliminates, whatever the level of the others | Dimensions tied to the safety of training |
| Successive hurdles | Tests in series, each one eliminating | It does not carry over to the next stage | It stops the process at the stage where it appears | Selections with a high volume of applications |
Aviation selection uses different rules for combining test results. Some systems let you compensate for a weak area with a strong one, while others require you to meet every minimum standard. If you fail one critical test, you might be rejected even with high scores elsewhere.
A candidate for an Airbus A350 position scores 90% on multitasking (a critical dimension) but only 60% on workload management. In a conjunctive system, they would be rejected despite their high multitasking score. In a compensatory system, their high score might compensate for the lower one, but only if workload management isn’t a hard threshold.
What the overall score hides: pace, errors, stability
The result of a psychomotor or attention test does not reduce well to a single figure, because it aggregates three independent quantities: response rate, error rate — to be separated into omissions and wrong answers, which say different things about the strategy used — and stability. The same person can hold a high level over the first few minutes of a test and then degrade, something the average hides entirely.
The trade-off between speed and accuracy can be adjusted, and that regulation is precisely what is being measured. A candidate who pushes the pace to the maximum obtains an impressive throughput figure and an error rate that crosses the permitted limit; another, slightly slower, stays under the limit and will be preferred. Neither has “done better”: they have adopted two different regulations, and only one is compatible with the job. The same logic applies to drift at the end of the battery, which says far more about holding up under load than a peak score does.
Your test score isn’t just about how many answers you get right. It also looks at how fast you work, how many mistakes you make, and whether you can keep performing well over time. Just going fast might actually hurt your score if you make too many errors.
On a psychomotor test for a Boeing 787 first officer position, Candidate A completes 120 items with 15 errors (87.5% accuracy) while Candidate B completes 100 items with 5 errors (95% accuracy). Even though Candidate A completed more items, their higher error rate means they might be rejected in favor of Candidate B, who maintained better accuracy.
The right reflex. Treat the accuracy instruction as an operating limitation: you do not exceed it to save time. In practice sessions, set your pace by the error rate you observe, not by the number of items completed. If the test announces that errors are penalised, it has already given you the aggregation rule.
The battery does not decide on its own
In a properly built selection process, the psychometric result is not a verdict but a set of hypotheses to be checked. The structured interview takes up the dimensions that left the band and looks for facts: a situation where the workload rose, what was delegated, what was dropped, how a disagreement was handled. Group exercises and simulated situations serve the same function, with behavioural markers observed by several independent assessors and consolidated afterwards.
This is where part of the rejections that seem to contradict the scores is settled. A profile declared cooperative that the interview backs up with no verifiable example becomes a contradiction, not a neutral asset. Conversely, an average score together with a precise, dated operational account, in which the person relates what they misjudged and what they drew from it, weighs more than a high percentile. The documentation in the file acts as a factual counterweight to everything the battery merely suggests. Exchanges between candidates on the FL330 forum in fact describe rejections at this stage far more often than at the testing stage.
A score carries its own margin of error
Every psychometric measurement has a reliability below one, and therefore a standard error of measurement. In concrete terms, a score is not a point but an interval: sitting the test again the next day, the same person would obtain a slightly different result without anything having changed in them. That margin is known to the publisher of the battery and normally appears in its technical manual. It explains the grey zone around the threshold: two candidates whose scores differ by less than that margin cannot be distinguished by the measurement, and an operator who knows this then decides on other elements rather than making the figure say what it does not say.
Two practical consequences follow. The first is the retest policy: the waiting period imposed is not administrative, it targets the practice effect, which inflates scores without improving the aptitude measured at all and contaminates the comparison with the norm. The second is portability: a result obtained with one operator does not hold with another, because neither the battery, nor the norm, nor the target profile, nor the weightings are the same. A rejection travels no better than a success.
There remains the factor nobody writes into a feedback report: the number of vacancies. When the candidates clearing every threshold outnumber the places, the selection shifts from a threshold regime to a ranking regime, and the level of the last candidate admitted then varies from one session to the next without any threshold having moved. This is not an injustice, it is a change in the stopping rule, and it is rarely communicated as such.
Test scores aren’t exact measurements. There’s always some margin of error, meaning your “true” score could be a bit higher or lower than what you actually got. If your score is very close to a passing threshold, the test might not be able to tell for sure whether you passed or failed.
On a cognitive test for an Embraer E-Jet first officer position, the passing threshold is 70. Candidate X scores 69, Candidate Y scores 71. If the test’s margin of error is ±2 points, Candidate X’s “true” score could actually be 71 (passing) and Candidate Y’s could be 69 (failing). In this case, the selection committee would look at other factors to make their decision.
What to ask before the next session
A few questions asked at the right moment are worth more than an appeal. Before registering, ask whether an individual feedback report is provided, what retest interval applies, in what language and on what equipment the tests are run. After a rejection, ask whether the decision came from one dimension in particular or from the overall aggregation: the answer is not always given, but the question is legitimate, and a decision resulting from entirely automated processing opens, in Europe, under data protection law, a right to information and the possibility of requesting human intervention.

For preparation, the dividing line is clear. Becoming familiar with the formats — instructions, interfaces, time constraints — removes needless noise. Running hundreds of repetitions of the same test moves the score without moving the aptitude, and it backfires if the next test comes from another publisher. What really can be worked on lies upstream: the regulation between speed and accuracy, sustaining attention over a long period, and above all interview material, that is, real situations one is able to recount along with what one got wrong in them. The programmes offered on formations.fl330.com cover that part of the preparation, the part that is not played out in front of a screen.
A grey area remains and needs naming: nothing obliges an operator to publish its target profile, its weightings or its thresholds, and the confidentiality of the scoring keys is even a condition of their validity. The candidate is therefore working on a system they will never see in full. This opacity is also why relentless efforts to decode the test are a poor investment, whereas the quality of the application file and the precision of the professional account remain entirely under control.
Each operator’s selection system, the technical manual of the battery used and the part of the operations manual devoted to training take precedence over everything described here: this article sets out general mechanisms, not the procedure of any given employer.
Questions fréquentes
Does a high percentile announced verbally guarantee eligibility?
No, because a percentile only makes sense in relation to the reference group used for the norming. The same raw score can be a tail of the distribution against a general population and a median result against the pool of candidates who have already sat that selection. As long as the norm is not named, the figure allows no conclusion about the application’s real standing.
Can you be ruled out for a score that is too high on one dimension?
Yes, on the behavioural dimensions, whose expected profile is a range bounded above as well as below. Assertiveness or concern for detail far above the band reads as a risk for crew work: the first can silence the other pilot’s contribution, the second can lock the crew onto a minor anomaly. On the purely cognitive dimensions, by contrast, a high level is never penalising in itself.
Why did an excellent overall result not make up for a single poor test?
Because the selection process probably does not apply a compensatory rule on that dimension. Capacities judged critical for the safety of training, such as divided attention or holding up under load, are often handled by a threshold to be cleared independently of the rest. Under this conjunctive logic, no peak obtained elsewhere buys back the threshold that was missed, and the overall average only serves to rank candidates who are already eligible.
Does practising on online test simulators improve your chances?
Familiarisation with the formats, the instructions and the time constraints removes needless noise and is justified. Intensive repetition of the same exercise, on the other hand, moves the score without moving the aptitude measured and distorts the comparison with the norm, since the reference group did not have the benefit of the same practice. The benefit is lost in any case as soon as the next test comes from another publisher, whereas the return remains high on the regulation between speed and accuracy and on preparing interview material.
Is a result obtained with one operator usable with another?
As a general rule, no. The battery, the norming, the target profile for the job and the weightings differ from one employer to another, and a selection system is only valid for the job it was built for. A rejection travels no better than a success, which explains why candidates turned down on one side are taken on the other without anything having changed in their aptitudes.
Why is a waiting period imposed before retaking the tests?
The waiting period targets the practice effect: taken again after a short interval, the same test gives a higher score simply through familiarisation, with no improvement in aptitude. That artificial rise would make the comparison with the norm wrong and the decision less reliable. The waiting period therefore protects the validity of the measurement, not the organisation of recruitment.

Leave a Reply
You must be logged in to post a comment.