Human factors in certification: how a cockpit alerting philosophy is judged

Maquette de banc d'un panneau d'alerte de poste de pilotage, faisceau de câbles sortant à l'arrière

Certification simulator evaluation session: engine failure shortly after rotation, initial climb on one engine, reconfiguration. The crew calls out the alarm, silences the aural repeat, applies the procedure — and misses an amber message that appeared once the inhibition lifted, which they will not discover until cruise. At the debrief, nobody talks about a piloting error: the inhibition logic and the display order are reopened instead. Treating a trained crew’s error as a property of the design rather than a human failing is exactly what human factors requirements demand during type certification.

What certification requires of an interface

The certification specifications applicable to transport aircraft do not require a cockpit to be pleasant, or even modern. They require three things, and the order matters. First, that the minimum flight crew provided for can accomplish, with average skill and attention, the full set of tasks the design assigns to it, without exceptional skill or effort. Second, that the design does not make error probable. Third, that errors occurring despite all this be detectable by the crew and recoverable before the consequence becomes unacceptable.

Certification checks if a cockpit works for average pilots, not just experts. It must prevent mistakes and help crews fix any that happen.

This third requirement is the least intuitive and the most structuring. It acknowledges that no interface eliminates human error, and it shifts the question: not « did the crew make a mistake? » but « did the design anticipate that someone would make a mistake, and what did it put in place to meet it? ». A submission claiming error is impossible would go nowhere; a submission that names the plausible errors, classifies them, and shows a barrier against each one is acceptable, even if the list is long.

On an A320, if a pilot misses an amber warning due to inhibition logic, the design must ensure the crew can catch it later or recover safely. The system must account for this possibility.

Design assumptions, the founding act of the submission

Before any evaluation, the applicant writes down what it assumes about the user: what licence, what prior experience, what transition training, what procedures, what operating environment, what usual level of automation. These assumptions are not decorative — they set the boundary of the demonstration. An interface can be perfectly compliant under the assumption « crew freshly qualified on type, in multi-crew operation » and prove nothing at all once taken outside that frame.

The depth of the evaluation is then set by a few characteristics of the function: its novelty relative to what already flies, its complexity, its degree of integration with the rest of the cockpit, its criticality, and the amount of subjective judgement it leaves to the crew. A familiar, isolated, low-criticality control is handled by a reasoned comparison with existing designs. A new, integrated, critical function calls for crewed sessions, on means increasingly close to the real aircraft.

The alerting philosophy is a grammar, not a catalogue

The practitioner’s reflex, faced with an alerting system, is to think about the messages. The certification authority looks first at the grammar that governs them, and this grammar is written before the messages themselves: how many levels, what response each level imposes, what attention-getting device is reserved for it, in which flight phases it is inhibited, whether it is latched, how it clears and is recalled, and above all how two simultaneous conditions are arbitrated.

It is this grammar that is the subject of a philosophy document, and it is against this document that every message is subsequently checked. The question asked is almost never « is this message well worded » but « is this message at the right level, given what the philosophy has declared ». A message classified too high consumes attention at a moment when it is scarce; a message classified too low will not be dealt with when it should be. Both faults have a cost, and the second one is paid later.

Placement of the three alert levels according to available time and attention demandedTwo axes: on the horizontal axis, time available before the consequence; on the vertical axis, attention demanded of the crew. Three blocks are arranged diagonally along a dashed line. At top left, the alarm, short time and immediate action. In the centre, caution, awareness then action. At bottom right, advisory, long time and simple awareness. No boundary is drawn between the blocks: it is set by the declared philosophy.attention demanded of the crewtime available before the consequenceWarningimmediate actionCautionawareness then actionAdvisoryawareness only
The level of an alert is derived from the response it demands, not from the severity of the failure. The shorter the time available before the consequence, the more attention the alert must impose. The boundary between two blocks is nothing natural: it is the declared philosophy that places it, and each message is then judged consistent with it or not.
The three alert levels compared on what actually defines them
Level Response required of the crew Attention-getter Treatment in critical phase
Warning Immediate action, ahead of any other task in progress Red signalling, continuous tone or voice message, dedicated master light Retained, except in limited cases explicitly declared in the philosophy
Caution Immediate awareness, action afterwards, at a time chosen by the crew Amber signalling, brief tone, master light distinct from the warning one Frequently inhibited during the busiest phases, then restored
Advisory Awareness only, no immediate action expected Display only, with no dedicated tone or master light Inhibited, with no systematic restoration once the phase has passed

Two principles recur across accepted philosophies. The first: no essential information relies on colour alone. Red and amber code urgency, but the wording, the rank in the list and the aural device also carry it — otherwise a colour-vision deficiency, unfavourable lighting, or a simple reflection would be enough to cancel the message. The second, widely followed: at rest, the cockpit tends toward dark and quiet. Any light on in cruise is information; a cockpit that permanently displays « normal » indications has lost the power to catch the eye.

From this comes a criterion that is underestimated: the number of alerts is itself a design parameter. A system that trips often for nothing teaches the crew to silence before reading. Nuisance and false alerts are not annoyances to be dealt with later, in service; they degrade the detection barrier that the certification file credited to the system.

Design-induced error

« Design-induced error » is not a polite way of saying « piloting error ». It is a technical category: an error whose probability is increased by an identifiable characteristic of the interface, and which can therefore be named before it occurs. The important word is identifiable — if the design trait that favours it cannot be shown, the error no longer belongs to this category and the argument shifts to training.

The families are known and recur from one type to another. Similarity, first: two adjacent controls of comparable shape, or two messages whose wording differs by only one word, while the expected action differs entirely. Mode change next, whether spontaneous or commanded, whose indication is not distinct enough from its surroundings. Absent, late or ambiguous state feedback, which prevents verifying that what was commanded actually happened. The familiar sequence that, in a particular case, has one step more or one step fewer: the hand completes the learned gesture before the mind has re-read it. Finally, memory load, when the interface requires retaining a value or a state that it stops displaying.

The demonstration then consists of going through the tasks one by one, listing the plausible errors for each, and then showing where they are caught. Three barriers are acceptable, and it is necessary to state which one applies.

The three barriers opposed to a design-induced errorA sequence from left to right. A possible error enters a first barrier, avoid by design, then a second, detect by the alert, then a third, recover by the procedure. From each barrier an arrow points downward to a consequence-free outcome: error avoided, error seen, error caught. Whatever passes all three barriers exits to the right and becomes an in-service event.possibleerrorAvoidby designDetectby the alertRecoverby the procedureeventerror avoidederror seenerror caughtone barrier is enough — but you must be able to say which one
Certification does not require that error be impossible, but that it be stated where it stops. Every plausible error must be met by at least one named and justified barrier. An error point with none, or one that relies only on the crew’s sustained vigilance, reopens the design.

Nothing requires having all three. But an error point that presents none, or that relies only on the crew’s sustained vigilance, is the kind of finding that reopens a design late in a programme — and the cost of that reopening explains a good part of the architectural choices visible in recent cockpits.

Means of compliance: from paper to the aircraft

Compliance is not demonstrated by a single test. It is obtained through a stack of means, negotiated early with the authority, where each tier answers a question the previous one could not settle. Task analysis says nothing about what a hand does under time pressure; the bench mock-up says nothing about competition between two systems; the simulator says nothing about noise, real motion or aircraft systems’ idiosyncrasies.

Each means of compliance settles one question — and leaves the next one open
Means What it establishes What it does not establish
Task and error analysis The inventory of plausible errors and the barrier opposed to each What a rushed, surprised or interrupted crew actually does
Bench mock-up, partial test Accessibility, legibility, consistency of an isolated gesture, bulk Interaction with the rest of the cockpit and competition for attention
Engineering simulator The sequence of tasks, perceived workload, rapid design iterations The representativeness of the crew, still close to the project team
Certification simulator sessions The behaviour of trained crews facing chosen scenarios The frequency of a rare error, beyond the reach of a small sample
Flight tests The actual behaviour of the aircraft, its systems and the sound environment Human variability, reduced to a few highly experienced crews

The economic paradox is known to everyone who has worked on a programme: the fidelity of the means increases with time, while the capacity to correct decreases. A bench mock-up can be reworked within a day; an embedded software standard is re-modified in months, requalification included. The permanent temptation is therefore to freeze before evaluating, then to evaluate in order to confirm — the exact opposite of what would produce useful information.

What test-pilot evaluation cannot see

Crewed sessions are indispensable, and nobody proposes doing without them. But their detection power has structural limits, which owe less to the people than to the set-up itself.

Two pilots in uniform review a document in an aircraft cockpit at dusk.
Human factors shape cockpit alerting system design and evaluation.

System knowledge. Someone who has followed the design for years can no longer judge its learnability. They know why the mode returns, which light to watch, which page to call up. Their success proves nothing about a crew discovering the interface fresh out of transition training.

Recovery skill. A test pilot recovers a great deal; that is precisely their trade. Every brilliant recovery erases the trace of the flaw that made it necessary, unless the observer records not the outcome but the path and the time taken to get there.

Expectation. An announced scenario is not the same event. Surprise is the ingredient that turns a minor ambiguity into a major error; a briefing that removes it removes, at the same time, the essential part of the phenomenon it claims to measure. Hence the value of blind scenarios, where the crew flies a mission and not a test.

The validity of the situation. No fatigue, no end of duty, no deferred item on the minimum equipment list, no unfamiliar crew member, no commercial pressure, no cabin call and no route change at the worst possible moment. The evaluation observes an available crew, which is rarely the crew that will actually encounter the case.

Sample size. This is the hardest limit, because it is corrected by neither competence nor method. An evaluation campaign mobilises a few dozen crews at best; an in-service fleet produces as many in a few hours. The consequence is arithmetic.

Chance of encountering an error at least once, by number of crews evaluatedTwo curves. On the horizontal axis, the number of crews evaluated, from zero to thirty; on the vertical axis, the chance of encountering the error at least once, from zero to one hundred percent. As a solid line, an error occurring one time in three: the curve nears one hundred percent before the tenth crew. As a dashed line, an error occurring one time in fifty: the curve stays below the fifty-percent line at the thirtieth crew. A dashed horizontal line marks this fifty-percent level.chance of encountering it at least oncecrews evaluated100 %50 %00102030frequent error: 1 in 3rare error: 1 in 50
An evaluation campaign quickly finds what is frequent and almost certainly misses what is rare. Reading: if an error occurs one time in three occasions, about a dozen crews are enough to see it appear; if it occurs one time in fifty, thirty crews do not even have a fifty-fifty chance of encountering a single occurrence. This is a property of sampling, not a flaw in the method.

The remedies exist and cost little: bring in, late in the programme, crews representative of the target population and unfamiliar with the design; mix experience levels within a single crew rather than pairing two experts; record detection times, hesitations and eye movements rather than the verdict alone; and not treat entry into service as the end of the evaluation. The first months of operation remain the only moment when the sample becomes large enough to reveal what is rare — provided the reports come back, and come back usable. Discussions among operators of the same type, including on FL330’s professional forums, play a role here that no test campaign can replace.

The trap. You catch the flaw every time, so you don’t report it. This is exactly how a fictitious barrier ends up credited: the file assumes crew detection, operational experience appears to confirm this assumption, and the one case where detection fails will be the one where it was most needed — a mismatched crew, end of a rotation, a second anomaly in the same sequence. A systematically compensated flaw remains a flaw; it is even the definition of a design-induced error that has not yet found its day.

The right reflex. In a report, describe the chronology of the interface before describing your actions: what was displayed, in what order, what was inhibited, what was cleared and by whom, how much time separated the appearance from the reading. A report that concludes « crew error » closes the file; a report that restores the sequence enables an induced-error analysis, the only route to a design or procedure change.

What remains to be checked, in your own operation

A type certificate reads in the past tense: it attests that a demonstration was made, on a given definition, under written assumptions. Three areas deserve close attention from an operator, a safety officer, or someone preparing a type qualification.

  • Variants of the same type. Two standards can share the same certificate without sharing the same alerting grammar: an identical message does not necessarily have the same priority, the same inhibition or the same clearing mode. Differences documentation first describes procedures; the logic itself must be sought in the chapter describing the alerting system itself, and in the operational suitability data that set what differences training must cover. On a mixed fleet, this is precisely where habits are formed that are right for one aircraft and wrong for the other.
  • Later additions. Equipment installed through a subsequent modification comes to place its messages within a philosophy written before it existed. Overall coherence is re-examined, but rarely to the depth of the initial certification, and never with the same volume of crewed sessions. This is the point where an operator has the most to gain from reading what its approved documentation actually says, rather than assuming the whole thing was replayed.
  • Everything outside the certified chain. A tablet application, a tracking device, an ambient alert have, by construction, no place in the priority hierarchy: nothing arbitrates them against the aircraft’s system, nothing inhibits them in the phases where the certified system falls silent. They borrow attention without taking part in the grammar. The operator’s policy on these tools therefore becomes, in practice, part of the cockpit’s alerting philosophy, without ever having had its demonstration.

One grey area remains, deliberately accepted: the detection barrier credited to the second crew member. It is solid as long as the design assumption is, and any move toward a reduced crew, an asymmetric workload or an in-flight rest period shifts it. It is precisely at this level — that of assumptions, not of screens — that the ongoing debates about tomorrow’s cockpits are being played out.

The right reflex, before a qualification. Don’t learn the list of messages, learn the grammar: how many levels, what each requires, what is inhibited and during which phase, how two simultaneous alerts are ranked, what is latched and what is recalled after clearing. This is what a check session on a non-standard case actually tests, and it is what transposes from one type to another. FL330’s training programmes start from this entry point rather than from the catalogue.

This article describes an evaluation mechanism, not a rule applicable to any given type. The certification specifications in force, the decisions and special conditions issued by the authority, and the operator’s approved documentation — flight manual, operations manual, differences documents and standard operating procedures — prevail in all circumstances over what is written here.

Questions fréquentes

Is an error made during a certification session attributed to the crew or to the design?

The session is not meant to grade the crew. An error made by a trained, representative crew is first treated as data about the design: the interface trait that made it likely is sought, then it is checked whether a barrier would have detected or caught it. Attribution to the crew is retained only if no identifiable characteristic of the interface is at fault.

What exactly does an alerting philosophy document contain?

It is the document that sets the rules before the messages are written: number of levels, response required by each, attention-getting device reserved for it, inhibition phases, latching, clearing and recall, arbitration between simultaneous conditions. Every message on the type is then judged consistent or not with this grammar, rather than assessed in isolation.

Why not settle for an evaluation by test pilots alone?

They know the system from the inside, which makes them unfit to judge its learnability, and their recovery skill masks the flaws they compensate for. On top of that comes an arithmetic limit: a few dozen crews cannot reveal an error that occurs only once in several dozen occasions. Hence the call for representative crews unfamiliar with the design, and close monitoring of the first months of operation.

Do two aircraft of the same type necessarily share the same alerting logic?

No. Two standards can share a type certificate without sharing the same priorities, the same inhibitions or the same clearing modes, and a message with identical wording can have neither the same rank nor the same behaviour. Differences training and the operational suitability data indicate what has changed; the detail of the logic is found in the chapter describing the alerting system itself.

What should be written in a report for it to actually be useful?

The chronology of the interface before that of the actions: what was displayed and in what order, what was inhibited, what was cleared and by whom, the delay between the appearance and the reading. A report that concludes with “crew error” opens no analysis; a report that restores the sequence makes it possible to qualify a design-induced error and, where applicable, to change the design or the procedure.

Can a tablet application supplement the aircraft's alerting system?

It can provide information, but it does not enter the priority hierarchy: nothing arbitrates it against the certified system, nothing inhibits it in the phases where that system falls silent. It therefore consumes attention without taking part in the alerting grammar. The operator’s policy, which specifies the phases in which its use is permitted, is what fills this gap in operation.

Commentaires

Leave a Reply

Découvrez l’écosystème FL330

FL330, c’est toute l’aviation professionnelle — explorez nos autres espaces.