The Switch That Taught Us to Stop Blaming Pilots

When capable people keep making the same mistake, the design is inviting it. A World War II bomber cockpit taught us that lesson, and teams shipping AI need it today.

When an incident review ends at "user error," the investigation stops one step too soon. The real cause stays in the product, ready to fire again. Your customers pay for it first. Then your support team pays for it, and then your reputation does.

What happened in the B-17 cockpit?

The B-17 was a four-engine heavy bomber, flown throughout the war. Crews were bringing their planes home and landing safely, then retracting the landing gear on the runway. The planes dropped onto their bellies, and the reports said pilot error.

Alphonse Chapanis, a young psychologist working with the Army Air Forces, went and sat in the cockpit. He found the landing gear control and the wing flap control side by side, with the same size and the same shape. A tired pilot reaching for the flaps after touchdown could grab the gear instead, and nothing under their hand would warn them.

How did a small change fix it?

A wheel-shaped knob went on the landing gear control, and a wedge-shaped knob went on the flaps. Now a pilot could tell them apart by touch alone, without looking. Shape coding became standard, and US rules for transport aircraft still require cockpit control knobs to follow set shapes.

At the Aero Medical Laboratory, Paul Fitts and Richard Jones studied hundreds of incidents filed as "pilot error." The patterns pointed back at the cockpit, where controls looked, felt, or sat too much alike. Their work helped found the field of human factors, the study of designing systems around the people who use them.

Where does the same switch show up in AI products?

The cockpit is gone, but the identical switches are everywhere.

• Actions look alike. "Regenerate" and "Delete" sit side by side in the same style, and only one of them can be undone.

• Confidence looks alike. An assistant sounds just as sure when it is guessing as when it is right, so the answer feels the same either way.

• Sources look alike. Generated text and verified fact arrive in the same font, the same box, and the same tone.

Good design gives each of these its own shape. Different consequences look and feel different, so the person using the product can tell them apart under pressure, on a bad day, without stopping to think.

How can you run Chapanis's method on your own product?

1. Retire "user error" as a root cause. In each incident review, I suggest asking what the design made easy, and what it made hard to notice.

2. Hunt for look-alikes. List every pair of controls, labels, states, or outputs that look the same but lead to different outcomes. Rank them by how bad the wrong choice would be.

3. Code each difference in more than one way. Use shape, position, wording, and color together. A cue that lives in only one channel fails someone, whether that is a screen reader user, a person with low vision, or anyone under stress.

4. Put distance around what can't be undone. Separate it in space, slow it down, or make it reversible. Undo beats a confirmation box people learn to click through.

5. Give uncertainty a shape. Show when an AI answer is a guess, where it came from, and what the tool is about to do before it does it.

6. Test in real conditions. Add interruptions, fatigue, time pressure, and a small screen in bright sun. The B-17 crews met that switch at the end of a long mission, not in a calm lab.

Where can you read the original sources?

• Fitts, P. M., and Jones, R. E. (1947). Analysis of Factors Contributing to 460 "Pilot-Error" Experiences in Operating Aircraft Controls. Aero Medical Laboratory, Wright-Patterson Air Force Base. Summarized in Building better cockpits, Cognitive Science Society.

• Chapanis, A. (1953). "Psychology and the Instrument Panel." Scientific American 188(4).

• 14 CFR 25.781, Cockpit control knob shape, Electronic Code of Federal Regulations.

In short

• When skilled people repeat the same mistake, it is worth looking at the design before blaming the person.

• Shape coding fixed the B-17 gear and flap mix-up by making different consequences feel different.

• AI products need the same care: actions, confidence, and sources each need their own shape.

This is the thinking behind my book Prompt: Good Design for the AI Era. See the book, or read more of my articles on design history.