Why One-Factor-at-a-Time Fails

Design of experiments (DOE) is a planned way of changing several process inputs at the same time, so that one set of runs tells you which inputs matter, how much they matter, and whether they act together. It is the main tool for turning "we think temperature matters" into a measured, defensible answer.

Most engineers learn a different habit first. They hold everything constant, change one input, look at the result, then move to the next input. This is called one-factor-at-a-time (OFAT) experimentation. It feels careful and scientific. In practice it has three weaknesses that a designed experiment removes:

• It cannot see interactions. If the effect of one input depends on the level of another, OFAT gives a different answer depending on where you started.
• It wastes runs. Each comparison uses only the runs at two settings. A factorial design uses every run to estimate every effect.
• It is easily fooled by drift. OFAT runs are usually done in a fixed order, so a change in raw material or ambient conditions during the study looks like a factor effect.

A worked example (invented figures)

Brenlow Engineering applies a powder coating to steel brackets and wants higher pull-off adhesion, measured in megapascals (MPa). Three inputs are suspected: cure temperature (160 or 180 °C), cure time (10 or 20 minutes) and blast surface profile (50 or 75 micrometres). Later in this course the team runs all eight combinations twice. The average adhesion for each combination was:

Temperature	Time	Profile	Mean adhesion (MPa)
160	10	50	11.30
180	10	50	12.65
160	20	50	11.25
180	20	50	16.05
160	10	75	13.30
180	10	75	15.15
160	20	75	13.65
180	20	75	18.05

What an OFAT engineer would have concluded

Suppose an engineer starts at the current setting (160 °C, 10 minutes, 50 µm) and first tries the longer cure time. Adhesion moves from 11.30 to 11.25 MPa, which is no change. The conclusion "cure time does not matter" is written down, and time stays at 10 minutes.

Next the engineer raises temperature to 180 °C (with time still 10 minutes): adhesion rises to 12.65 MPa. Then the rougher profile: 15.15 MPa. The study ends at 180 °C, 10 minutes, 75 µm, with 15.15 MPa, and a report that says "time has no effect".

The full table shows the truth. At 180 °C, the longer cure time adds about 3 MPa. Time matters a great deal, but only when the temperature is high. This is an interaction, and the OFAT path walked straight past it. The best combination, 180 °C, 20 minutes, 75 µm, gives 18.05 MPa, almost 3 MPa more than the OFAT answer.

The efficiency argument

In the factorial version of this study, every one of the 16 runs contributes to the estimate of the temperature effect: eight runs at 160 °C are compared with eight runs at 180 °C. The same 16 runs also estimate the time effect, the profile effect and every interaction. To estimate three main effects with the same precision by OFAT, you would need eight runs at each end of each comparison, which is far more runs, and you would still learn nothing about interactions.

This is the idea R. A. Fisher developed for agricultural field trials in the early twentieth century: vary factors together, in a planned pattern, and let the arithmetic separate their effects. The rest of this course shows how to plan, run, analyse and confirm such experiments in a plant or laboratory.