How the patient mix can flip a treatment comparison
A treatment’s overall success rate is a weighted average: each group’s result counts in proportion to how many patients in that group received the treatment. If one treatment is given mostly to easier cases and another mostly to harder ones, the totals reflect both effectiveness and patient mix.
Imagine two treatments, A and B. In mild cases, A succeeds 90% of the time and B 80%; in severe cases, A succeeds 20% and B 10%. A is better in both comparisons. But suppose A is used for 90 severe patients and 10 mild ones, while B is used for 10 severe patients and 90 mild ones. A’s overall success rate is 27%, compared with 73% for B—not because A performs worse within either group, but because its patients are far more likely to have severe illness.
The arithmetic behind the reversal
The pooled totals answer a different question from the like-for-like comparisons. They ask, “What happened among all patients who received each treatment?” The group comparisons ask, “What happened among patients with similar severity?” When the treatments have very different mixes of patients, those answers can point in opposite directions. This kind of reversal is known as Simpson’s paradox.
The same logic applies beyond medicine whenever groups differ in an important way—for example, comparing schools that enroll students with different starting preparation. Averages can conceal that difference because the larger groups dominate the totals.
What a fair comparison needs
To judge treatments, analysts can compare patients within meaningful severity groups or adjust statistically for differences between them. A well-designed randomized trial helps by assigning treatment so that, on average, the groups begin with similar characteristics. Neither a pooled average nor a subgroup result should be read without asking who was included and how the comparison was made.
The striking lesson is that a correct calculation can still answer the wrong practical question: the total may describe the patients treated, while the decision depends on how the treatments compare for patients alike.
