Why log odds?

When analyzing categorical data, different results are sometimes obtained using logistic regression instead of ANOVA or a t-test on proportions. This raises the question: which is the "right" analysis? Defenses of logistic regression often appeal to statistical authority—i.e., we should do logistic regression because that's what statisticians tell us to do—rather than any principled argument in favor of the log odds scale. To justify use of the log odds scale, I resurrect an analysis from one of the foundational papers on probit analysis by Bliss (1934). The mystery was why each unit increase in dosage gave rise to a curved rather than a linear mortality function.


An idealized S-shaped "mortality" curve is shown above. One can see that an increase of 50 mg of pesticide kills fewer pests when the increase is from 50 to 100 mg than when the increase is from 100 to 150 mg. Thus, the dependence of pest mortality on dosage is nonlinear: the effect of an increase in dosage depends upon where that increase occurs. Why?

Bliss' idea was that each individual pest has a threshold dosage, such that dosages greater than the threshold would kill the pest. Importantly, he assumed that this threshold varies across pests, following a normal distribution. The figure at left illustrates the distribution of thresholds for 1000 pests. The figure shows the proportion of pests dying given four hypothetical dosages. The blue line represents the administered dosage. The part of the distribution colored in red represents the pests that would die given this dosage (i.e., whose dosage thresholds are less than or equal to the administered dosage).

When no dosage of pesticide is given (0 mg), only 2 pests die (due to normal mortality unrelated to the pesticide). This is shown in the top panel. If we had administered a dosage of 50 mg instead, 23 pests (.023) would have died (second panel). A 100 mg dosage would have killed 164 pests (.164), and a 150 mg dosage would have killed 515 (.515) pests. If we continue this exercise and then plot the proportion killed as a function of dosage, we will get a sigmoid shaped function as shown above.

Note that what we are doing is slicing up a normal distribution, and the proportion that gets sliced off to the left depends upon where we cut. Because of this, the sigmoidal shape of the mortality curve corresponds to a cumulative distribution function (cdf) for a normal distribution. In probit analysis, this cdf is the link function. Logistic regression is has a different link function, but gives highly similar results to a probit analysis.

So, how does this analogy map back onto categorical variables such as accuracy or point of gaze? One can imagine that at the point of producing some response, individual subjects are in some state of belief regarding some event (e.g., whether a string of letters is a word or not, whether some stimulus is a member of category A or B, whether image X is the target or not). For example, a group of subjects might have a distribution around a mean corresponding to a probability of .4. Importantly, although the underlying state of belief is continuous, subjects can only emit a binary response (e.g., yes or no, look at the target or at some other region). If the state of belief at the time of response is above some threshold, they will tend to emit the corresponding response (e.g., look at the target); if they are below the threshold, they will tend to not emit the response (e.g., look at a region other than the target). It is where the distribution of belief states among the subject population intersects a response threshold that determines the resulting proportion.


Bliss, C.I. (1934). The method of probits. Science, 79, 38-39.

Author: Dale Barr

Date: 2010-12-03 11:44:04 GMT

HTML generated by org-mode 7.02 in emacs 23