Likelihood and Probability in R. A. Fisher’s
Statistical Methods for Research Workers
Introduction
There is a passage in Chapter 1 of Fisher’s Statistical Methods for
Research Workers describing the proper roles of probability and likelihood.
It criticises the Bayesian misuse of probability (termed “inverse probability”)
and seems to anticipate the 1960s discussion of the likelihood principle. (See
the entries Bayesian, Likelihood and Likelihood Principle
in Probability & Statistics on the
Earliest Uses Pages.) The purpose of
this presentation is to make the passage available and, more especially, to
show the changes Fisher made in the later editions of the book.
The book was first published in 1925 and the division of responsibility
between probability and likelihood, which it lays down, follows what
Fisher had written in On the "Probable Error" of a
Coefficient of Correlation Deduced from a Small Sample (1921). This paper not only describes the
division (p. 24), it shows how to do likelihood inference, or at least how to
construct likelihood intervals (see p. 25). However the wide-ranging and
masterly On the Mathematical Foundations of
Theoretical Statistics (1922)
was not a likelihood text in the same sense. It also presents maximum
likelihood but the properties it emphasises are properties in ‘repeated
sampling.’ The Theory of Statistical Estimation (1925) takes the same approach. Two New Properties of Mathematical
Likelihood (1934) added a
conditional dimension to Fisher’s theory of estimation but the concern was still with repeated sampling properties.
Although likelihood inference was proclaimed as a fundamental form of inference
in all editions of Statistical Methods for Research Workers, it was not
developed any further for more than 30 years. Likelihood inference re-appears
in Statistical Methods and Scientific Inference (1956) and the
discussion there picks up from where the 1921 paper and Statistical Methods
for Research Workers had left off, “The likelihood supplies a natural order
of preference among the possibilities under consideration” (chapter III, §6).
By 1925 Fisher’s opposition to Bayesian inference (as it would now be
called) was already fixed; see his On the Mathematical Foundations of
Theoretical Statistics. Over the life-time of Statistical Methods
for Research Workers Fisher’s views on Bayes, the person, changed and he
also debated with Bayesians, such as Harold Jeffreys (see Harold Jeffreys as a Statistician).
However these developments led
to no rewriting of the 1925 passage. The rewriting was done primarily to
accommodate the fiducial argument, which gave probability a role in
inference—a legitimate role in Fisher’s eyes, unlike the Bayesian
pretence of a role. The changes began to appear in the 1932 (4th)
edition, following the publication of his Inverse Probability which
introduced the fiducial argument. The article appeared in 1930 and Fisher
slowly fiducialised the book by rewriting the passage
as well as by making changes elsewhere. He did not revise his account of
likelihood except to describe the power function of Neyman and Pearson as a
specialised application of it!
Click
here to see the passage as it appeared in the 14 editions of Statistical
Methods for Research Workers. The trail of red
ink
shows the changes.
Primary
Literature
Works
by Fisher
Most of the relevant documents can be found on the web: Fisher’s
articles from the University of Adelaide’s R. A. Fisher Digital Archive
and the first edition of SMRW on Christopher Green’s site. These sites
give the full references.
- Statistical
Methods for Research Workers (The first edition is available in Christopher Green’s Classics in the History of
Psychology.)
- Statistical
Methods for Research Workers (The 14th edition is in print as part of Statistical
Methods, Experimental
Design and Scientific Inference, Oxford: Oxford University
Press, 1990.)
- On the "Probable Error"
of a Coefficient of Correlation Deduced from a Small Sample. (1921)
- On the Mathematical Foundations of
Theoretical Statistics.
(1922)
- Theory of Statistical Estimation. (1925)
- Inverse Probability. (1930)
- Two New Properties of Mathematical
Likelihood. (1934)
- Statistical
Methods and Scientific Inference, 3 editions, 1956/59/74, Edinburgh: Oliver & Boyd. (The 3rd
edition is in print as part of Statistical
Methods, Experimental
Design and Scientific Inference, Oxford: Oxford University
Press, 1990.)
Work on the likelihood principle by others
George Barnard developed
Fisher’s division between probability and likelihood in a 1949 paper but the
work had little impact. The same cannot be said of Allan Birnbaum’s effort,
which appeared when there was a strong revival of interest in foundations.
Edwards’s book develops a system of statistics based on likelihood. Berger&
Wolpert provide a survey.
- G. A.
Barnard (1949) Statistical Inference, Journal of the Royal Statistical
Society. Series B (Methodological), 11, 115-149. JSTOR. (For a personal appreciation of
Barnard see Lindley's George. Barnard (1915-2002)
)
- Barnard
sent Fisher a copy of his paper and in reply Fisher congratulated him on
his “enterprise”: see pp. 5-6 of J. H. Bennett (1990) (ed) Statistical
Inference and Analysis: Selected Correspondence of R. A. Fisher,
Oxford, University Press.
- Allan
Birnbaum (1962) On the Foundations of Statistical Inference" Journal
of the American Statistical Association, 57, 269-306 (with
comments by L. J. Savage, G. A.
Barnard, J. Cornfield, Irwin Bross, G. E. P. Box, I. J. Good, D. V. Lindley, C. W. Clunies-Ross, J. W. Pratt, H. Levene, T. Goldman, A. P. Dempster,
O. Kempthorne and reply
by Birnbaum, 307-326). JSTOR
- Between 1958 and –60 Birnbaum sent Fisher copies of
his papers but Fisher was not interested in Birnbaum’s project of improving Neyman’s theory.
Birnbaum did not send Fisher a copy of his likelihood paper which was
presented at a meeting in December 1961. However the paper may have been
enclosed with a letter from Kempthorne to which Fisher replied on 19th
February 1962; Fisher described Birnbaum as “a very bewildered type.” In a
letter to H. E. Kyburg dated 14th May
Fisher mentioned that “Likelihood Statements” had “recently been
rediscovered by Birnbaum”: see pp. 188-9 of J. H. Bennett (1990) (ed). Birnbaum's paper appeared in the June issue
of JASA. On 29th July Fisher died.
- A. W. F.
Edwards (1972/1992) Likelihood (the second expanded edition
reprints the first with some related articles and a new preface),
Baltimore: Johns Hopkins University Press.
- J. O.
Berger, R. L. Wolpert (1988) The
Likelihood Principle (2nd. Edition), Hayward, Calif.:
Institute of Mathematical Statistics.
Secondary
Literature
For a perspective on Fisher’s Statistical Methods for Research Workers
see Edwards, (2005). Aldrich (2008) describes Fisher’s attitudes to Bayesian
inference and to Bayes the man. Edwards (1999) provides a nice introduction
to likelihood and its history. Aldrich (1997) gives a detailed account of
Fisher’s ideas on likelihood to 1922, while Aldrich (2000) describes the
development of the fiducial argument and the consequences of this development
for the likelihood/probability division. Hald has a
longer perspective. The Guide provides additional information on Fisher,
including references on the fiducial argument. Figures contains general
information on the history of statistics.
- A. W. F. Edwards, (2005) “R. A. Fisher, Statistical Methods for Research Workers, 1925” in I. Grattan-Guinness (ed.) Landmark Writings in Western Mathematics : Case Studies, 1640-1940, Amsterdam: Elsevier.
- John Aldrich (2008) R. A. Fisher on Bayes and Bayes’ Theorem. Bayesian Analysis, 3, 161-170. here
- A. W. F.
Edwards (1999) “Likelihood” preliminary version of IESBS entry.
- John
Aldrich (1997) R. A. Fisher and the Making of Maximum Likelihood 1912-22, Statistical
Science, 12, 162-176. Project Euclid JSTOR
- Anders Hald (1999) On
the History of Maximum Likelihood in Relation to Inverse Probability and
Least Squares, Statistical
Science, 14, 214-222. Project Euclid JSTOR
- John
Aldrich (2000) Fisher’s “Inverse Probability” of 1930, International
Statistical Review, 68, 155-172. pdf
- John
Aldrich A
Guide to R. A. Fisher
- John
Aldrich Figures from the History of
Probability and Statistics
John
Aldrich, University of Southampton, Southampton, UK. (home) December
2003. Most recent changes February 2015.
