Harold Jeffreys as a Statistician
The Jeffreys prior is part of the furniture
of Bayesian statistics but otherwise the work of Harold Jeffreys is little
known. Jeffreys was a noted physical scientist who re-established the
statistical theory of his time on Bayesian foundations. This page is a guide to
literature and websites which may be useful to anyone interested in Jeffreys’s
statistical work and its background. The emphasis is on Jeffreys’s own writings
and on the older literature.
John Aldrich,
Original version February
2003
Most recent changes November
2011
Links
This
guide has links to some free sites,
including MacTutor
for biographies and the University of
Adelaide Library for
_____________________________________________________________
Around
1930 Harold Jeffreys F.
Jeffreys
gave a succinct account of what he did in probability/statistics and why he did
it in
Inverse
probability was how Jeffreys referred to the Bayesian approach; the terms Bayesian,
classical, and frequentist only became current after his main
work was done. There are many relevant
entries on the Earliest known uses pages, see Probability
& Statistics on the Earliest Uses Pages: a search for Jeffreys on the front page will show on
which of the individual pages he appears and a further search on the page will
bring up all his appearances.
Earlier writers used Bayesian methods but Jeffreys has a
good claim to be considered the first Bayesian statistician in that he used only Bayesian methods. Earlier Bayesian
writing—from authors such as Laplace, Gauss, Edgeworth and Pearson—is described
in
Jeffreys’s
knowledge of this early literature was limited and he spent some time
re-inventing the wheel.
There
is a short MacTutor
biography focussing on Jeffreys’s activities as an applied mathematician—it
has photographs and conveys something of his personality. There are
encyclopedia articles on Jeffreys as a statistician by Lindley and Zellner.
Cook’s (1991) memoir concentrates on Jeffreys’s contributions to physics and
applied mathematics. The festschrift edited by Zellner includes accounts of
Jeffreys’s contribution to statistics by Good, Geisser and Lindley. There are
memorial pieces in Chance including
one by Lady Jeffreys (Bertha
Swirles Jeffreys, Bertha
Swirles, Lady Jeffreys Bertha Jeffreys), his wife and collaborator on Methods of Mathematical Physics.
(available on Internet
Archive. Howie’s book on the
controversy with Fisher discusses other phases of Jeffreys’s life and is
particularly strong on his work in the 1920s. Aldrich looks at Jeffreys’s
preparation for writing the Theory of Probability.
Photographs.
The MacTutor biographies of Jeffreys
and his wife
have photographs. The four photos of Jeffreys at the Emilio Segrè Visual Archives include one at
the time of their wedding.
Video. The Royal Statistical Society created a
series of filmed interviews with distinguished statisticians: Jeffreys was
interviewed by Lindley. Copies can be purchased from the Society.
The Jeffreys papers at
Jeffreys wrote more than 400 papers, usually alone, on subjects ranging across celestial mechanics, fluid dynamics, meteorology, geophysics and probability: see here for a list. There is a six volume set of papers:
·
H. Jeffreys and B. Swirles (eds.)
(1971-77) Collected Papers of Sir Harold
Jeffreys on Geophysics and other Sciences in six volumes,
The statistics papers are in volume 6, Mathematics, Probability & Miscellaneous Other Sciences. The coverage is not comprehensive for Jeffreys omitted papers that had been superseded by his books Scientific Inference and Theory of Probability.
Jeffreys is often described as the founder of modern British geophysics. Many of his contributions are summarised in his book The Earth. Brush (1980) describes one of Jeffreys’s main discoveries, that the core of the Earth is liquid. Jeffreys figures prominently in Brush’s three volume A History of Modern Planetary Physics (Cambridge University Press 1996).
· H. Jeffreys (1924) The Earth, further editions in ‘29, ’52, ’59, ’70 and ’76, Cambridge, Cambridge University Press.
·
E.
· B. A. Bolt (1991) Sir Harold Jeffreys and Geophysical Inverse Problems, Chance, 4, 15-17.
Jeffreys talked about
his work in seismology in an interview
with Henry Spall. Jeffreys was elected to the Royal Society in 1925 and the election
certificate shows how he was regarded at the time.
Jeffreys
spent virtually his entire life in
A.W.F.
Edwards tells me that Jeffreys’s influence on Barnard was
considerable and that Jeffreys probably influenced Alan Turing. I. J. Good does not discuss the question of
influence in his “A. M.
Turing’s Statistical Work in World War II” (Biometrika, 66,
(1979), pp. 393-396 JSTOR)
but there seem to be clear echoes of Jeffreys in Turing’s thinking.
_____________________________
Scientific Inference back
to contents
Jeffreys
first used probability to deal with problems in the philosophy of science.
Jeffreys read Karl
Pearson’s Grammar of Science in 1914 and it made a great impression
on him, with its emphasis on the probabilistic basic of scientific inference.
Jeffreys treated probability as a degree of reasonable belief, an epistemic
conception common to several
Howie is very informative on the
Wrinch-Jeffreys collaboration. Aldrich (2005) also discusses this phase of Jeffreys’s
career.
Jeffreys summarised and extended the
Wrinch-Jeffreys work in his book Scientific
Inference.
The first edition was noticed in the philosophy journals. See
The second edition (which was essentially a new book) was noticed more widely. See
·
Anon.
Biometrika, 44, No. 3/4 (Dec., 1957), 538-539. (available
through JSTOR)
The third edition, more or less a second edition of the second edition, got some attention. See
In
the preface to the 2nd
edition of the Probability Jeffreys
explained his relationship to the other
According to Howie, Jeffreys met Keynes
once—on a train. Jeffreys reviewed Keynes’s Treatise on Probability,
welcoming it but faulting it for its caution.
·
The
Theory of Probability Nature, 109, (1922), 132-133.
There is a brief account of Jeffreys’s
view of Keynes in
·
J.
Aldrich ((2008) Keynes among the Statisticians. History of Political Economy, 40, 265-316. Revised version of Southampton University
Economics Discussion
Paper 2006
Jeffreys knew
Frank Ramsey (1903-1930) (N.-E.
Sahlin St.
Andrews D. H. Mellor) but
did not know until after he had died death that he was interested in probability.
Jeffreys discusses Ramsey’s views in the Theory of Probability.
Jeffreys
did not know the work of the Italian subjectivist, Bruno de Finetti
(1906-1985), although de Finetti wrote about Jeffreys’s Scientific Inference
in a 1938 article translated as
A recent article compares Jeffreys with Ramsey and de
Finetti
Placing Jeffreys has also been an issue
in the statistical literature: see Zellner and Kass.
Philosophers of probability have
generally taken Keynes as the representative
Bayesianism
is now fashionable in philosophy of science. An early exponent building on
Jeffreys was
These
references relate to philosophy but Scientific Inference took the Wrinch & Jeffreys ideas on
probability off in a new direction by considering how physical scientists used
probability when they analysed data. Chapter V on Errors includes a Bayesian derivation of Student’s distribution.
_____________________________
The controversy with Fisher back to
contents
Around
1930 Jeffreys, now concentrating on empirical geophysics, began devising
methods for analysing data based on epistemic probability. He was extending the
methods used by physical scientists and did not know much about, or greatly esteem,
the efforts of statisticians. Meanwhile Ronald Fisher (1890-1962), probably the
most influential twentieth century statistician, had rejected the Bayesian
approach and based his work, including maximum likelihood, on frequentist
foundations. There is a fine biography of Fisher by his daughter
For more on Fisher see A Guide to R. A. Fisher.
Fisher and Jeffreys first took serious
notice of each another in 1933. About all they knew of each other's work was
that it was founded on a flawed notion of probability. Jeffreys (1933)
criticised Fisher (1932) and Fisher (1933) criticised Jeffreys (1932) with a
rejoinder, Jeffreys (1933a). The journal called a halt to the controversy by
getting the parties to coordinate their last words, Fisher (1934) and Jeffreys
(1934). (The Fisher articles are all available from
·
R. A. Fisher
(1932) Inverse Probability and the Use of Likelihood, Proceedings of the
·
H.
Jeffreys (1933) On the Prior Probability in the Theory of Sampling, Proceedings
of the
·
H.
Jeffreys (1932) On the Theory of Errors and Least Squares, Proceedings of
the Royal Society, A, 138, 48-55. (available through JSTOR)
·
R. A. Fisher
(1933) The Concepts of Inverse Probability and Fiducial Probability
Referring to Unknown Parameters, Proceedings of the Royal Society, A, 139,
343-348. (available through JSTOR)
·
H.
Jeffreys (1933a) Probability, Statistics, and the Theory of Errors, Proceedings of the Royal Society, A, 140,
523-535 (available through JSTOR)
·
R. A. Fisher
(1934) Probability, Likelihood and the Quantity of Information in the Logic
of Uncertain Inference, Proceedings of the Royal Society, A, 146,
1-8. (available through JSTOR)
·
H.
Jeffreys (1934) Probability and Scientific Method, Proceedings of the Royal
Society, A, 146, 9-16. (available through JSTOR)
·
J. Aldrich (2008) R. A. Fisher on Bayes and Bayes’ Theorem. Bayesian Analysis, 3, 161-170.
Howie gives a full account of the dispute but Lane’s brief account is still useful. There is an even briefer account in Aldrich (2004). Seidenfeld considers how Jeffreys and Fisher used Keynes’s work in the debate.
Aldrich (2005) discusses the controversy
but goes beyond it to consider how Jeffreys treated Fisher’s ideas in the Theory
of Probability. Fisher did not mention modern work on inverse probability
in his Statistical Methods for Research
Workers (see Likelihood
and Probability in Fisher’s Statistical Methods ) although Jeffreys asked
him to—see correspondence
p. 170.
Fisher and Jeffreys never accepted the
validity of each other’s approach but their relationship mellowed into one of
relaxed toleration. Their developing relationship can be followed in their
letters which have been printed with notes in Bennett. There are more
personal memories in Swirles and Box.
·
J.
H. Bennett (1990) (ed.) Statistical
Inference and Analysis: Selected Correspondence of
·
G.
A. Barnard (1992) Review of Statistical
Inference and Analysis: Selected Correspondence of
_____________________________
Theory of
Probability back to contents
In the years following Scientific Inference Jeffreys worked hard on statistics. In 1937 an enlarged edition of the book came out but he soon judged this to be inadequate, telling Fisher in September 1938, “I wish some public benefactor would subsidize [the publisher] to scrap Scientific Inference and give me a chance of writing something up to date.” Bennett (p. 170)
In
1939 he had a new book with a different publisher that incorporated the papers
he had been writing on statistics. The Theory
of Probability by Harold Jeffreys, Reader in Geophysics,
Jeffreys used the phrase “theory of probability” in a unique way—to refer to a theory of inductive inference founded on the principle of inverse probability. The Probability is largely a treatise on theoretical statistics, though it begins with the foundations of probability and covers a range of applications comparable to that in Fisher’s Statistical Methods for Research Workers. Jeffreys was impressed by the solutions Fisher produced. In May 1937 he told Fisher of his impression that “only once in a blue moon” would we disagree about the “inference to be drawn in any particular case.” Bennett (p. 162). The trouble with Fisher’s inferences was that they lacked proper foundations!
The scope of the
book
In all its versions the Probability was a book of 8 chapters and appendices. The titles of the chapters and their general scope remained unchanged.
I Fundamental notions explains how induction rests on probability and presents axioms for probability. This is the chapter with the strongest links with Scientific Inference although the treatment is much more elaborate. Whitehead and Russell’s Principia Mathematica is a powerful presence.
II Direct probabilities presents basic distribution theory, including the standard distributions and transform techniques. The term “direct” contrasts with “inverse.” The former indicates concern with inference from “laws” to observations, the latter with the converse.
III Estimation problems are ones “where we are given the form of the law, in which certain parameters can be treated as unknown, no special consideration needing to be given to any particular values, and we want the probability distribution of these parameters given the observations.” (1939, p. 94). The statement (p. 96), “Our first problem is to find a way of saying that the magnitude of a parameter is unknown, when none of the possible values need special attention” leads into a discussion of ignorance priors. Jeffreys then went through some standard problems.
IV Approximate methods and simplifications covers maximum likelihood, minimum χ2 and other non-Bayesian arguments. Jeffreys treats them in a remarkably positive spirit.
V Significance tests: one new parameter. In significance testing “our problem is to compare a suggested value of a new parameter, often 0, with the aggregate of other possible values.’’ (1939, p. 193) Jeffreys was the first writer in the inverse probability tradition to write in detail about testing. In the preface he (p. v) complained, “Modern statisticians ... for the most part have rejected the notion of the probability of a hypothesis, and thereby deprived themselves of any way of saying precisely what they mean when they decide between hypotheses.”
VI Significance tests: various complications treats various situations do not conform to the one new parameter format.
VII Frequency definitions & direct methods treats frequency definitions of probability—Venn, von Mises etc.—and the “direct methods” (frequentist methods) of Karl Pearson, Fisher and Neyman-Pearson.
VIII General questions recapitulates the main arguments and considers some broader issues. On this chapter and the preceding one Wilks (1941, p. 194) commented, “The discussion is almost entirely informal and non-mathematical and as such it must be regarded in the category of personal opinion.”
Appendices Appendices came and went but the one constant was the set of Tables of K. K was Jeffreys’s symbol for what Good later called the “Bayes factor.” The introductory material explains the use of K and discusses the relationship between inferences based on K and on conventional tail areas—the “P integral”, as Jeffreys called it.
In its final form—the corrected impression (1966) of the third edition—the Probability had 459 pages compared to the first edition’s 380. The growth of the book can be seen from the table of contents of successive editions.
|
Chapters\Editions |
3rd |
2nd |
1st |
|
I. Fundamental Notions |
p. 1 |
p. 1 |
p. 1 |
|
II. Direct Probabilities |
57 |
47 |
46 |
|
III. Estimation Problems |
117 |
99 |
94 |
|
IV. Approximate Methods and
Simplifications |
193 |
168 |
145 |
|
V. Significance Tests: One New Parameter |
245 |
220 |
193 |
|
VI. Significance Tests: Various
Complications |
332 |
305 |
268 |
|
VII. Frequency Definitions and Direct
Methods |
369 |
341 |
300 |
|
VIII. General Questions |
401 |
372 |
332 |
|
Appendices |
425 |
396 |
356 |
|
Index |
455 |
408 |
377 |
A
detailed table
of contents of the third edition has been prepared by
Jeffreys wrote articles but most of their substance went into the big book. Thus overall assessments of his work like those in the festschrift are essentially assessments of the Probability. Zellner and Kass discuss Savage’s way of classifying Jeffreys. Lindley reflects on the book and Aldrich (2005) describes the first edition against the background of Jeffreys’s intellectual development. Aldrich (2002) mentions how Jeffreys translated some of Fisher’s ideas into Bayesian terms.
·
D.
V. Lindley (1986) On Re-reading Jeffreys, pp. 35-46 of I. S. Francis et al
(eds) Pacific Statistical Congress, New York: Elsevier.
·
J.
Aldrich (2002) How Likelihood and Identification went Bayesian, International Statistical Review, 70, 79-98.
For wider perspectives see
·
Jim
Berger Could
Fisher, Jeffreys and Neyman Have Agreed Upon Testing? This Fisher memorial
lecture has been published (with discussion) in Statistical Science, 18,
1-32 (2003) Euclid JSTOR.
The next sections give some details about the three editions and how they were received. Of the three editions the first was the most important for presenting the main ideas. The second edition introduced Jeffreys’s rule the topic in the Probability which has attracted most attention. The third edition appeared at the most propitious time.
_____________________________
The first edition 1939 back to contents
In 1939 Jeffreys was up-to-date with the work of the English statistical school, familiar with the work of Karl Pearson, Student, Fisher and Neyman-Pearson. Like most other English writers, he did not know the recent Continental work on probability, notably Kolmogorov’s Grundbegriffe. He knew only H. Cramér’s Random Variables and Probability Distributions (1937) which he recommended for a rigorous treatment of the central limit theorem.
In
the preface (reproduced in the current edition) Jeffreys thanks Fisher, Wishart
and the philosopher Richard
Braithwaite for their help. None agreed with him on probability. In the
text he mentions the
The first edition of the Probability was reviewed in the leading journals by prominent statisticians; Irwin had worked with both Karl Pearson and Fisher while Neyman (1894-1981) (ASA St Andrews) and Wilks (1906-64) (ASA.) would both be leaders of post-war statistics in the USA. Curiously the review in Nature was not written by a physicist but by the statistician Maurice Kendall. The reviewers respected Jeffreys’s scientific credentials but were sceptical of his system.
· M. G. Kendall (1940) Theory of Probability by Harold Jeffreys, Nature, 145, 607-608.
· J. Neyman (1940) Review of Theory of Probability by Harold Jeffreys, Journal of the American Statistical Association, 35, 558-559. (available through JSTOR)
· J. O. Irwin (1941) Theory of Probability by Harold Jeffreys, Journal of the Royal Statistical Society, 104, 59-64. (available through JSTOR)
· S. S. Wilks (1941) Theory of Probability by Harold Jeffreys, Biometrika, 32, 192-194. (available through JSTOR)
As well as the reviews by statisticians there were reviews by mathematicians and philosophers.
Jeffreys was a well-known physicist and his book appeared in the International Series of Monographs on Physics but it appears to have gone unnoticed by physicists.
As far as statistics was concerned, Wilks’s words seemed set to be prophetic: “It is doubtful that there will be many scholars thoroughly familiar with the system of statistical thought initiated by R. A. Fisher and extended by J. Neyman, E. S. Pearson, A Wald and others who will abandon this system in favour of the one proposed by Jeffreys in which inverse probability plays the central role.” The Annals of Mathematical Statistics the leading theoretical journal of the immediate post-war era carried no articles extending Jeffreys’s Bayesian theory before the 1960s.
_____________________________
The second edition 1948 back to contents
The flow of articles on statistics abated after the publication of the Probability and in 1940-5 Jeffreys published only three articles on statistics. In 1946, however, came a major development, the introduction of the Jeffreys prior as it is usually called.
In 1948 Jeffreys brought out a second edition of the Probability. He was now Plumian Professor of Astronomy in succession to Arthur Eddington who had taught him the theory of errors as an undergraduate. The main change in the book was an account of the invariant prior: see the preface. Jeffreys had been interested in invariance for a long time. In Scientific Inference he had discussed the consistent assignment of priors to the standard deviation, σ, and the modulus, h = 1/σ√2 , a measure of precision often used in the theory of errors.
In 1939 Jeffreys had sought to reconstruct modern statistics on the probability foundations he first proposed in 1919. After 1939 he did not follow the literature, nor was he impressed with the way the literature treated him. In the preface to the second edition he wrote, “I have not attempted to answer explicitly the criticisms made by reviewers, because on examination I found that they were all dealt with in the book already.”
The second edition was not widely
reviewed. Robbins and David were respectful but sceptical. Robbins noted that
Jeffreys did not discuss the costs of making errors while David found the
approach necessarily subjective. Neither was interested enough in Jeffreys’s
project to comment on the changes in the new edition.
_____________________________
The third edition 1961 and the Bayesian revival back to contents
Between 1948 and 1960 Jeffreys published very little on probability: reviews of philosophical works by Reichenbach, Russell and Carnap, an account of the “present position” and a note on the exponential family in 1960; see bibliography. The Russell review was the most elaborate of the reviews and the BJPS article described the present position as it was in the second edition of the Probability.
· Harold Jeffreys (1950) Bertrand Russell on Probability, Mind, 59, 313-319 (available through JSTOR).
· Harold Jeffreys (1955) The Present Position in Probability Theory, British Journal for the Philosophy of Science, 5, 275-289 (available through JSTOR)
In 1961 Jeffreys produced a third edition of the Probability. He had retired from his chair but he went on working until his death in 1989. This edition added material to the first chapter and some mathematical appendices. The only significant new reference was to Carnap’s Logical Foundations of Probability which he reviewed in 1952. In the corrected impression of 1966 material on time series analysis was added.
In its third edition the Probability remained a
conversation with Fisher and Principia
Mathematica. Raiffa and Schlaifer’s Applied Statistical Decision Theory was published in the same year. That work
reflected a huge change in the statistical climate as more statisticians became
interested in Bayesian ideas. There were many publications that contributed to
the Bayesian revival. In Britain Good (ASA
conversation
St
Andrews) was influenced by Jeffreys and by Cambridge probability more generally;
like Jeffreys and unlike most statisticians, Good was interested in logic and
the philosophy of science. The more influential US line of Savage
and Schlaifer was
very different. Their foundations added to the “personalism” (subjective
probability) of Ramsey
and de Finetti the “behavioralism” (decision-orientation) of Neyman and Wald;
the expected utility theory of von
Neumann and Morgenstern was also a great influence. Jeffreys was known to
the American statisticians but was not much of an influence. For Savage (1954,
p. 276) the Probability was “an ingenious and vigorous
defense of a necessary view, similar to, but more sophisticated than
Jeffreys
seems to have paid no attention to these developments in the statistical
literature; his expected utility was the moral expectation of
The
third edition, like the second, was not widely reviewed. However, for the first
time there were Jeffreys enthusiasts to receive it.
·
D. V. Lindley (1962), Theory of Probability. by Harold Jeffreys, Journal of the
American Statistical Association, 57, 922-924 (available through JSTOR:)
·
F.
Zitek Mathematical Reviews, (available through MathSciNet)
·
M.
Arcones (1998 reprint) Mathematical Reviews, (available through MathSciNet)
Good and Lindley wrote in very different
terms from earlier reviewers. Lindley’s review begins
This
is probably the most original and important book in statistics that has
appeared in the last forty years. The only serious competitor is Fisher’s
“Statistical Methods for Research Workers” [of 1925]. The distinction between
the two is that Fisher is usually right for the wrong reasons, whereas Jeffreys
gets the reasoning broadly correct, as well as the answers.
Good has some
philosophical remarks on how Jeffreys failed to notice other writers, including
“Pioneers often ignore the work of those who have stood on their shoulders” and
“if Jeffreys had tried to bring the book up to date, rather than to revise it
in part, we might have had much longer to wait for the present edition.”
_____________________________
Afterwards back to
contents
Jeffreys’s influence on Savage and
Schlaifer was not great but soon there were works that did reflect his
influence, including
Apart from Arnold Zellner ET interview
the most forceful advocate of Jeffreys’s ideas has been the physicist E. T. Jaynes (1922-98). His
magnum opus is
Of Jeffreys’s contributions the one that
found most application and criticism was his rule for constructing
uninformative priors. Invariance and the paradoxes associated with the improper
priors that application of his rule generates are discussed by Hartigan and
Dawid et al. The review by Kass & Wasserman follows the evolution of
Jeffreys’s thought as well as considering later work
·
P.
Dawid, M. Stone & J. V. Zidek (1973) Marginalization Paradoxes in Bayesian
and Structural Inference, (with discussion) Journal of the
Royal Statistical Society. B, 35, 189-233.
(available through JSTOR)
·
Robert E. Kass &
Larry Wasserman (1996) The Selection of Prior Distributions by Formal
Rules, Journal of the American Statistical Association, 91,
1343-1370. (available through JSTOR)
Seventy
years after the publication of the first edition of Theory of Probability the journal Statistical Science had a
symposium
Other links
·
The
International Society for Bayesian Analysis (ISBA)
·
Peter
Lee’s History
of Statistics has many useful links.
·
My
Figures from
the History of Probability and Statistics sketches
the history of probability and statistics and has notes on some key people.