Harold Jeffreys as a Statistician
The Jeffreys
prior is part of the furniture of Bayesian statistics but otherwise the
work of Harold Jeffreys is little known. Jeffreys was a noted physical
scientist who re-established the statistical theory of his time on Bayesian
foundations. This page is a guide to literature and websites which may be
useful to anyone interested in Jeffreys’s statistical work and its background.
The emphasis is on Jeffreys’s own writings and on the older literature.
John Aldrich,
Original version February
2003
Most recent changes February
2010
Links
This guide has links to some free sites, including MacTutor for
biographies and the University of
Adelaide Library for
_____________________________________________________________
Around 1930 Harold Jeffreys F.
Jeffreys gave a succinct account
of what he did in probability/statistics and why he did it in
Inverse probability was
how Jeffreys referred to the Bayesian approach; the terms Bayesian, classical,
and frequentist only became current after his main work was done. There are many relevant entries on the Earliest
known uses pages, see Probability
& Statistics on the Earliest Uses Pages: a search for Jeffreys on the front page will show on
which of the individual pages he appears and a further search on the page will
bring up all his appearances.
Earlier writers
used Bayesian methods but Jeffreys has a good claim to be considered the first
Bayesian statistician in that he used only
Bayesian methods. Earlier Bayesian writing—from authors such as Laplace, Gauss,
Edgeworth and Pearson—is described in
Jeffreys’s knowledge of this
early literature was limited and he spent some time re-inventing the wheel.
There is a short MacTutor
biography focussing on Jeffreys’s activities as an applied mathematician—it
has photographs and conveys something of his personality. There are
encyclopedia articles on Jeffreys as a statistician by Lindley and Zellner.
Cook’s (1991) memoir concentrates on Jeffreys’s contributions to physics and
applied mathematics. The festschrift edited by Zellner includes accounts of
Jeffreys’s contribution to statistics by Good, Geisser and Lindley. There are
memorial pieces in Chance including
one by Lady Jeffreys (Bertha
Swirles Jeffreys, Bertha
Swirles, Lady Jeffreys Bertha
Jeffreys), his wife and collaborator on Methods
of Mathematical Physics. (There is a photo of them together
in 1940.) Howie’s book on the controversy with Fisher discusses other phases of
Jeffreys’s life and is particularly strong on his work in the 1920s. Aldrich
looks at Jeffreys’s preparation for writing the Theory of Probability.
The Jeffreys papers at
Jeffreys wrote more than 400 papers, usually alone, on subjects ranging across celestial mechanics, fluid dynamics, meteorology, geophysics and probability: see here for a list. There is a six volume set of papers:
·
H. Jeffreys and B. Swirles (eds.) (1971-77) Collected Papers of Sir Harold Jeffreys on
Geophysics and other Sciences in six volumes,
The statistics papers are in volume 6, Mathematics, Probability & Miscellaneous Other Sciences. The coverage is not comprehensive for Jeffreys omitted papers that had been superseded by his books Scientific Inference and Theory of Probability.
Jeffreys is often described as the founder of modern British geophysics. Many of his contributions are summarised in his book The Earth. Brush (1980) describes one of Jeffreys’s main discoveries, that the core of the Earth is liquid. Jeffreys figures prominently in Brush’s three volume A History of Modern Planetary Physics (Cambridge University Press 1996).
· H. Jeffreys (1924) The Earth, further editions in ‘29, ’52, ’59, ’70 and ’76, Cambridge, Cambridge University Press.
·
E.
· B. A. Bolt (1991) Sir Harold Jeffreys and Geophysical Inverse Problems, Chance, 4, 15-17.
Jeffreys talked about his work in seismology
in an interview
with Henry Spall. Jeffreys was elected to the Royal Society in 1925 and the election
certificate shows how he was regarded at the time.
Jeffreys spent virtually his
entire life in
A.W.F. Edwards tells me that
Jeffreys’s influence on Barnard was
considerable and that Jeffreys probably influenced Alan Turing. I. J. Good does not discuss the question of
influence in his “A. M.
Turing’s Statistical Work in World War II” (Biometrika, 66,
(1979), pp. 393-396 JSTOR)
but there seem to be clear echoes of Jeffreys in Turing’s thinking.
_____________________________
Scientific Inference back
to contents
Jeffreys first used probability to
deal with problems in the philosophy of science. Jeffreys read Karl
Pearson’s Grammar of Science in 1914 and it made a great impression
on him, with its emphasis on the probabilistic basic of scientific inference.
Jeffreys treated probability as a degree of reasonable belief, an epistemic
conception common to several
Howie
is very informative on the Wrinch-Jeffreys collaboration. Aldrich (2005) also discusses this phase of Jeffreys’s
career.
Jeffreys summarised and extended the Wrinch-Jeffreys work in his book Scientific Inference.
The book was noticed in the philosophy journals. See
In
the preface to the 2nd edition
of the Probability Jeffreys explained
his relationship to the other
According
to Howie, Jeffreys met Keynes once—on a train. Jeffreys reviewed Keynes’s Treatise
on Probability, welcoming it but criticising its caution.
·
The
Theory of Probability Nature, 109, (1922), 132-133.
There
is a brief account of Jeffreys’s view of Keynes in
·
J.
Aldrich ((2008) Keynes among the Statisticians. History of Political Economy, 40, 265-316. Revised
version of Southampton University Economics Discussion
Paper 2006
Jeffreys knew Frank Ramsey (1903-1930) (N.-E. Sahlin St.
Andrews D. H. Mellor) but
did not know until after his death that he was interested in probability.
Jeffreys discusses Ramsey’s views in the Theory of Probability.
Jeffreys did not know the work of the
Italian subjectivist, Bruno
de Finetti (1906-1985), although de Finetti wrote about Jeffreys’s Scientific
Inference in a 1938 article translated as
A
recent article compares Jeffreys with Ramsey and de Finetti
Placing
Jeffreys has also been an issue in the statistical literature: see Zellner and Kass.
Philosophers
of probability have generally taken Keynes as the representative
Bayesianism is now fashionable in
philosophy of science. An early exponent building on Jeffreys was
These references relate to philosophy but Scientific
Inference took the Wrinch
& Jeffreys ideas on probability off in a new direction by considering how
physical scientists used probability when they analysed data. Chapter V on Errors includes a Bayesian derivation of
Student’s distribution.
_____________________________
The controversy with Fisher back
to contents
Around 1930 Jeffreys, now concentrating
on empirical geophysics, began devising methods for analysing data based on
epistemic probability. He was extending the methods used by physical scientists
and did not know much about, or greatly esteem, the efforts of statisticians.
Meanwhile Ronald Fisher (1890-1962), probably the most influential twentieth
century statistician, had rejected the Bayesian approach and based his work,
including maximum likelihood, on frequentist foundations. There is a fine
biography of Fisher by his daughter
For more on Fisher see A Guide to R. A. Fisher.
Fisher
and Jeffreys first took serious notice of each another in 1933. About all they
knew of each other's work was that it was founded on a flawed notion of
probability. Jeffreys (1933) criticised Fisher (1932) and Fisher (1933)
criticised Jeffreys (1932) with a rejoinder, Jeffreys (1933a). The journal
called a halt to the controversy by getting the parties to coordinate their
last words, Fisher (1934) and Jeffreys (1934). (The Fisher articles are all
available from
·
R. A. Fisher
(1932) Inverse Probability and the Use of Likelihood, Proceedings of the
·
H.
Jeffreys (1933) On the Prior Probability in the Theory of Sampling, Proceedings
of the
·
H.
Jeffreys (1932) On the Theory of Errors and Least Squares, Proceedings of
the Royal Society, A, 138, 48-55. (available through JSTOR)
·
R. A. Fisher
(1933) The Concepts of Inverse Probability and Fiducial Probability
Referring to Unknown Parameters, Proceedings of the Royal Society, A, 139,
343-348. (available through JSTOR)
·
H.
Jeffreys (1933a) Probability, Statistics, and the Theory of Errors, Proceedings of the Royal Society, A, 140,
523-535 (available through JSTOR)
·
R. A. Fisher
(1934) Probability, Likelihood and the Quantity of Information in the Logic
of Uncertain Inference, Proceedings of the Royal Society, A, 146,
1-8. (available through JSTOR)
·
H.
Jeffreys (1934) Probability and Scientific Method, Proceedings of the Royal
Society, A, 146, 9-16. (available through JSTOR)
·
J. Aldrich (2008)
Howie gives a full account of the dispute but Lane’s brief account is still useful. There is an even briefer account in Aldrich (2004). Seidenfeld considers how Jeffreys and Fisher used Keynes’s work in the debate.
Aldrich
(2005) discusses the controversy but goes beyond it to consider how
Jeffreys treated Fisher’s ideas in the Theory of Probability. Fisher did
not mention modern work on inverse probability in his Statistical Methods for Research Workers (see Likelihood
and Probability in Fisher’s Statistical Methods ) although Jeffreys
asked him to—see correspondence
p. 170.
Fisher
and Jeffreys never accepted the validity of each other’s approach but their
relationship mellowed into one of relaxed toleration. Their developing
relationship can be followed in their
letters which have been printed with notes in Bennett. There are more
personal memories in Swirles and Box.
·
J.
H. Bennett (1990) (ed.) Statistical
Inference and Analysis: Selected Correspondence of
·
G.
A. Barnard (1992) Review of Statistical
Inference and Analysis: Selected Correspondence of
_____________________________
Theory of
Probability back to contents
In the years following Scientific Inference Jeffreys worked hard on statistics. In 1937 an enlarged edition of the book came out but he soon judged this to be inadequate, telling Fisher in September 1938, “I wish some public benefactor would subsidize [the publisher] to scrap Scientific Inference and give me a chance of writing something up to date.” Bennett (p. 170)
In 1939 he had a new book with a
different publisher that incorporated the papers he had been writing on
statistics. The Theory of Probability
by Harold Jeffreys, Reader in Geophysics,
Jeffreys used the phrase “theory of probability” in a unique way—to refer to a theory of inductive inference founded on the principle of inverse probability. The Probability is largely a treatise on theoretical statistics, though it begins with the foundations of probability and covers a range of applications comparable to that in Fisher’s Statistical Methods for Research Workers. Jeffreys was impressed by the solutions Fisher produced. In May 1937 he told Fisher of his impression that “only once in a blue moon” would we disagree about the “inference to be drawn in any particular case.” Bennett (p. 162). The trouble with Fisher’s inferences was that they lacked proper foundations!
The scope of the book
In all its versions the Probability was a book of 8 chapters and appendices. The titles of the chapters and their general scope remained unchanged.
I Fundamental notions explains how induction rests on probability and presents axioms for probability. This is the chapter with the strongest links with Scientific Inference although the treatment is much more elaborate. Whitehead and Russell’s Principia Mathematica is a powerful presence.
II Direct probabilities presents basic distribution theory, including the standard distributions and transform techniques. The term “direct” contrasts with “inverse.” The former indicates concern with inference from “laws” to observations, the latter with the converse.
III Estimation problems are ones “where we are given the form of the law, in which certain parameters can be treated as unknown, no special consideration needing to be given to any particular values, and we want the probability distribution of these parameters given the observations.” (1939, p. 94). The statement (p. 96), “Our first problem is to find a way of saying that the magnitude of a parameter is unknown, when none of the possible values need special attention” leads into a discussion of ignorance priors. Jeffreys then went through some standard problems. The discussion
IV Approximate methods and simplifications covers maximum likelihood, minimum χ2 and other non-Bayesian arguments. Jeffreys treats them in a remarkably positive spirit.
V Significance tests: one new parameter. In significance testing “our problem is to compare a suggested value of a new parameter, often 0, with the aggregate of other possible values.’’ (1939, p. 193) Jeffreys was the first writer in the inverse probability tradition to write in detail about testing. In the preface he (p. v) complained, “Modern statisticians ... for the most part have rejected the notion of the probability of a hypothesis, and thereby deprived themselves of any way of saying precisely what they mean when they decide between hypotheses.”
VI Significance tests: various complications treats various situations do not conform to the one new parameter format.
VII Frequency definitions & direct methods treats frequency definitions of probability—Venn, von Mises etc.—and the “direct methods” (frequentist methods) of Karl Pearson, Fisher and Neyman-Pearson.
VIII General questions recapitulates the main arguments and considers some broader issues. On this chapter and the preceding one Wilks (1941, p. 194) commented, “The discussion is almost entirely informal and non-mathematical and as such it must be regarded in the category of personal opinion.”
Appendices Appendices came and went but the one constant was the set of Tables of K. K was Jeffreys’s symbol for what Good later called the “Bayes factor.” The introductory material explains the use of K and discusses the relationship between inferences based on K and on conventional tail areas—the “P integral”, as Jeffreys called it.
In its final form—the corrected impression (1966) of the third edition—the Probability had 459 pages compared to the first edition’s 380. The growth of the book can be seen from the table of contents of successive editions.
|
Chapters\Editions |
3rd |
2nd |
1st |
|
I. Fundamental Notions |
p. 1 |
p. 1 |
p. 1 |
|
II. Direct Probabilities |
57 |
47 |
46 |
|
III. Estimation Problems |
117 |
99 |
94 |
|
IV. Approximate Methods and
Simplifications |
193 |
168 |
145 |
|
V. Significance Tests: One New Parameter |
245 |
220 |
193 |
|
VI. Significance Tests: Various
Complications |
332 |
305 |
268 |
|
VII. Frequency Definitions and Direct
Methods |
369 |
341 |
300 |
|
VIII. General Questions |
401 |
372 |
332 |
|
Appendices |
425 |
396 |
356 |
|
Index |
455 |
408 |
377 |
A detailed table of contents of the third edition has been prepared by Peter M Lee.
Jeffreys wrote articles but most of their substance went into the big book. Thus overall assessments of his work like those in the festschrift are essentially assessments of the Probability. Zellner and Kass discuss Savage’s way of classifying Jeffreys. Lindley reflects on the book and Aldrich (2005) describes the first edition against the background of Jeffreys’s intellectual development. Aldrich (2002) mentions how Jeffreys translated some of Fisher’s ideas into Bayesian terms.
·
D.
V. Lindley (1986) On Re-reading Jeffreys, pp. 35-46 of I. S. Francis et al
(eds) Pacific Statistical Congress, New York: Elsevier.
·
J.
Aldrich (2002) How Likelihood and Identification went Bayesian, International Statistical Review, 70, 79-98.
For wider perspectives see
·
Jim
Berger Could
Fisher, Jeffreys and Neyman Have Agreed Upon Testing? This Fisher memorial
lecture has been published (with discussion) in Statistical Science, 18,
1-32 (2003) Euclid JSTOR.
The next sections give some details about the three editions and how they were received. Of the three editions the first was the most important for presenting the main ideas. The second edition introduced Jeffreys’s rule the topic in the Probability which has attracted most attention. The third edition appeared at the most propitious time.
_____________________________
The first edition 1939 back to contents
In 1939 Jeffreys was up-to-date with the work of the English statistical school, familiar with the work of Karl Pearson, Student, Fisher and Neyman-Pearson. Like most other English writers, he did not know the recent Continental work on probability, notably Kolmogorov’s Grundbegriffe. He knew only H. Cramér’s Random Variables and Probability Distributions (1937) which he recommended for a rigorous treatment of the central limit theorem.
In the preface (reproduced in the
current edition) Jeffreys thanks Fisher, Wishart
and the philosopher Richard
Braithwaite for their help. None agreed with him on probability. In the
text he mentions the
The first edition of the Probability was reviewed in the leading journals by prominent statisticians; Irwin had worked with both Karl Pearson and Fisher while Neyman (1894-1981) (ASA St Andrews) and Wilks (1906-64) (ASA.) would both be leaders of post-war statistics in the USA. The reviewers respected Jeffreys’s scientific credentials but were sceptical of his system.
· J. Neyman (1940) Review of Theory of Probability by Harold Jeffreys, Journal of the American Statistical Association, 35, 558-559. (available through JSTOR)
· J. O. Irwin (1941) Theory of Probability by Harold Jeffreys, Journal of the Royal Statistical Society, 104, 59-64. (available through JSTOR)
· S. S. Wilks (1941) Theory of Probability by Harold Jeffreys, Biometrika, 32, 192-194. (available through JSTOR)
As well as these reviews by statisticians there were reviews by mathematicians and philosophers.
As far as statistics was concerned, Wilks’s words
seemed set to be prophetic: “It is doubtful that there will be many scholars
thoroughly familiar with the system of statistical thought initiated by
_____________________________
The second edition 1948 back to
contents
The flow of articles on statistics abated after the publication of the Probability and in 1940-5 Jeffreys published only three articles on statistics. In 1946, however, came a major development, the introduction of the Jeffreys prior as it is usually called.
In 1948 Jeffreys brought out a second edition of the Probability. He was now Plumian Professor of Astronomy in succession to Arthur Eddington who had taught him the theory of errors as an undergraduate. The main change in the book was an account of the invariant prior: see the preface. Jeffreys had been interested in invariance for a long time. In Scientific Inference he had discussed the consistent assignment of priors to the standard deviation, σ, and the modulus, h = 1/σ√2 , a measure of precision often used in the theory of errors.
In 1939 Jeffreys had sought to reconstruct modern statistics on the probability foundations he first proposed in 1919. After 1939 he did not follow the literature, nor was he impressed with the way the literature treated him. In the preface to the second edition he wrote, “I have not attempted to answer explicitly the criticisms made by reviewers, because on examination I found that they were all dealt with in the book already.”
The second edition was not widely reviewed.
Robbins and David were respectful but sceptical. Robbins noted that Jeffreys did
not discuss the costs of making errors while David found the approach
necessarily subjective. Neither was interested enough in Jeffreys’s project to
comment on the changes in the new edition.
_____________________________
The third edition 1961 and the Bayesian revival back to contents
Between 1948 and 1960 Jeffreys published very little on probability: reviews of philosophical works by Reichenbach, Russell and Carnap, an account of the “present position” and a note on the exponential family in 1960; see bibliography. The Russell review was the most elaborate of the reviews and the BJPS article described the present position as it was in the second edition of the Probability.
· Harold Jeffreys (1950) Bertrand Russell on Probability, Mind, 59, 313-319 (available through JSTOR).
· Harold Jeffreys (1955) The Present Position in Probability Theory, British Journal for the Philosophy of Science, 5, 275-289 (available through JSTOR)
In 1961 Jeffreys produced a third edition of the Probability. He had retired from his chair but he went on working until his death in 1989. This edition added material to the first chapter and some mathematical appendices. The only significant new reference was to Carnap’s Logical Foundations of Probability which he reviewed in 1952. In the corrected impression of 1966 material on time series analysis was added.
In its third
edition the Probability remained a conversation with Fisher and Principia Mathematica. Raiffa and
Schlaifer’s Applied Statistical
Decision Theory was
published in the same year. That work reflected a huge change in the
statistical climate as more statisticians became interested in Bayesian ideas.
There were many publications that contributed to the Bayesian revival. In
Britain Good (ASA
conversation)
was influenced by Jeffreys and by Cambridge probability more generally; like
Jeffreys and unlike most statisticians, Good was interested in logic and the
philosophy of science. The more influential US line of Savage
and Schlaifer was
very different. Their foundations added to the “personalism” (subjective
probability) of Ramsey
and de Finetti the “behavioralism” (decision-orientation) of Neyman and Wald;
the expected utility theory of von
Neumann and Morgenstern was also a great influence. Jeffreys was known to
the American statisticians but was not much of an influence. For Savage (1954,
p. 276) the Probability was “an ingenious and
vigorous defense of a necessary view, similar to, but more sophisticated than
Jeffreys seems to have paid no
attention to these developments in the statistical literature; his expected
utility was the moral expectation of
The third edition, like the
second, was not widely reviewed. However, for the first time there were
Jeffreys enthusiasts to receive it.
·
D. V. Lindley (1962), Theory of Probability. by Harold Jeffreys, Journal of the
American Statistical Association, 57, 922-924 JSTOR:)
Good
and Lindley wrote in very different terms from earlier reviewers. Lindley’s
review begins
This is probably the most original and
important book in statistics that has appeared in the last forty years. The
only serious competitor is Fisher’s “Statistical Methods for Research Workers”
[of 1925]. The distinction between the two is that Fisher is usually right for
the wrong reasons, whereas Jeffreys gets the reasoning broadly correct, as well
as the answers.
Good has some philosophical remarks on how Jeffreys failed to
notice other writers, including “Pioneers often ignore the work of those who
have stood on their shoulders” and “if Jeffreys had tried to bring the book up
to date, rather than to revise it in part, we might have had much longer to
wait for the present edition.”
_____________________________
Afterwards back to contents
Jeffreys’s
influence on Savage and Schlaifer was not great but soon there were works that
did reflect his influence, including
Apart
from Arnold
Zellner ET
interview the most forceful advocate of Jeffreys’s ideas has been the
physicist E. T. Jaynes
(1922-98). His magnum opus is
Of
Jeffreys’s contributions the one that found most application and criticism was
his rule for constructing uninformative priors. Invariance and the paradoxes
associated with the improper priors that application of his rule generates are
discussed by Hartigan and Dawid et al. The review by Kass & Wasserman
follows the evolution of Jeffreys’s thought as well as considering later work
·
P.
Dawid, M. Stone & J. V. Zidek (1973) Marginalization Paradoxes in Bayesian and
Structural Inference, (with discussion) Journal of the
Royal Statistical Society. B, 35, 189-233.
(available through JSTOR)
·
Robert
E. Kass & Larry Wasserman (1996) The Selection of Prior
Distributions by Formal Rules, Journal of the American Statistical
Association, 91, 1343-1370. (available through JSTOR)
Seventy years after the
publication of the first edition of Theory of Probability the journal Statistical Science had a symposium
Other
links
·
The International Society for Bayesian Analysis (ISBA)
· Tom Loredo’s Bayesian Inference for the Physical Sciences.