Information Theory and Statistics: A Tutorial is concerned with applications of information theory concepts in statistics, in the finite alphabet setting. The topics covered include large deviations, hypothesis testing, maximum likelihood estimation in exponential families, analysis of contingency tables, and iterative algorithms with an "information geometry" background. Also, an introduction is provided to the theory of universal coding, and to statistical inference via the minimum description length principle motivated by that theory. The tutorial does not assume the reader has an in-depth...

In 1881 the American philosopher Charles S. Peirce published a remarkable paper in The American Journal of Mathematics called "On the Logic of Number." Peirce's paper marked a watershed in nineteenth century mathematics, providing the first successful axiom system for the natural numbers. Awareness that Peirce's axiom system exists has been gradually increasing but the conventional wisdom among mathematicians is still that the first satisfactory axiom systems were those of Dedekind and Peano. The book analyzes Peirce's paper in depth, placing it in the context of contemporary work, and...