Information Theory: A Tutorial Introduction (2nd Edition)

Information Theory: A Tutorial Introduction (2nd Edition)

Originally developed by Claude Shannon in the 1940s, information theory laid the foundations for the digital revolution, and is now an essential tool in telecommunications, genetics, linguistics, brain sciences, and deep space communication. In this richly illustrated book, accessible examples are used to introduce information theory in terms of everyday games like ‘20 questions’ before more advanced topics are explored. Online MatLab and Python computer programs provide hands-on experience of information theory in action, and PowerPoint slides give support for teaching. Written in an informal style, with a comprehensive glossary and tutorial appendices, this text is an ideal primer for novices who wish to learn the essential principles and applications of information theory. This Second Edition includes two new chapters on on the theory and applications of Transfer Entropy and Rate Distortion Theory. "This is a really great book. Stone has managed to distil all of the key ideas in information theory into a coherent story. Every idea and equation that underpins recent advances in technology and the life sciences can be found in this informative little book." Professor Karl Friston, Fellow of the Royal Society. Scientific Director of the Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London. "Its tutorial approach develops a deep intuitive understanding using the minimum number of elementary equations. Thus, this superb introduction not only enables scientists of all persuasions to appreciate the relevance of information theory, it also equips them to start using it." Simon Laughlin, Professor of Neurobiology, Fellow of the Royal Society, Department of Zoology, University of Cambridge, England. "Textbooks on information theory can seem impenetrable to those outside the discipline. Stone's tutorial treatment provides a much needed introduction, which explains relevant details while maintaining the integrity of the topic. This book should be useful to students and researchers in related scientific fields, including machine learning and biological signal analysis." Jerry Gibson, Distinguished Professor, University of California, USA. This Second Edition is identical to the First Edition, except for the addition of two new chapters: Rate Distortion Theory and Transfer Entropy. Claude Shannon introduced the idea of Rate Distortion Theory in his original book published in 1949 ("The Mathematical Theory of Communication"), and he followed this with a complete theory published in 1959 ("Coding Theorems for a Discrete Source with a Fidelity Criterion").  More recently, information theory has been applied to the problem of inferring causality, in the form of Transfer Entropy. The underlying theory of both Rate Distortion Theory and Transfer Entropy are explained in the context of recent applications in signal processing and biology. Reader reviews for the First Edition can be seen by searching Amazon for ISBN 1739672704. James V Stone is an Honorary Associated Professor at the University of Sheffield, UK.