Information Theory for Systems Engineers

ISBN-13: 9783540042549 / Angielski / Miękka / 1968 / 209 str.

L.P. Hyvarinen

Opis

This book is based on lectures given by the author at the IBM European Systems Research Institute (ESRI) in Geneva. Information Theory on the syntactic level, as introduced by Claude Shannon in 1949, has many limitations when applied to information processing by computers. But in spite of some obvious shortcomings, the underlyin~ principles are of fundamental importance for systems engineers in understanding the nature of the problems of handling information, its acquisition, storage, processing, and interpretation. The lectures, as presented in this book, attempt to give an exposition of the lovical foundation and basic principles, and to provide at the same time a basis for further study in more specific areas of this expan1in~ theory, such as coding, detection, pattern recognition, and filtering. Most of the problems in Appendix C are intended as extensions of the text, while calling for actjve participation by the stu1ent. Some other problems are direct applications of the theory to specific situations. Some problems require extensive numerical calculations. It is assumed in those cases that the student has access to a computer and that he is capable of writing the necessary programs. The stu1ent is assumed to have a good command of the calculus, and of the theory of probability as well as statistics. Therefore no basic mathematical concepts are discussed in this IV book. The Fourier transform and some related mathematical concepts are introduced in Appendix A.

Spis treści

Capter 1 Introduction.- 1.1 Properties of Information.- 1.2 Levels of Information.- 1.3 Information Media.- 1.4 Reversible and Irreversible Processes.- I.- Capter 2 Noiseless Channels.- 2.1 Character Set and Channel.- 2.2 Channel Capacity.- 2.3 Information Content, Entropy.- 3 Coding for Noiseless Channels.- 3.1 Decision Tree.- 3.2 Example of Coding and its Efficiency.- 3.3 Optimum Codes.- 3.4 End of Message Condition.- 3.5 Decipherable and Instant Codes.- 4 Joint Events, Natural Languages.- 4.1 Joint Events.- 4.2 Conditional Information.- 4.3 Natural Languages.- 5 Noisy Channel.- 5.1 Channel Matrix and Information Rate.- 5.2 Capacity of Noisy Channel.- 5.3 Ideal Observer.- 6 Error Checking and Correcting Codes.- 6.1 Hamming Distance, Parity Checking.- 6.2 Error Correcting Codes, Hamming Bound.- 6.3 Some Other Redundancy Codes.- II.- 7 Properties of Continuous Channels.- 7.1 Continuous Signals.- 7.2 Sampling and Desampling.- 7.3 Gaussian Noise.- 7.4 Entropy of Continuous Information.- 7.5 Information Rate and Channel Capacity.- Capter 8 Detection of Continuous Signals.- 8.1 Principle of Ideal Observer.- 8.2 Reception of Steady Voltage in Noise.- 8.3 Correlation Reception.- 9 Information Filters.- 9.1 Convolution Filter.- 9.2 Real-time and Off-line Filters.- 9.3 Filtering Redundant Time Series.- References.- Appendix A Fourier Transform and Related Concepts.- Appendix B Binary logarithms and Entropies.- Appendix C Problems.