Information theory applications

Information theory began as a theoretical science. Nevertheless, its insights led to a revolution in the design of digital transmission and storage systems. Three major areas that have directly benefited from information theory are transmission systems, storage systems, and the Internet.

The first transmission systems to be influenced by information theory were spacecraft communication systems. Since the late 1960s, long-range space probes, such as Pioneer, Voyager, Galileo, and Cassini, have used digital communication systems enhanced by advances made through research in information theory. In the early days of space exploration, transmission speeds for signals from distant probes were measured in tens of bits per second. Thanks in large part to information theory, these rates have increased to hundreds of thousands, and in some cases millions, of bits per second. Computer modems have also benefited from information theory. In the 1980s modems often operated at speeds no faster than 300 bits per second. By the late 1990s, modems routinely operated at speeds up to 56,000 bits per second.

Computer storage systems are also designed using the guidelines provided by information theory. The random-access memory, or RAM, in modern computers would be impossible without error-control coding designed by information theorists. High-capacity hard disks and CD ROMs are similarly protected. Many of today’s consumer electronic devices would also be impossible without information theory. Recording engineers have used concepts such as channel capacity and entropy to guide the design of compact disc, DAT (digital audio tape), and DVD (digital video disc) players and recorders.

The Internet and the World Wide Web are computer networks that store and transmit large amounts of data. Sending and receiving large amounts of information accurately over these networks require large amounts of channel capacity. Information theory, especially its data compression algorithms, has played a large part in making the Internet practical. For example, sending and receiving still or moving color images require large amounts of memory and would ordinarily overwhelm the capacity of the Internet. With data compression algorithms, large images can be reduced to an efficient and manageable size, making rapid exchanges of information possible.