This is the third reading one of my all time favorite technical books. It's not a rigorous text, but the explanations are nevertheless compelling (for the most part). It defines information (via the entropy equation), channels; derives channel capacity for discrete and analog channels; and discusses detection as a communication process. [Actually these are chapter titles.] It also contains some useful tidbits of information (for example, standard text contains about 1 bit per character of information), as well as a useful (but not very extensive) bibliography.
The book is copyrighted 1964 by MIT and published by the MIT Press. The book is dedicated to George Polya and the dedication implies that GP was his teacher (in a direct sense).
Section 1.1 titled "Why a New Definition?" is my favorite part, because it brings up one of my interests: definitions in context. All five paragraphs of this section are worth reading, but the third is particularly appropriate (because it emphasizes what I think needs to be emphasized):
"When a new technical concept is named with a common word, the word acquires a new meaning. It is impossible to use the word in a technical context until that new meaning has been defined. Pressing a suit does not mean the same thing to a lawyer that it does to a tailor. And information does not mean the same thing to a communications engineer that it does to a police detective. There is no reason to expect anyone to know what the word information means to an information theorist unless he has been told."
I love this. If everyone read this and thought about it and its repercussions, a lot of arguments would evaporate. There are lots of examples of this phenomenon. 'Order' means different things to a dictator, a judge, an artist, a waiter, and a thermodynamicist. Before deciding this is a trivial observation, consider that one of the arguments put forth by the scientific creationists relies on just such an equivocation of the definitions of 'order.' In fact, a chemist friend of mine went so far as to suggest that scientists should just not use the word 'order' when referring to the second law of thermodynamics, because laymen are bound to misunderstand it (and creationists bound to misrepresent it). One might as well use the layman's notion of 'velocity' and 'acceleration' to disprove the first law of thermodynamics! Ask a layman to list ways to accelerate his car, for example. He'll give you a strange look and say, "What kind of an idiotic question is that? You've got no common sense! Of course, you step on the gas! Why do you think they call it an ACCELERATOR, Stupid?" But you ask a scientist or engineer the same question and you'll get at least the following (barring any sneaky answers like "Run into a brick wall" or "Shift weight"): Step on the gas (positive acceleration). Step on the break (negative acceleration). Turn the wheel! Yes! Turn the wheel. That's because speed is not the same thing as velocity to an engineer. To a technical person, velocity is a speed AND a direction. Acceleration is not a change in speed, but a change in velocity. So changing either the speed or the direction of one's car is an acceleration.
There are other examples of this, but the more startling fact for some people is the observation that the same word can have different definitions in different technical contexts....to different technical people in different fields, and to different technical people in the same or similar fields!
I'll start with a simple example. The 'K' prefix which means 'kilo-'. Any computer scientist will tell you that 'K' means 1024 and that 'M' means 1024*1024. This is (one reason) why when you get your 200MByte hard drive, and look at the amount of space available, it doesn't equal 200,000,000. (The other reason is that formatting a drive can affect how much is available.) 70 K is 70 times 1024. 100 Meg is 100 times 1024 times 1024. When you read about jobs offereing salaries for 50K in the paper, though, they mean 50*1000, or $50,000. What do you think 'K' means to a communications engineer? The same as it means to a computer scientist (1024) or to the personnel director (1000)? If you guessed the personnel director (1000), you're right. [In fact, there are a few books written by popular quasi-technical authors who get this wrong.]
Do you think baud rate and bits per second are the same? Do you think your modem is a 14.4 Kilobaud modem? You're probably wrong. For an excellent discussion of the distinction, check out Stallings' book Data and Computer Communications , for example. I think there's also a discussion in Shay's Understanding Data Communications and Networks . Baud rate is signaling rate. Imagine you're sending messages to someone across a room. You're holding up a piece of paper with either a 0 or a 1 on the paper and you're raising the paper once per second. You'ld be transmitting 1 bit per second and your baud rate is 1. Now imagine you're holding the paper up at the same rate, but instead of putting one digit each time, you're putting up three each time. Then your data rate is 3 bits per second, but your baud rate is still 1! Your modem achieves a similar effect by using a a thing called QAM (Quadrature Amplitude Modulation).
There are other examples of this that I've collected through the ages, ranging from definitions of 'atheist,' 'whole number,' 'right hand rule,' and 'verification.' The odd thing is that everyone using the term insists that their (and ONLY their) definitions are the correct ones. I'm really going to have to sit down and write this essay some time, but not tonight. In any case, Gordon Raisbeck should be commended for bringing the issue to light.
