Information Theory Research Report
Leah Clarke
For my research report, I read excerpts from The Mathematical Theory of Communication
by Claude Shannon and Warren Weaver. This book is an in-depth description of their theory.
While I will focus mostly on Weaver's translation and application of information, I will also
touch on the theory's core ideas as explained by Shannon.
The information theory is the extentsion of Nyquist's and Hartley's origingal ideas on the subject.
However, Claude
Shannon includes new factors such as "the effect of noise in the channel, and savings possible
due to statistical structure of the original message and due to the nature of the final destination of
information." Shannon's ideas were based on the fundamental problem in communication, which he
described as the "difficulty of reproducing at one point the message selected at another point."
The most significant aspect is that the actual message sent is one "selected from a set" of possible
messages. A system had to be formed to work for each possible selection. Shannon also constructed
the
Linear Model of Communication. It is rare to see a communication text book that doesn't include this model, or
a model that is based on this one. However, modifications had to be made because Shannon and Weaver both
overlooked the importance of feedback.
Warren Weaver helps us understand Shannon's complex theory by explaining it in layman's terms.
Weaver uses the word "communication" in a broad sense to incorporate all the different ways one
person's mind could affect another. Basically, it encompasses all human behavior.
Weaver simplistically describes the three levels of communication problems.
The first level, Level A, is the technical problem. The technical problems deal with the transfer
of sets of signals from sender to sender. These sets of signals could be as simple as written speech
or as complex as telephone or television transmission.
The second level (Level B) is concerned with the understanding and interpretation of the meaning by
the receiver when compared with the intended meaning of the sender. Level B is referred to as
semantics problems.
Level C, the effectiveness problems, is the final communication problem. It is concerned with how
well the meaning is conveyed to the receiver.
Weaver goes on to describe numerous problems within each of the previous three. One of the major
problems in each of these categories was that of entropy. Entropy may be simply defined as randomness.
Significantly reducing entropy by using bits of information, will result in a clearer understanding of meanings that
can be reached. This concept at first is challenging to grasp, yet it is simplistic when broken down.
Entropy is an issue in everyone's life daily. It is similar to problem solving. You are given a fact at
a time and it narrows your scope of thinking down so that you can come up with the correct solution.
"The concept in this theory at first seems disappointing and bizarre- disappointing because it has
nothing to do with meaning, and bizarre because it deals not with a single message but rather with
the statistical character of a whole ensemble of messages..." This is Weaver's own quote about the
somewhat confusing information theory. He goes on, however, to say that this should be only a
temporary reaction.
Shannon and Weaver's theory was centered around the belief that communication systems should try to
incorporate all possible messages. Also, Shannon and Weaver uphold the idea that the base of
communication systems should be the source.
Shannon's mathematical explanations of the information theory were very difficult to understand,
but Weavers more clear-cut descriptions offered insight into a rather complicated theory. I believe
the theory's major contributions were the linear model of communication as well as the way the theory was
broken down to describe communication problems and solutions.
Works Cited
Shannon, C., & Weaver, W. (1949). The Mathematical Theory of Communication. Urbana: University of Illinois
To see Marks application of this theory, click here.
For a critique of the information theory, check out Steve's page.
Back to Steve's homepage.
Visit a site on Information Systems
in Organizations which also deals with the transmission of information.