The Standing Invitation

Posts Tagged ‘Information Theory

A Bit of News

leave a comment »

Information is news. But what is information? A note, written in pencil on a page, contains information; but information is not made out of graphite deposited on cellulose. A TV broadcast is information; but information is not made out of radio waves, and it is not the vibration in the air molecules between the radio and your ear.

It is formless, shapeless, intangible. Nevertheless, it can be measured, quantified, treated mathematically.

Information reduces uncertainty. Toss two coins in the air – a penny and a pound coin, say ­– and catch them in your hand without looking at them. You are uncertain about whether they are heads or tails. This uncertainty is measurable: if you had to guess the outcome of the toss, you’d have a 25% chance of getting it right.

But you peek at your hand. You see that the pound coin has come up tails, but you can’t see the penny. With this new knowledge of the state of one of your coins, you have a 50% chance of guessing right. The information has doubled your chances; the uncertainty has dropped by half.

Information is measured in bits. One bit, short for binary digit, lowers your uncertainty about the world by one half. It reduces the number of yes/no questions you have to ask in life by one.

Note that the information content of a fact, measured in bits, depends on how uncertain you were to begin with. If I tossed a double-headed coin and told you it came out heads, that statement contains no information. It was always going to happen; no uncertainty reduced. Conversely, if I tell you I rolled a die and it came out six, that fact is worth 2.6 bits ­– more than one bit, because one yes/no question would not have been enough to remove the uncertainty. Identifying a card pulled from a pack of 52 cards is worth 5.7 bits.

How much information is contained in the expression “It will rain in the UK tomorrow”? Not much ­– the answer isn’t surprising. You could have guessed anyway.

After a week trapped in a mineshaft, a woman is rescued. The news report quotes her as saying “She is glad to be out.” Not much information. You would be surprised to hear her say, “Actually I preferred it down there.” That would be information.

But if that was what she’d said, would the journalist report it? Information-rich or not, does it make a good story? Or would the journalist have quietly turned off the camera, choosing not to show those bits?

The tanks of liberation roll into a bombed-out city. A flak-jacketed war correspondent films crowds of people welcoming the brave soldiers who have freed them, cheering them on. Surely very surprising, very information-rich. But how photogenic would the opposite case have been? A crowd of angry shopkeepers whose lives have been wrecked ­– would that footage be aired? Perhaps not. Regardless of what happened, it is likely that any images shown will be positive. Positive, and unsurprising. Information content low.

Information is news – but not all news is information.


Information theory stuff from Richard Dawkins’s essay “The Information Challenge” as found in The Devil’s Chaplain; definitely worth a read.


Written by The S I

August 19, 2011 at 11:59 pm