Shannon Entropy: Information Is Not Intelligence

Imagine an AI assistant that produces a thousand different answers to the same question. Another assistant produces one answer consistently. Which is more intelligent?

The question is incomplete. We need to know whether the answers are correct, whether the question permits several solutions, and what the person asking is trying to achieve. Variety alone cannot settle any of those issues.

Shannon entropy gives us a precise way to discuss uncertainty. Its precision makes the boundaries of the idea especially useful when thinking about AI.

What Shannon entropy measures

In his 1948 paper, Claude Shannon separated the engineering problem of transmitting messages from their semantic meaning. For a discrete probability distribution, entropy measures average uncertainty: H = -sum(p × log2(p)). Using base-two logarithms gives bits. Here, each p is an outcome's probability. This definition describes a distribution, rather than the importance of a particular message. Shannon's original paper, introduction and section 6.

For a coin with known probabilities:

Fair coin: -0.5 × log2(0.5) - 0.5 × log2(0.5) = 1.

Biased coin: -0.9 × log2(0.9) - 0.1 × log2(0.1) ≈ 0.469.

The biased coin is more predictable on average, even though a rare tail can surprise us. The probabilities are assumed known.

A bit can matter enormously

Consider a deliberately simple thought experiment. A laboratory publishes one of two equally likely messages: an experiment succeeded, or it failed. Separately, a toy generates one fair random binary digit. Both sources have one bit of entropy per message under these assumptions.

For a researcher deciding what to build next, the messages can have very different value. The laboratory result might determine the next six months of work. The toy's output might change nothing.

Now give the researcher a task that requires a random choice between two equally acceptable experimental orders. The toy becomes useful. Its usefulness comes from the job we assign it.

This is why attaching a practical interpretation to a mathematical quantity requires care. To evaluate a message's value, specify the recipient, the available decisions, and the consequences. Those details were absent from the entropy calculation. Adding them changes the question we can answer.

Turing's useful change of question

Alan Turing's 1950 essay approached machine intelligence by replacing an ambiguous philosophical question with an imitation game involving written exchanges. The proposed setup made particular behavior available for investigation. It did not introduce an entropy threshold for intelligence. Turing's paper in Mind; accessible university copy.

My practical takeaway is to specify what would count as evidence before choosing a score. If an assistant's purpose is to help someone complete a task, an evaluation should make success observable. A fluent explanation and a completed task can coincide, but an evaluator needs a way to distinguish them when they diverge.

That distinction becomes concrete as soon as the assistant can act on a computer.

An agent evaluation thought experiment

Suppose we are comparing agents that organize a folder of research papers. Each must identify duplicates, preserve annotations, and produce a correctly grouped library.

Agent A writes lively explanations and takes a different sequence of actions every time. Agent B uses a predictable sequence. Either might succeed. Either might silently delete an annotated copy. The diversity of their action sequences does not tell us which preserved the user's work.

A useful evaluation would inspect the resulting files. Were all distinct papers retained? Did every annotation survive? Were unrelated documents left alone? If the agent reports uncertainty about a duplicate, does it ask for clarification before making a consequential change?

For this hypothetical task, I would track four things:

• Outcome: the library satisfies the requested organization and preservation rules.
• Reliability: repeated runs succeed across different folder layouts and naming conventions.
• Recovery: interrupted or failed actions can be detected and corrected.
• Cost: successful completion requires an acceptable amount of time and interaction.

These are proposed design choices for this example, not results from a benchmark. They make the evaluation answerable to the task. They also expose disagreements about what success means before those disagreements disappear inside an aggregate score.

Ask what uncertainty belongs to

Even the phrase “agent entropy” needs a definition. Are we measuring the distribution of next actions in one state, complete trajectories across repeated runs, or success and failure across tasks? Those are different random variables.

Changing the grouping can change the answer. If we count every filename spelling as a distinct action, harmless differences may dominate. If we group actions by their effect, the same runs can look much more consistent. A comparison should declare its categories and sampling procedure before interpreting the number.

This connects to a broader question about why mathematics describes physics: a model becomes useful through a disciplined relationship between symbols and observations. It also matters when discussing a possible technological singularity. A claim about increasing intelligence needs an account of which capabilities are improving and how we would notice their failure.

For the folder agent, the next useful experiment is straightforward: change the documents, preserve a known answer, and inspect what happens. A mathematical score earns its place when it helps explain that result or choose a better next experiment.

Further reading

Start with Shannon's introduction and section 6 in A Mathematical Theory of Communication, then the opening sections of Turing's Computing Machinery and Intelligence. Read each alongside its stated question; the differences are as instructive as the connections.