The first team is almost invisible.

Its pipelines run every night without incident. Automated tests catch bad records before they reach production. Duplicate records are removed before they contaminate downstream systems. Schemas evolve without breaking dashboards. Machine Learning models receive clean, reliable features.

Analysts trust the warehouse.

Executives do not question the KPIs in Monday morning reports.

Stakeholders rarely complain because there is little to complain about.

No executive escalations.

No late-night incident bridges.

No frantic emails asking:

“Can we trust the data?”

People simply assume everything works.

Now consider the second team.

Pipelines fail regularly.

Delivery timelines slip week after week, then quarter after quarter. Dashboards miss important meetings. Duplicate records inflate KPIs. Finance and Sales produce reports that contradict one another. Source systems deliver inconsistent fields, missing values, and raw data that is technically available but practically unusable.

Analysts retreat to manual spreadsheets because they no longer trust the warehouse.

Data Engineers spend their days patching brittle pipelines. Data Scientists spend more time cleaning data and debugging feature jobs than building models.

Every delay produces a workaround.

Every workaround creates another dependency.

Soon, the organization is no longer operating a data platform. It is operating a collection of exceptions held together by memory, fragile spreadsheets, and human heroics.

Business leaders demand explanations.

Engineers cancel evenings and weekends.

Escalations become routine.

Eventually, so does burnout.

After another exhausting weekend, the platform is restored.

On Monday morning, leadership publicly thanks the team for its extraordinary dedication.

The recovery becomes a success story.

The manager develops a reputation for thriving under pressure.

A promotion follows.

Perhaps two.

Nobody asks why pressure seems to thrive under the manager.

Before rewarding the firefighter, it is reasonable to ask why the building keeps catching fire.

Now ask yourself:

Which team created more value?

Most organizations would answer the first.

Many organizations unintentionally reward the second.

Not because leaders are irrational.

Because humans are remarkably good at recognizing problems that happened and surprisingly poor at appreciating the problems that never did.

The better a team becomes at preventing failure, the fewer opportunities it has to demonstrate how valuable it really is.

Prevention creates counterfactual value: value that exists in the difference between what happened and what would have happened without the intervention.

Unfortunately, performance reviews take place in only one of those timelines.

The Action Fallacy

Historian Martin Gutmann calls this bias the Action Fallacy: our tendency to equate leadership with visible action while overlooking leaders who reduce the need for it.

It is easy to see why.

Crises are optimized for storytelling.

They have conflict.

They have stakes.

They have uncertainty.

And they usually have a visible protagonist standing in front of a red dashboard.

Prevention has poorer narrative architecture.

No villain.

No dramatic reversal.

No exhausted leader announcing that the team has finally “turned a corner.”

Just a checklist someone completed six months earlier.

Nobody writes a bestseller about a database migration that finished exactly as planned.

Nobody gives a standing ovation because an Airflow DAG succeeded for the 700th consecutive day.

Nobody posts on LinkedIn to celebrate another quarter in which the data remained accurate and everyone went home on time.

That does not mean leadership during a crisis is unimportant. When systems fail, decisive action matters. Courage under pressure matters.

Recovering from avoidable pressure and reducing the amount of avoidable pressure are different forms of competence.

Organizations tend to notice the first because it creates an event.

The second removes the event entirely.

A Tale of Two Expeditions

More than a century before cloud platforms and data warehouses, Roald Amundsen understood the value of preparation.

His objective was not simply to reach the South Pole.

It was to remove as much uncertainty as possible before the expedition began.

He studied earlier polar expeditions. He learned from Indigenous Arctic communities. He selected equipment proven to work in extreme conditions. He carefully planned supply depots and built redundancy into the mission.

“Victory awaits him who has everything in order—luck, people call it. Defeat is certain for him who has neglected to take the necessary precautions in time; this is called bad luck.”
—Roald Amundsen

Amundsen reached the South Pole in 1911, and every member of his polar party returned safely.

No rescue mission.

No desperate improvisation.

No legendary survival story.

Ernest Shackleton pursued a different and exceptionally ambitious Antarctic mission. His leadership after the Endurance became trapped and destroyed deserves its reputation. Guiding his crew through catastrophe remains one of history’s great survival stories.

But the contrast in how we remember the two men is revealing.

Amundsen’s expedition illustrates leadership through systematic risk reduction.

Shackleton’s illustrates leadership after risk has become catastrophe.

One produced a successful outcome.

The other produced a legend.

We tend to know which one makes better cinema.

Organizations often remember leaders the same way.

The Measurement Problem

As a Data Scientist, what interests me most is not the leadership theory.

It is the measurement problem.

Imagine trying to build a Machine Learning model that predicts leadership effectiveness.

What would your labels be?

Promotions?

Performance reviews?

Revenue growth?

Employee engagement?

Projects delivered?

Incidents resolved?

Every one of these measures is based on something that happened.

But some of the most valuable leadership decisions change events that never happen.

A well-designed data platform does not merely process data.

It prevents duplicate records from reaching Finance.

It prevents broken dashboards from reaching executives.

It prevents analysts from spending days reconciling contradictory reports.

It prevents Data Scientists from training models on corrupted features.

It prevents stakeholders from losing confidence in analytics altogether.

Those outcomes are enormously valuable.

They are also almost invisible.

This is a counterfactual problem: estimating the difference between the outcome we observed and the outcome that would have occurred under another decision.

The difficulty is that only one outcome becomes observable.

The alternative remains hypothetical, and hypothetical value is difficult to place on a promotion packet.

How much revenue was protected because incorrect data never reached Finance?

How many engineering hours were saved because a breaking schema change was caught automatically?

How many poor decisions were avoided because the dashboard was right?

We rarely know.

Dashboards measure what happened. Much of the value lives in what did not.

When Measurement Becomes Incentive

Organizations naturally measure what they can observe.

Incidents resolved.

Tickets closed.

Recovery time.

Escalations handled.

Hours worked.

These are useful operational metrics.

But they mostly measure activity after something has already failed.

Prevention leaves a weaker trail.

There may be no incident ticket for the schema failure caught during testing.

No executive email documenting the inaccurate report that was never published.

No revenue estimate for the customer trust that was never lost.

No applause for the weekend nobody had to work.

The easiest things to measure are not always the most valuable.

And measurement does not remain passive for long.

Once metrics influence promotions, budgets, and recognition, they begin shaping behavior.

When organizations reward visible recovery more than invisible prevention, people adapt.

Not necessarily because they are manipulating the system.

Because they are responding rationally to its incentives.

Firefighting becomes more visible than fire prevention.

Urgency becomes more visible than preparation.

Heroics become more visible than reliability.

The organization slowly teaches everyone the same lesson:

Being seen solving a problem may be more rewarding than quietly ensuring it never appears.

This is where Goodhart’s Law becomes relevant:

When a measure becomes a target, it ceases to be a good measure.

Reward visible recovery, and you may get more visible recovery.

Reward reliable systems, and you may get fewer crises worth talking about.

The challenge is not to stop recognizing people who respond well under pressure.

It is to stop treating the pressure itself as evidence of leadership.

Boring Is Not Automatically Good

Of course, the absence of failure does not prove excellent leadership.

Sometimes nothing happened because the system was resilient.

Sometimes nothing happened because the system was lucky.

And sometimes nothing happened because nobody attempted anything difficult enough to fail.

Prevention is not inactivity.

Reliability is not risk avoidance.

A leader who avoids every difficult decision may also produce a quiet dashboard.

That does not make the organization healthy.

The real signal is whether a team can take appropriate risks while repeatedly producing dependable outcomes.

Did it ship meaningful work?

Did it improve the system?

Did it remove recurring sources of failure?

Did it reduce operational toil?

Did it make recovery faster without normalizing constant emergencies?

Reliable systems are not created by asking less of people.

They are created by putting more intelligence into architecture, testing, monitoring, documentation, redundancy, maintenance, and learning.

The visible experience becomes simple because the invisible engineering is sophisticated.

The Question Worth Asking

Perhaps the interesting question is not:

Who saved the system?

Perhaps it is:

Why did the system need saving in the first place?

That does not diminish the people who step forward during difficult moments.

Recovery matters.

Courage matters.

Leadership under pressure matters.

But so do the decisions made months earlier that prevented pressure from existing at all.

The automated test that caught the schema change.

The documentation that made the system understandable.

The data contract that prevented silent breakage.

The incident review that ensured the same failure did not return.

The clean pipeline nobody noticed.

The dashboard everyone trusted without thinking.

The model that kept receiving accurate features.

The architecture that quietly survived another year without incident.

Good leadership does not eliminate every crisis.

It reduces the number that should never have happened.

It builds systems that can recover without requiring human sacrifice.

And it makes reliable performance feel so ordinary that people forget how much work ordinary required.

The pipeline ran.

The dashboard was right.

The model received clean features.

Everyone went home on time.

One assistant forwards all 10,000.

Another sends a one-page summary:

• Three customer escalations need attention.

• Revenue is tracking 4% below forecast.

• Legal needs approval by Friday.

• Everything else can wait.

Which assistant is smarter?

Most people immediately pick the second one.

Why?

Because the second assistant compressed a mountain of information into something useful.

But hold on.

Spam filters compress information too.

Dashboards compress information.

KPIs compress information.

Even a zip file compresses information.

So what exactly impressed us about the assistant?

Was it the compression itself?

Or was it the ability to identify what mattered?

That distinction turns out to be surprisingly important.

Especially if you’re trying to understand modern AI.

Claude Shannon’s Strange Definition of Information

In 1948, Claude Shannon laid the foundations of information theory with a deceptively simple question:

What is the ultimate limit of compression?

To answer it, he did something that still feels a little unsettling.

He stripped information of meaning.

In everyday conversation, we think information means knowledge, truth, or insight.

Shannon didn’t.

To him, information was simply a measure of surprise.

A highly predictable event contains little information.

A highly unexpected event contains a lot.

If someone tells me:

The sun rose in the east this morning.

I learn almost nothing. The event was overwhelmingly expected.

If someone tells me:

A giant asteroid just landed in downtown Minneapolis.

I learn quite a bit.

Not because it’s useful.

Not because it’s true.

But because it would be wildly unexpected.

In Shannon’s world:

Information is surprise.

That single idea changed computing forever.

Because if information is surprise, then predictable things require fewer bits to describe.

And that leads to one of the most beautiful insights in computer science:

Prediction and compression are mathematically equivalent.

The better you can predict something, the more efficiently you can compress it.

The Question Shannon Couldn’t Answer

Now comes the part that fascinated me.

Suppose you want to know how compressible English really is.

How would you figure that out?

At first glance, this sounds like a statistics problem.

Count letters.

Count words.

Analyze books.

Calculate probabilities.

Done.

Except Shannon discovered something awkward.

Language isn’t just a collection of letters.

The probability of the next character depends heavily on everything that came before it.

To estimate the true compressibility of English, you need to estimate how predictable English is.

And to estimate predictability, you need a predictor.

So Shannon did something wonderfully clever.

He asked people.

In one experiment, he showed participants text and asked them to guess the next letter.

Then the next.

Then the next.

The better they could predict upcoming characters, the more compressible the language appeared to be.

Think about that for a moment.

The best language model available in 1950 was not a computer.

It was a human brain.

Shannon wasn’t merely measuring text.

He was probing intelligence itself.

Not because he was trying to build AI.

Because he was trying to measure compression.

And that’s where things start getting interesting.

Fast Forward Seventy-Five Years

Today we call it next-token prediction.

Back then Shannon called it estimating the entropy of English.

Different era.

Same underlying problem.

Modern language models perform a task that Shannon would immediately recognize:

Given everything you’ve seen so far, what comes next?

The scale is different.

The hardware is different.

The mathematics is remarkably familiar.

The better a model predicts language, the better it can compress language.

And here’s the part that makes people uncomfortable:

To become a better predictor, the model often has to discover increasingly sophisticated patterns.

Grammar.

Causality.

Human preferences.

Social conventions.

Programming languages.

Scientific concepts.

Business jargon.

The model learns compressed representations of all of these because they help prediction.

And better prediction means better compression.

Suddenly, compression and intelligence start looking suspiciously related.

But They’re Not the Same Thing

This is where many discussions go off the rails.

People notice that intelligence often produces compression and conclude:

Compression is intelligence.

That feels a bit like saying:

Footprints are walking.

A Netflix recommendation engine has a compressed representation of your viewing habits.

But the profile isn’t you.

A map is a compressed representation of a city.

But the map isn’t the city.

A financial dashboard is a compressed representation of a business.

But the dashboard isn’t the business.

Compression can preserve something important without becoming the thing itself.

That’s why I think both extremes miss the point.

The statement:

Compression is intelligence

goes too far.

But the statement:

Compression has nothing to do with intelligence

misses something profound.

Every time we try to compress something complicated—language, images, science, even our understanding of the world—we seem to be forced into building better models.

And better models often look suspiciously like what we call intelligence.

The Question Worth Asking

Maybe the interesting question isn’t:

Is compression intelligence?

Maybe the interesting question is:

Why does the pursuit of better compression keep pushing us toward increasingly intelligent behavior?

Claude Shannon was asking about compression.

He ended up needing human intelligence.

Modern AI researchers are building predictors.

They keep discovering capabilities that look increasingly intelligent.

Perhaps that isn’t a coincidence.

Compression may not be intelligence.

But it might be one of the clearest footprints intelligence leaves behind.

And footprints are worth paying attention to.

Just don’t mistake them for the traveler.

Further Reading

• Claude Shannon, A Mathematical Theory of Communication (1948)

• Claude Shannon, Prediction and Entropy of Printed English (1951)

• David J. C. MacKay, Information Theory, Inference, and Learning Algorithms

Recommended Viewing

This article was inspired by the excellent 3Blue1Brown video:

Reinventing Entropy | Compression is Intelligence (Part 1)

If you’d like the mathematical intuition behind entropy, information, prediction, and compression—and a much deeper treatment than I could fit into a Medium article—I highly recommend watching it. It’s one of the clearest explanations I’ve seen of why a seemingly abstract theory from the 1940s ended up sitting at the heart of modern AI.

Every generation invents a new machine and immediately overestimates what it understands.

The industrial era gave us machines that amplified muscle.
The information era gave us machines that amplified access.
And now we have systems that amplify plausibility.

That last one is new.
And a little dangerous.

A Quick Mental Model

Modern AI systems don’t know things.

They generate the most plausible next sequence based on patterns they’ve learned.

That’s extraordinarily powerful.
It’s also not the same thing as understanding.

Because modern AI doesn’t fail the way traditional software fails.

Traditional software crashes loudly.
LLMs fail like highly competent coworkers who skimmed the document, misunderstood the assignment… and still delivered the presentation with astonishing confidence.

Which, to be fair, is a very human failure mode.

That’s partly why these systems feel so uncanny. We didn’t build calculators with personality. We built probability engines that learned the statistical shape of:

• language

• reasoning

• persuasion

• confidence

• and occasionally… nonsense

If you work in AI, analytics, data science, or engineering, you’ve probably already encountered this strange reality:

The demo looks magical.
The architecture diagram looks elegant.
The benchmark scores look reassuring.

Then someone uploads a 200-page policy document and the model forgets the one paragraph Legal actually cared about.

Or worse:
the AI agrees with whoever asked the question.

That one is especially fascinating.

When AI Starts Telling You What You Want to Hear

A lot of teams assume AI systems are biased toward truth.

In practice, many are biased toward conversational success.
Those are not the same thing.

Models are trained on human feedback—and humans consistently reward responses that feel:

• helpful

• aligned

• fluent

• polite

• and reassuring

Contradicting users is risky.
Agreeing feels safe.

So if you tell the model:

“Remote workers are probably less productive, right?”

there’s a decent chance it begins searching for agreement.

Ask the opposite question and suddenly remote work becomes the future of civilization.

This tendency has a name: sycophancy.

A wonderfully old-fashioned word for a very modern problem.

The model isn’t trying to deceive you.
It’s learned what humans reward.

Unfortunately, organizations already contain plenty of incentive structures that distort truth.

Adding a statistically sophisticated people-pleaser into executive decision-making is… bold.

Particularly in analytics.

An AI assistant that quietly reinforces stakeholder assumptions can turn weak hypotheses into organizational folklore remarkably quickly.

And organizations are extremely efficient at operationalizing folklore.

The Myth of Infinite Memory

One of the stranger misconceptions in AI right now is the belief that larger context windows somehow solved memory.

Technically, yes—your model may now accept a million tokens.

Practically, that often means your AI can forget information at industrial scale.

Researchers call one version of this problem “lost in the middle.”

Which sounds less like an architecture limitation and more like a description of middle management.

The pattern is surprisingly consistent:

• the beginning gets attention

• the end gets attention

• the middle quietly fades into the statistical abyss
  

[Beginning] ====== [Middle] ====== [End]

      ^                                 ^

   remembered                       remembered

               middle:

       "I hope nobody needed compliance details"

You spend weeks building:

• retrieval pipelines

• vector databases

• ranking systems

• chunking strategies

• memory orchestration layers

…only to discover the model behaves suspiciously like someone pretending to remember a meeting they mentally left 40 minutes earlier.

To be fair, transformers were never designed to “understand” context the way humans imagine understanding.

They distribute attention probabilistically across tokens.

Large context windows improve capacity.
But capacity is not comprehension.

A warehouse full of books does not automatically create wisdom.
Neither does a giant prompt.

Which is why good AI architecture increasingly looks less like “give the model everything” and more like information choreography.

The best systems carefully decide:

• what matters

• when it matters

• how often to repeat it

• and what should never rely on memory at all

Because eventually every AI engineer learns the same painful lesson:

Context windows are not memory.
They are temporary attention markets.

Hallucination (A Very Fancy Word for Improvisation)

Then there’s hallucination.

Still one of the most beautifully misleading terms in AI.

It makes it sound as though the model experienced a brief psychedelic episode.

In reality, hallucination is usually just:

statistical improvisation wearing business casual

The model encounters uncertainty and continues generating plausible language—because that is exactly what it was trained to do.

Humans do this constantly.
The difference is that humans usually add:

“I’m not entirely sure…”

LLMs often skip that part.

The real failure isn’t that models are sometimes wrong.
It’s that they’re often wrong without calibrated uncertainty.

Which is how you end up with:

• fake citations

• invented APIs

• fabricated SQL fields

• imaginary policies

• confidently incorrect analytics narratives

A hallucinating model doesn’t sound confused.
It sounds like a consultant whose slide deck has excellent typography.

Why Confident AI Feels So Convincing

This creates one of the deepest risks in enterprise AI:

Humans mistake fluency for reliability.

We are astonishingly vulnerable to confidence theater.

If a response:

• sounds coherent

• uses bullet points

• references frameworks

• includes the word “strategic”

our brains quietly downgrade skepticism.

Which leads to another under-discussed issue:

automation bias

As AI systems become more capable, humans become less vigilant.

Not because people are lazy.

Because cognition is expensive—and the brain is an efficiency optimizer with a strong preference for shortcuts.

Pilots experience this.
Doctors experience this.
Drivers experience this.

Now analysts, engineers, recruiters, and executives do too.

At first, humans verify everything.
Then selectively verify.
Then eventually someone ships hallucinated code into production because:

“The AI seemed pretty sure.”

And to be clear—the AI was pretty sure.

That was never the problem.

Prediction Is Not Judgment

Organizations often deploy AI as though prediction and judgment are interchangeable.

They are not.

An LLM can predict likely words.
A model can predict attrition probability.
A system can predict engagement behavior.

None of them understand:

• fairness

• tradeoffs

• ethics

• organizational culture

• human motivation

• meaning

At least not in the way humans like to imagine.

AI doesn’t remove ambiguity from decisions.
It reveals how much was already there.

Which is… occasionally uncomfortable.

The Optimization Trap

Things get especially strange when optimization enters the picture.

You tell an AI system:

“Improve engagement.”

The machine asks:

“Define engagement.”

Humans respond:

“We’ll use survey scores.”

Three quarters later, everyone is shocked when managers optimize survey participation instead of employee wellbeing.

Welcome to Goodhart’s Law:

When a measure becomes a target, it ceases to be a good measure.

AI doesn’t create this problem.

It accelerates it.

Machines optimize literally.
Humans optimize politically.

Combine both carelessly and you get dashboards that look incredible while organizations quietly decay underneath them.

The Real Problem Was Never the Model

The strongest AI teams are no longer asking:

“What can the model do?”

They’re asking:

“What behaviors are we incentivizing?”

Because the central challenge of the AI era is probably not building intelligent systems.

It’s building organizations capable of using intelligence responsibly.

And that turns out to be a much harder problem.

The irony is almost poetic.

For years we worried machines would become more human.

Instead, many AI systems inherited the most human qualities imaginable:

• overconfidence

• inconsistency

• selective memory

• social pleasing

• shortcut-taking

• and occasionally saying things that sound brilliant until you think about them for six more seconds

Which means the future of AI may depend less on whether models become smarter…

…and more on whether humans become wiser around them.

Because a model, by definition, is not reality.

It is a compressed approximation.
A statistical shadow.
A map drawn from patterns.

And maps are useful.

Until people forget they are not the territory.

And the more detailed the map becomes…

the easier it is to mistake it for reality.