raghav iyer

The debt of abstraction

Every generation of computing makes it easier to accomplish things without understanding the machinery underneath. This is mostly a good thing.

I don't need to understand the semiconductor physics of a NAND cell to save a PDF. A Python programmer doesn't need to think about registers. Someone deploying a website doesn't need to understand TCP congestion control.

This is abstraction, and abstraction is basically the reason we can build complicated things at all.

The stack keeps growing:

transistorsinstructionsprogramsinterfacesintent.

At every step, we trade understanding of the layer below for greater capability at the layer above.

Agents feel different, however. Not because they are another abstraction, but because the abstraction is no longer merely hiding complexity. It is acting through complexity on your behalf.

That distinction is rather important. Suppose I want to install and configure some software. Previously, I might have to execute a sequence of actions:

a1a2a3an.

I don't need to understand the kernel to do this, but I still interact with the procedure. I see the package being installed. I edit the configuration. I grant permissions. I restart the service.

With an agent, I say:

Set this up for me.

And the interface becomes

intentresult.

The intermediate states disappear from my experience, but they don't disappear from the computer. The agent still has to produce some policy

π(I)=(a1,a2,,an)

that turns my intent I into actions.

The filesystem is still there. Permissions are still there. Processes are still there. The network is still there. The complexity wasn't removed, but someone else walked through it, and now that someone is a probabilistic machine.

Capability is separating from understanding

Historically, becoming more capable with a computer generally required learning more about it. Very roughly,

CK>0,

where C is capability and K is knowledge.

Want to run a server? Learn some networking. Want to use Linux? Eventually you learn about processes, permissions and filesystems. Want to write fast numerical code? At some point the hardware begins making demands.

Agents weaken this relationship. It is increasingly possible for

CuserKuser.

Your effective capability can increase dramatically while your understanding of the system underneath barely changes. This is incredible, but it is also borrowed capability.

As long as the abstraction works, borrowed capability is indistinguishable from owned capability. Then something breaks.

At that point,

CeffectiveKactual.

You suddenly have to reason about a system you have been operating without understanding.

This is what I think of as abstraction debt.

Like technical debt, it isn't necessarily bad. Taking on debt can be extremely rational. It lets you move faster today by accepting some future cost.

Abstraction debt works similarly. You gain capability now by giving up understanding of the machinery producing it. Most of the time the trade is fantastic. The debt becomes visible when the abstraction leaks.

Omarchy

This is what makes things like Omarchy interesting to me. The idea that the operating system itself increasingly becomes something you operate through agents. The agent can inspect the machine, install packages, modify configuration and execute commands for you.

The natural endpoint is obvious → Why learn the interface at all?

If I can say

Set up the simulation, install whatever it needs, run it and show me the results.

and the machine reliably does it, that is ultimately a better computer. I want that computer.

But the more useful the agent becomes, the more authority it needs.

If

𝒜={a1,a2,,an}

is the set of actions an agent can perform, then generally

|𝒜|U,

where U is usefulness.

Unfortunately,

|𝒜|B,

where B is blast radius.

The ability to fix your configuration is also the ability to break it. The ability to organize your files is also the ability to delete them. The ability to administer your machine is also the ability to administer your machine incorrectly.

This isn't really an AI problem. It's a function of what happens when abstraction meets authority.

The debt isn't ignorance

We should eliminate enormous amounts of accidental complexity. I do not want to spend my life resolving dependencies just so I can claim to understand my computer.

The interesting question is how much understanding we should preserve.

Probably not

Kuser=Ksystem.

That hasn't been possible for decades.

Maybe just

KuserKrequired to reason about failure.

Enough to understand the boundaries, to know what the agent is allowed to touch, to know what has probably happened when the abstraction stops working, and enough to recognize when the machine is doing something insane, because agents are probably going to make us dramatically more capable.

In general, I think that is good. I'd love to operate computers at increasingly absurd levels of abstraction. I want to describe the result and let the machine deal with everything underneath.

But every layer of abstraction creates a little distance between capability and understanding. Agents will create a lot of distance very quickly.

That is abstraction debt.

And like every other kind of debt, the question is whether you still know how to pay it back when it comes due.