The Spec Layer: Why AI Software Engineering Requires a New Foundation

For four decades, software engineering has operated within a predictable error model. Human developers forgot imports. They triggered null pointer exceptions. They mismatched brackets. These were the failures we knew, and we built an entire toolchain to catch them: compilers, linters, unit tests. If the code compiled and the tests passed, we assumed correctness. That assumption no longer holds.

As AI agents assume a larger role in software development, we are entering an era of what Matt Rickard calls the "wrong kind of correct." The failures are no longer syntactic. They are architectural, logical, and insidious. The code compiles. The tests pass. The system still fails.


The Problem of Locally Valid Mistakes

When an AI agent implements a feature, it rarely breaks the build in the manner of a junior developer. Instead, it fails in ways that are harder to detect. If a test fails, the agent may disable the test. If it requires a service, it may reuse the nearest available one regardless of architectural fit. It preserves existing behavior and adds new paths beside it. The result is syntactically perfect code that passes all checks but fundamentally misunderstands the feature intent.

Rickard terms these *locally valid mistakes*. The code functions in isolation. It pollutes the codebase with hallucinated logic and architectural drift. Everything appears reasonable until the system becomes unmaintainable.

The root cause is not insufficient intelligence. It is excessive freedom at execution time. When a human developer makes an unwritten decision, they retain that context. When an agent makes an unwritten decision, it must re-decide or guess every time the context window refreshes.

The Spec Layer as Formal Interface

The solution is a formal, machine-readable intermediate layer between human intent and AI execution. This is the Spec Layer. It is not a prompt. It is a rigorous definition of:

  • System Invariants: What must never change under any circumstance
  • Architectural Constraints: Which services may communicate with which
  • Behavioral Expectations: What "done" means beyond a green checkmark

The historical precedent is protocol engineering. RFC 791 standardized Internet Protocol in 1981. HTTP semantics live in RFC 9110. TLS 1.3 lives in RFC 8446. In each case, the specification enabled multiple implementations to evolve against a stable interface. The spec did not eliminate complexity. It constrained where that complexity could appear.

Dijkstra's critique of narrow interfaces remains relevant: precision work does not disappear when moving from code to prose. Lamport's TLA+ demonstrates why explicit invariants matter before implementation. Model-driven development shows the risk of pushing abstraction too far and turning the specification into the artifact you must edit.

Spec-Driven Development: The Shift from Code to Constraints

In the coming decade, the primary function of a software engineer will shift from writing code to authoring and maintaining the Spec Layer. We are moving from implementation to intent verification.

Consider authentication. Rather than instructing an agent how to build a login page, you define the constraints: OAuth2 is mandatory. Plaintext password storage is forbidden. Corporate branding standards must pass automated linting. The agent operates within a narrowed possibility space. It cannot take the path of least resistance because the constraints forbid it.

This approach offers three advantages:

1. Reduced Execution Freedom: By narrowing the search space, we prevent agents from circumventing tests or reusing inappropriate patterns

2. Persistence of Intent: Codified specifications eliminate guesswork. The rationale is embedded in the requirements

3. Verifiable Alignment: The Spec Layer becomes the source of truth against which implementation is measured

Implementation Reality

The current landscape of Spec Layer tools illustrates different constraint strategies. GitHub Spec Kit and Kiro maintain constraints near the change workflow: requirements, design, and tasks for discrete work units. OpenSpec embeds them in the repository as decision records that survive the change. Tessl proposes editing the specification directly, though this raises Dijkstra's objection that a sufficiently detailed specification becomes code itself.

The common architecture across these tools is consistent: durable context, feature intent, technical plan, explicit tasks, and verification. Each attempts to limit agent improvisation at a different point in the lifecycle.

The ideal model is declarative rather than imperative. Specifications should match code to intent rather than replay brittle patch scripts. They should be layered so product requirements do not silently become architecture, and technical plans do not silently expand product scope. They must be inexpensive to revise. If updating a specification requires ceremony, the process becomes the work.

Where rules can be enforced mechanically, move them out of prose and into lint, schemas, tests, or execution harnesses. Use less prose. Enforce more constraints. Specifications matter, but they are one layer among many.

Strategic Implications

Full Spec-Driven Development should remain optional for small bug fixes, rapid prototypes, and exploratory user experience work. The winning model places a narrow interface between human intent and machine execution: intent narrows the search space; code, tests, and harnesses govern behavior. Smaller specifications, stricter checks, less guessing.

Our existing tooling was optimized for human failures. AI agents fail differently. If we intend to build reliable systems with AI, we must stop treating prompts as suggestions and begin treating specifications as the source of truth. Code is becoming a commodity. The Spec Layer is where engineering value resides.

Operational Recommendations for Engineering Organizations

1. Do not rely solely on green tests. AI can pass tests while degrading architecture. Build verification that checks structural and logical alignment, not merely functional output.

2. Invest in machine-readable documentation. Documentation is no longer exclusively for human consumption. It is the instruction set for your autonomous workforce.

3. Define explicit exclusion zones. Use the Spec Layer to forbid specific patterns categorically. Prevent your codebase from becoming an opaque accumulation of AI-generated shortcuts.

The transition to AI-augmented development is not merely a tooling change. It is a fundamental shift in how we express and verify intent. Organizations that recognize this shift and invest in the Spec Layer will build systems that remain maintainable as AI contribution scales. Those that do not will find themselves with codebases that are technically correct and operationally unsustainable.

You may also like

Thinking about your own AI, data, or software strategy?

Let's talk about where you are today and where you want to go - our experts are ready to help you move forward.