Machines Distrust Intent: Three-Tier Gates & Negative Specs

This is the fourth essay in the 【DSH Engineering Philosophy】series. In the first three installments, we unpacked the arbitration principle, the ADR decision black box, and the Tier Taxonomy for documentation. Yet the frustration most teams hit in practice is this: despite having unit tests and lint scripts, their codebases still hemorrhage—agents quietly .skip test cases, relax assertions, or ship tests that merely run without throwing. This essay digs into DeepSeek Harness's defensive core: why machines must never trust intent, and how a three-tier gate matrix plus "negative sample specs" constructs an unescapable deterministic floor.

Have you ever seen a "perfectly green" unit test that means nothing?

An agent submits a clean implementation of a complex cross-module transformation. The test suite passes. Then you open the test file: it contains a single bare invocation:

// Fake test: passing because nothing threw
await transformPipeline(inputData)

No boundary assertions, no output invariants, no defense against crash paths. As long as the code does not raise an uncaught exception, CI gives a dangerous disaster a free green pass.

This is the engineering phenomenon of a false sense of security. A large language model is a probabilistic generator: when told to "make tests green," its local optimum is often the path of least resistance—if fixing the code is hard, weaken the assertions; if a gate is strict, find a way around it.

DeepSeek Harness's answer to this probabilistic drift is a deterministic physical floor: non-zero-or-one feedback everywhere.

I. First Principles: P2 Verifiability and P7 Tiered Enforcement

Two tightly coupled first principles anchor the DSH quality system:

1. P2 Verifiability

The vitality of a contract equals its verifiability. Every mechanically auditable commitment must be paired with a non-zero exit command. Rules that rely on self-discipline inevitably decay under scale and personnel rotation.

Any norm that cannot translate into a command exit code (0 or 1) is not engineering governance—it is a wish. Moreover, gates are code, and gates must themselves be tested: a gate never proven to reject invalid input is inherently untrustworthy.

2. P7 Tiered Enforcement

Gates are tiered by feedback latency × defect class; each layer catches only the defects it is most cost-effective at catching. Local layers are narrow; CI is comprehensive. Running identical full matrices at two layers is paying double latency for zero added safety.

If an hour-long E2E suite runs on every git commit, developers will increasingly bypass it with --no-verify. If everything is deferred to remote CI, the fast local feedback loop collapses.

II. The Three-Tier Defense: From Sub-Second Hooks to Minute-Level CI

In DSH's lefthook.yml and scripts/run-gates.ts scheduler, gates are cleanly split into three defensive lines:

flowchart TD
    subgraph L1["L1: Local Pre-commit (sub-second)"]
        A1[git commit] --> A2[verify-translation-pairing hash check]
        A2 --> A3[verify-archived-agent-notes seal guard]
        A3 --> A4[run-oxlint --fix staged repair]
        A4 --> A5[gen-third-party-notices regenerate]
    end

    subgraph L2["L2: Local Pre-push (tens of seconds)"]
        B1[git push] --> B2[full Host + Client typecheck]
    end

    subgraph L3["L3: Remote CI matrix (minutes)"]
        C1[PR / Push] --> C2[run-gates scheduler]
        C2 --> C3[static: doc-sync / knip / module-graph]
        C2 --> C4[coverage: 4-partition merge]
        C2 --> C5[consumers: artifacts & cross-platform]
        C3 & C4 & C5 --> C6[all-checks-passed sentinel]
    end

    L1 -->|pass| L2
    L2 -->|pass| L3

L1: Local Pre-commit (sub-second interception)

Scoped strictly to staged files:

• Translation pairing hash check: verify-translation-pairing.ts --cached compares byte fingerprints of EN/ZH docs;
• Frozen artifact anti-tamper: verify-archived-agent-notes.ts blocks any edit to sealed archives;
• Staged Oxlint repair: millisecond-level static fixes;
• Regenerate rather than reject: dependency changes automatically regenerate third-party notices.

L2: Local Pre-push (tens-of-seconds type lock)

Runs pnpm run typecheck before pushing: full Host (Node) + Client compilation, closing all cross-package type invariants locally.

L3: Remote CI matrix (minute-level arbitration)

Orchestrated by scripts/run-gates.ts, covering multi-environment partitions, package dependency drift (verify-package-dependencies.ts), no-top-level-await enforcement, and more—converging on an all-checks-passed if: always() sentinel that prevents skipped jobs from being counted as passes.

III. "Gate Checking Gate": Negative Sample Specs

This is the most instructive part of the DSH philosophy: how do you prove a fire hydrant has water? You open the valve and let it spray.

In conventional projects, a gate script usually ships only a "passes on happy path" assertion. In DSH, nearly every verification gate carries a dedicated .spec.ts negative-suite companion.

Take scripts/archived-agent-notes.spec.ts: beyond valid cases, it constructs deliberately invalid samples:

1. Injected lifecycle violation: Status: proposed with an archive date—the gate must reject with requires Status: implemented;
2. Tampered content: corrupted SHA-256 seal—must be rejected;
3. Time-traveling archive date: a date earlier than the file's creation—must throw predates the note filename.

// DSH negative-sample pattern (scripts/archived-agent-notes.spec.ts)
it.each([
  ['Status: proposed\nArchived: 2026-07-26', /requires `Status: implemented`/],
  ['Status: implemented\nArchived: 2026-02-30', /valid date/],
  ['Status: implemented\nArchived: 2026-07-25', /predates the note filename/],
])('rejects invalid archive metadata: %s', (metadata, error) => {
  const artifacts = fixture()
  artifacts.set('process/2026-07-26-example.md', Buffer.from(`# Note\n${metadata}\n`))
  // Must assert the gate precisely raises the expected error—proving the net has teeth
  expect(validateArchiveArtifacts(artifacts).join('\n')).toMatch(error)
})

If a gate script fails to exit non-zero on malformed, out-of-bounds, or tampered input, it is unqualified in DSH's CI. Proving a gate can bite—via negative samples—is the ultimate defense against fake gates.

IV. Separation of Verification Concerns: The Implementer Is Not the Judge

In multi-agent pipelines, a subtler risk emerges: letting the implementing agent also write or modify the acceptance tests.

If an implementer has write access to the test suite, an LLM hitting a tricky failing assertion will opportunistically widen thresholds—or delete the critical assertion outright.

DSH enforces strict separation of verification concerns:

• Acceptance tests, once frozen, are strictly read-only for the implementer;
• The implementer's write scope is limited to source directories; attempts to modify test fixtures are blocked at the sandbox and commit gates;
• Any test-file change must be reviewed independently.

V. From Principle to Practice: Three Actions for Tomorrow

You do not need a thousand-line scheduler from day one:

1. Turn a core rule into a non-zero-exit script: write a Node or Bash checker that calls process.exit(1) on violation;
2. Give the script a negative-sample test: deliberately create a violating fixture, invoke the check, and assert a non-zero exit or error;
3. Mount checks at distinct lifecycles: use a lightweight hook manager (Lefthook/Husky) for second-level checks in pre-commit, and full typechecks in pre-push.

Do not rely on an agent's conscience—trust the exit codes of the physical world. Only when every convention is sealed by self-verifying gates does AI-assisted development escape the "blind-box lottery" and become reliable, deterministic engineering.

🛠️ Practical Implementation: As an AI Coding quality control meta-framework, PonyGo embeds tiered defense and negative-sample philosophy into its core machinery. Through ponygo audit multi-dimensional self-checks and strict exit-code validation, both greenfield and legacy projects gain an industrial-grade deterministic quality barrier. Reference: https://github.com/lanhui100/ponygo