The hardest problem in autonomous code remediation is not generating a plausible-looking diff—it is preventing the AI from rationalizing a broken fix. After four specialists and a synthesizer exchange thousands of tokens, the conversation history itself becomes a source of bias.
Why Agents Hallucinate in Multi-Step Pipelines
If you ask the same LLM session that diagnosed a bug to also grade its own patch, it already believes its own narrative. It overlooks an undefined variable, misses a syntax error, or ignores a constraint from the runbook because its context window is saturated with justifications.
Our solution is a two-layer defense: a deterministic code validator paired with an L4 Fresh-Context Verifier, followed by a durable Human-in-the-Loop (HITL) escalation gate.
Even at 94% confidence, api/checkout.py matches protected_paths. Graph freezes until human clicks Approve.
Layer 1: Deterministic Checks at $0 LLM Cost
Before invoking any reasoning model to audit a proposed fix, verifier_node runs fast deterministic checks in Python:
- Non-empty Unified Diff: Computes
difflib.unified_diff(original_lines, patched_lines)and verifies the patch actually modifies the target file. - Python AST Syntax Validation: For
.pyfiles, runsast.parse(patched_file_content). If a syntax error exists, the verifier immediately returnsis_valid=Falseandconfidence_score=0with the exactSyntaxErrorline—routing back to the Synthesizer without burning frontier tokens.
Layer 2: The Zero-History L4 Fresh-Context Verifier
Once deterministic checks pass, SentinelSRE spawns an isolated verifier prompt. Crucially, we strip specialist_findings and root_cause_report completely. The verifier sees only four raw artifacts:
- The raw runtime stack trace from Sentry/Railway/Vercel.
- The raw target file contents fetched directly from GitHub.
- The proposed unified Git diff.
- The workspace's
procedural_rules.
Acting like a strict senior reviewer with zero prior bias, it outputs a structured VerifierOutput(is_valid, confidence_score, critique). If is_valid is false and retry_count < 2, LangGraph loops back to the Synthesizer with the verifier's exact critique so it can self-correct before a human ever sees it.
Layer 3: Durable HITL Checkpointing with PostgresSaver & interrupt()
Verification proves a patch is syntactically sound and addresses the stack trace—but verification is not authorization. Whenever confidence_score < min_confidence_threshold (default 85%) or the target file matches protected_paths (such as api/checkout.py, payments/, auth/, or migrations/), the escalation_gate_node calls LangGraph's interrupt().
Because the graph is compiled with AsyncPostgresSaver backed by Neon Postgres, pausing at interrupt() freezes the entire IncidentState snapshot under thread_id = incident.id. Even if the FastAPI container restarts or scales down, the paused investigation survives intact.
In the Split-Screen War Room, the engineer inspects the side-by-side diff, the citations, and the verifier's confidence meter. Clicking Approve & Open Hotfix Draft PR calls POST /api/incidents/{id}/decision, resumes the exact Postgres checkpoint via Command(resume=...), invokes the GitHub MCP and Slack MCP servers to open a draft PR (never mutating main directly), and saves the decision into Episodic Memory.
Software verifies correctness; humans authorize production impact. And if an engineer rejects a patch with feedback, Episodic Memory ensures SentinelSRE doesn't make the same mistake twice.