Flaught
Independent review for agentic development.
Flaught reviews AI-assisted code with deterministic tools and a separate LLM reviewer.
It helps catch defects, weak tests, security issues, and unintended changes before they reach production. Run it in CI, from the command line, or inside an agentic development loop.
Set it up with one paste: drop this into your AI coding agent (Claude Code, Codex, Cursor, or pi):
Install Flaught (npm install -g @flaught/core), run flaught init to scaffold config if there's no .advreview.yml yet, then run the adversarial loop below. ## The adversarial loop 1. Run: flaught review --base main --output findings.json --quiet 2. Read findings.json. Fix a finding only if you're genuinely confident it's real and mechanical — a lint failure, a missing null check, an obvious off-by-one. Leave anything involving auth, architecture, or ambiguous intent alone. 3. Re-run step 1. This is the check: if the underlying issue is still there, Flaught will find it again, whether or not the code changed shape. 4. Repeat steps 2-3 at most twice more. Then STOP. Report every finding still open, verbatim. Do not run flaught dismiss — dismissing a finding without fixing it is a human decision, no matter how many passes this took. No API key needed for the deterministic pass. For the full LLM pass, set GROQ_API_KEY (or configure llm.provider in .advreview.yml).
Note what this prompt doesn't say: “re-run until clean” with no limit. A skeptic the author can dismiss isn't a skeptic. Dismissals (flaught dismiss) are your tool for suppressing false positives, never the agent's tool for getting to green.
Runs in GitHub Actions, GitLab CI, or any shell. No API key needed for the deterministic pass (flaught review --no-llm); set GROQ_API_KEY for the full LLM pass. Provider config →
Prefer the manual shell? Install with npm.
# install npm install -g @flaught/core # scaffold config + prompts, then review vs main flaught init flaught review --base main
01Who it's for
Solo founders and small teams shipping agent-written code. You're moving fast because the agent reviews its own work, which means nobody does. Flaught runs on every PR, says nothing when things are clean, and surfaces something only when it needs human eyes: two findings, ninety seconds of your attention.
Engineers landing agent-authored changes in enterprise codebases. Your team is accountable for code no human wrote. Flaught's timestamped, versioned JSON artifact is the audit record: independent scrutiny occurred, here's what it found, here's whether the model saw the whole change (analysis_completeness). Evidence for your reviewers today; evidence for your compliance regime tomorrow.
Open-source maintainers triaging AI-generated PRs. Drive-by agent PRs are rising and your review time isn't. Run Flaught first: scope-creep detection catches the unrelated hunks, test inversion catches the vacuous tests, and you spend your attention only on contributions that survive the skeptic.
02Five stages, one skeptical pass
-
Context assemblyDiff, changed files, and a one-hop dependency neighborhood so findings carry blast radius alongside the line number.
-
Deterministic tools deterministicSemgrep, a linter, and a vuln scanner. Every finding is tagged
source_type: "deterministic"so you know what came from a tool. -
LLM adversarial pass llmStructured skeptical review of security, architecture, scope-creep, and test quality, tagged
source_type: "llm", distinct from the tool findings. -
Test inversionRuns the test suite on pre-change code and flags tests that pass identically on both sides: a before/after test-sensitivity check that catches vacuous tests the change didn't actually exercise.
-
Scope-creep detectionHeuristic plus LLM; flags hunks unrelated to the PR's stated intent.
Output: a Markdown PR comment plus a versioned JSON artifact you can keep (one snapshot per run, for trend tracking).
For the full pipeline, component map, and single-run sequence: Architecture reference →
03Review does not have to be expensive
Deterministic checks can run without any LLM inference.
For model-based review, Flaught can use a lower-cost hosted model, a local model through Ollama, or a more capable model when the change warrants it.
The coding model and the reviewing model do not need to be the same.
That lets teams add independent review without paying frontier-model prices for every pass.
Every run records its token usage — review and refute passes separately — in the findings artifact and the PR comment, so you can see what each pass actually consumed and apply your own pricing.
04The adversarial loop
Flaught always reports what it finds — that part is fixed, not configurable. What happens next is up to you.
Route a finding to a human, or describe a loop to your coding agent: let it attempt the mechanical fixes itself, then check the result by simply running Flaught again. Re-running is the check. Flaught re-scans from scratch every time, so a fix that didn't actually work shows up again on the next pass — no special tracking required, just the same skeptical pass, repeated.
Dismissing a finding without fixing it always requires a human, no matter how you've set up the loop.
This is the loop in the quick-start prompt above — paste it once and it covers both the install and the loop. If you want just the loop part on its own, for an existing agent config or a CLAUDE.md, copy steps 1 through 4 out of that same block; nothing about it changes outside the install line.
Once this is working locally, the same review becomes a merge gate in CI. GitHub Actions has three tested workflow patterns including fork-PR handling — see the guide below. GitLab CI support is early and undocumented; issue #29 is open if you want to help close it.
05Review with a different model than the one that wrote the code
Self-review is the weak spot in AI-assisted development: the model that wrote your PR tends to agree with its own choices. Point .advreview.yml at a different provider and Flaught breaks that correlation, zero code change.
A sane pairing: code with Claude, review with Groq or GPT-4o. Code with anything, review with a different anything. Any OpenAI-compatible endpoint works via base_url; Anthropic has its own native adapter too. It works best as the coder here; pair the review step with a different provider.
06Extend it without forking it
The defaults are a starting point. Three surfaces let you reshape the review without touching Flaught's source: the prompt, the config, and the library API.
Full guides: prompt templates, configuration, programmatic API.
07How it compares
Existing review tools were built to check human-written code. Flaught's category is agent-authored code: where the characteristic failures are vacuous tests, scope drift, and confident self-approval, and where the reviewer's independence from the author is the whole point.
| Flaught | CodeRabbit | Open Code ReviewAlibaba · OCR | Gitar + SonarQube | |
|---|---|---|---|---|
| Built for agent-authored changes | ✓ | ✗ | ✗ | ✓ |
| Open source, self-hostable | ✓ | ✗ | ✓ | ✗ |
| Bring-your-own LLM (incl. local) | ✓ | ✗ | ✓ | ✗ |
| Free, no seat license | ✓ | ✗ | ✓ | ✗ |
| Deterministic tools + LLM, provenance-tagged | ✓ | ✗ | ✗ | ✓ |
| Adversarial refute / skeptic pass (cross-model) | ✓ | ✗ | ✗ | ✗ |
| Test inversion (vacuous-test detection) | ✓ | ✗ | ✗ | ✗ |
| Scope-creep detection | ✓ | ✗ | ✗ | ✗ |
✓ yes ✗ not a named feature
CodeRabbit combines 40-plus linters and SAST tools with AI review, but doesn't tag each finding by the source it came from. Open Code Review's deterministic layer handles file selection, comment positioning, and rule routing rather than running external tools like Semgrep with per-finding provenance tags, and its reflection module improves comment accuracy within a single pipeline rather than having a second, independent model check the first. SonarQube's free Community Build is open source but doesn't review pull requests at all — that capability only exists in Gitar or a paid SonarQube edition, and neither of those is open source. Gitar lets you bring your own Anthropic key, but not a local model, and your code is still sent to Gitar's servers either way.
The same rigor extends to how Flaught runs in CI. Fork PRs are handled with a two-workflow security pattern that keeps the LLM provider's API key out of reach of a malicious fork — the model never sees the secret, only the diff, so prompt injection can't exfiltrate it. And Flaught reviews its own pull requests: this repo dogfoods itself in production, not just in a demo.
08The honest caveat
⚠ The JSON artifact is evidence that scrutiny occurred, not evidence that findings are correct. LLM-asserted findings may include hallucinations. Deterministic-tool findings have their own false-positive rates. An adversarial reviewer that overclaims its own reliability would be the exact thing it exists to catch, so Flaught doesn't. When it flags something, look. A clean run means the skeptic didn't find anything. It doesn't mean there's nothing to find.
And honestly legible: every artifact carries an analysis_completeness field recording whether the LLM saw the whole change or only part of it: a large PR can truncate the prompt, so “review completed” is never mistaken for “comprehensively reviewed.” Field reference →
Named for Monsignor Flaught, the devil's advocate in A Canticle for Leibowitz.