The review pipeline
What codingassist does between a pull request event and the comments it posts: three stages, the gates between them, and which steps call a model.
Each commit pushed to a pull request starts one review. The review runs in three stages (context, static analysis, AI review), then codingassist posts the result on the pull request.
What starts a review
Azure DevOps sends a service hook event when a pull request is created or updated. codingassist then:
- reviews each commit once: a second event for the same commit is ignored
- ignores events caused by its own comments
- skips the review if the organization's plan is not active, and says so in the status check
While the review runs, the status check reads "Review in progress".
Stage 1: context
Three components collect what the AI review needs. If one of them fails, the review continues without its output.
| Component | What it collects |
|---|---|
| Work items | The work items linked to the pull request, their acceptance criteria, and each item's parent. See Work items |
| Impact map | A call graph of the changed files, built with Tree-sitter, for C#, Python, JavaScript, TypeScript, Go and PHP |
| Similar code | Snippets from the repository's code index that are similar to the change (5 by default) |
Stage 1 also reads the repository's .praxiom.yml, the list of changed files, and the content of up to 25 changed files at the head commit.
A pull request with more than 150 changed files stops here. codingassist posts a comment saying it is too large, and the status check fails with "Not reviewed".
Stage 2: static analysis
No model is called in this stage.
- Built-in rules check the diff for SQL built from strings, hard-coded secrets, secrets written to logs, blocking calls on async code (
.Result,.Wait()),Thread.Sleep, emptycatchblocks,Console.Write,TODOandFIXMEcomments, and missing null guards. - Sandbox analyzers run in an isolated container on the changed files: gitleaks, trufflehog, trivy, semgrep, bandit, pip-audit, ruff, pylint, eslint and golangci-lint.
- Your tools, declared in
sandboxToolsin.praxiom.yml, run in the same sandbox.
Token budget
Before any model call, codingassist reserves tokens from the organization's budget. If the budget cannot cover the review, the review is paused, codingassist posts a comment, and the status check fails with "Review paused". See Usage & token quota.
Stage 3: AI review and verification
- Review. One model call reviews up to 12,000 characters of the diff against the acceptance criteria, the impact map, the similar code, the static findings and your custom instructions. It returns findings, a summary, a verdict for each acceptance criterion and a risk score.
- Verification. Each finding is checked. A finding whose file or line does not exist at the head commit is dropped. Contested findings go to a second model call that can drop false positives. Critical security findings are never dropped. AI findings with a confidence below 0.80 are dropped.
- Fixes. Only when
review.autoFixistrue: for Critical findings, codingassist generates patches, validates them in the sandbox, and posts up to five of them as suggestion comments.
Publication
codingassist posts:
- one summary comment, updated in place on every push
- an inline comment for each finding that has a file and a line
- a status check named
PRaxiom, which fails when at least one Critical finding remains
Which steps call a model
| Step | Calls a model |
|---|---|
| Work items, impact map, similar code search | No |
| Built-in rules, sandbox analyzers | No |
Your tools with parser: text-llm | Yes, to turn the tool's text output into findings |
| AI review | Yes |
| Verification of contested findings | Yes |
Fixes (autoFix) | Yes |
Replies to @codingassist comments | Yes. See Commands in pull request comments |
Model output is not deterministic: reviewing the same commit twice can produce different findings. The static stage is deterministic.