- Nexa
- The gap, measured
- Why an Odoo linter does not already cover this
- What it finds that review does not
- What it is not
- One corpus, and why the Python rules are in it
- How it checks itself
- Install
- Usage
- The corpus is the source of truth
- The checkers are tested, then attacked
- Patterns are tested against their own samples
- Cross-checking measures false positives
- Then the checkers get attacked
- And then pointed at code nobody wrote for them
- The same treatment on a second ecosystem
- The rule that classification killed
- What is still wrong, named
- What it misses
- What the embedder drops
- The missed list is the worklist
- An outside opinion, which is the one thing the corpus cannot give itself
- Every version claim is checked against two kinds of evidence
- Is every rule findable?
- And what does
nexa checkactually report?
- Environment
- Requirements
- Documentation
- License
- What you get
- The gap, measured
ββ ββ ββββββ ββ ββ βββββ
βββ ββ ββ ββ ββ ββ ββ
ββ β ββ βββββ βββ βββββββ
ββ βββ ββ ββ ββ ββ ββ
ββ ββ ββββββ ββ ββ ββ ββ
ββββββββββββββββββββββββββββββ
Nexa
The Odoo defects that error nowhere, fail no test, and look correct in review.
Some bugs announce themselves. A stored @api.depends that omits a field does not: the
value is written once, never recomputed, and survives every restart. Nothing raises.
No test fails, because the test asserts what the compute returns today. Review sees a
decorator and a method body that both look right. It surfaces months later, when a
report disagrees with the source data and somebody has to work out how long it has been
wrong.
That class is what this is for.
It runs entirely on your machine. No API keys, no network at review time, no client code leaving the laptop β which for a consultancy under an NDA is not a preference.
No model weights live here. Nexa is the corpus, the checking pipeline, and the CLI β a few hundred kilobytes. It orchestrates open models you pull from Ollama's registry (Qwen2.5-Coder, Qwen3-Coder, nomic-embed-text), which keep their own upstream licenses. See INSTALL.md.
The gap, measured
Read all 448 fix commits from OpenSPP β a production Odoo 19
social protection platform β and ask each one how the bug came to light. Every body was
read by hand; the verdicts and the evidence line behind each are in
surfacing-openspp-2026-09.json, and python bin/surfaced.py <repo> --check re-derives
the commit set and recomputes every percentage below from the rows.
Two denominators, because one of them flatters and the other hides:
| How it surfaced | of all 448 | of the 295 that say |
|---|---|---|
| Review or an analyser caught it | 32.6% | 36.9% |
| Silent until the data was visibly wrong | 17.0% | 25.8% |
| Somebody using it hit it | 14.7% | 22.4% |
| A test said so | 6.5% | 9.8% |
| An upgrade or migration forced it out | 3.3% | 5.1% |
| The body does not say | 25.9% | β |
The second row is this tool's subject: 76 commits where nothing errored, nothing failed, review saw nothing, and the bug only became visible once the data had gone wrong. That is the one category where a checker beats a person, because the person has nothing to look at.
The right-hand column drops two things the left one counts: 90 commits with no body at all, and 37 whose whole content is lint, formatter or CI compliance β a fix commit that fixes no defect. A regression is recorded as a cause on its own axis rather than a channel, because "a previous fix caused it" answers a different question from "how did anyone find out"; 19 commits carry it.
The honest caveat is in the last row. A quarter of these commits do not say how they surfaced, and a bug found in review may have sat in production silently for months first with the commit having no reason to mention it. Both columns are shares of what is legible, and neither is a share of what happened.
A specimen, from that history. A uniqueness rule declared with the legacy
_sql_constraints attribute, which Odoo 19 ignores β the registry logs a warning
at load and the constraint is never created:
duplicate rules could be saved silently since 19.0.3.0.0 β fixed months later, with a pre-migration to delete what had accumulated, and a new lint rule of their own, because "nothing in CI caught the legacy attribute."
Why an Odoo linter does not already cover this
OpenSPP wrote seven linters of their own. Run them and you get 748 findings:
445 performance.n_plus_one 109 naming.boolean_prefix
111 ui.extension_points 18 logger.pii_in_log
14 performance.offset_pagination 13 acl.missing_file
N+1 queries, naming conventions, UI patterns, ACL files, PII in logs. Not one check
for @api.depends completeness, @api.constrains field lists, ondelete, or
check_company. A production Odoo team, writing their own tooling from scratch, did
not build this β the same is true of pylint-odoo, which is mostly convention.
So the two are complementary, and Nexa can now run theirs too:
nexa fieldtest --stack odoo . --with-json their-lint.json --with-name openspp
Their findings arrive as ours do β ranked together, one suppression convention, one
per-thousand-lines denominator β and at likelihood unmeasured, which is the accurate
value. Nobody has published a false-positive rate for performance.n_plus_one, which
is 60% of their output from a single check. This repository has the harness that can
(nexa recall --aim), and pointing it at somebody else's rules is the more interesting
half of the wrapper.
What it finds that review does not
stored-compute-needs-complete-depends |
A stored value that stops recomputing. Reads line_ids.price, declares line_ids. |
constrains-must-list-every-field-read |
A constraint that never fires. Validates date_end, declares date_start, so writing date_end alone skips it. |
ondelete-is-a-decision |
A relation left on the ORM default, then read through. set null yields False rather than raising. |
check-company-on-relations |
A record in one company pointing at another's. A UI domain constrains a dropdown and enforces nothing. |
Measured against reviewed, merged OCA code β 33,960 lines that passed human review
and shipped β the corpus produces 3.0 findings per thousand lines. All of them
classified by hand: 79 true positives, 10 false, 13 undecidable from a single file.
Every verdict is in classification-oca-2026-09.json, so the number can be re-opened
rather than taken on trust.
On 226,937 lines of OpenSPP the rate is 0.7, and those findings have been read too:
138 true, 124 false, 6 undecidable across classification-openspp-2026-09.json and
classification-openspp-rest-2026-09.json. One rule accounts for 97 of those false
positives and has been rewritten because of it β that story is
below, and it is the more useful half of
publishing a classification.
What it is not
Not a general linter. 27 rules. It will not replace pylint-odoo, and after the
wrapper above it does not have to.
Not a bug finder. Against real Odoo bugfixes the corpus catches 20 of 334 β about 6%, measured across three ecosystems and 1,164 fix commits: OpenSPP 6%, the two OCA repositories 6%, and odoo core itself 5%. Most bugs are domain logic ("finished plans should not be the current plan") and no rule corpus finds those. Measured and published here because a precision number without a recall number beside it is the more flattering half of the picture. It was 2β4% on two trees before the pass in Β§3.47; the way it moved was reading the missed list, not writing more rules.
Odoo only, and that is the whole of it. There was a TypeScript corpus and a
JavaScript one. They got their first field evidence in September 2026 and it was not
good β JavaScript classified out at 21 true against 22 false, TypeScript was never
classified at all β so both were removed rather than left in with a warning on
them. A .ts or .js file is now refused by name instead of reviewed against rules
that would find nothing in it, because nothing found reads as nothing to find.
The plain-Python rules did not go the same way, and the reason is in the composition.
One corpus, and why the Python rules are in it
There is one stack. Every rule in it is a rule for reviewing Odoo code: 19 about the
ORM, and 8 about the Python language, which apply because an Odoo module is a Python
module. The python. prefix on those eight records where the rule comes from and
which documentation it is checked against. It is not a second mode you can select.
There is no second mode.
They were unreachable until September 2026, and it was one line: selection was strict
equality on the stack field, so an Odoo file got the 19 ORM rules and none of the
other 8. fieldtest --stack python over OpenSPP found 32 hits that --stack odoo
never reported.
The first instinct was the opposite one β drop the Python corpus, because ruff covers
it. Checked against OpenSPP's actual .ruff.toml, that is wrong. They select B,
C90, E501, I, UP and ruff's defaults. S and DTZ are not selected:
| ruff rule | selected there? | |
|---|---|---|
assert-is-not-validation |
S101 | no |
eval-and-pickle-execute-their-input |
S301/S307 | no |
subprocess-shell-true-is-injection |
S602 | no |
naive-and-aware-datetimes-do-not-compare |
DTZ | no |
path-join-does-not-contain-a-path |
β | no such rule |
decimal-for-money-not-float |
β | no such rule |
mutable-default-argument |
B006 | yes |
bare-except-catches-the-exit |
E722 | yes (default) |
A narrow ruff selection is what most Odoo shops run, so six of the eight are genuinely unreported in that deployment. Against ruff this is not a competing tool; inside an Odoo review it is part of one.
What the composition actually found, all 50 read by hand
Composing them moves every Odoo number, so the standard the rest of the corpus is held
to applies here too: 50 findings across OpenSPP and OCA, each read in its own file.
Every verdict is in classification-python-2026-09.json.
| true | false | ||
|---|---|---|---|
naive-and-aware-datetimes-do-not-compare |
20 | 12 | 62% |
assert-is-not-validation |
6 | 4 | 60% |
decimal-for-money-not-float |
0 | 3 | |
path-join-does-not-contain-a-path |
0 | 4 | |
eval-and-pickle-execute-their-input |
0 | 1 | |
mutable-default-argument |
β | β | never fired |
bare-except-catches-the-exit |
β | β | never fired |
subprocess-shell-true-is-injection |
β | β | never fired |
| 26 | 24 | 52% |
One rule carries it. Twenty of the twenty-six true positives are the datetime rule,
and they are one shape: an Odoo Datetime field is naive UTC by contract, and a
naive local datetime reaches one. membership.ended_date > datetime.now() filters by
the server's offset. An autovacuum cron builds its deadline from local now and compares
it against create_date, deleting audit logs early or late. A linked-data proof appends
a literal Z β which asserts UTC β to a local time. A fiscal-year cron compares an ORM
Date against datetime.now().date(), so on any non-UTC server the year is created on
the wrong side of midnight.
Say what that makes them: on a server clocked to UTC every one is latent. On any other
it is wrong by the offset. OpenSPP's own history carries two fixes of exactly this class
β a newborn birthdate evaluated against the server's UTC date, and Date.today() storing
midnight UTC on a Datetime field β which is why it is graded a defect rather than a
theoretical one.
The twelve false ones are near enough one shape too: a timestamp that is formatted and
never compared β a Generated: line in a report, a backup filename, a %Y-%m period
label. Telling those from the twenty needs to see what happens to the value afterwards,
which a regex cannot.
What is wrong with the other half, named
Three rules fired and were never right. decimal-for-money-not-float was 0 of 3, and
this one has a reason behind it rather than a sample size: Odoo's Monetary field is
a float β the ORM defines it that way β and the framework's answer to the precision
problem is float_round and float_compare against the currency's rounding, not
Decimal. Inside an Odoo module the rule fired on the idiom and named a fix that cannot
be taken. Three findings settle nothing, so it was narrowed rather than culled, on the
line the argument itself draws: it now fires only where the value is still a string
when it arrives, which is the one place Decimal(value) is available. float(self.amount)
is the ORM's own type and is not reported. Re-run on both trees: 0 findings, from 3.
path-join-does-not-contain-a-path was 0 of 4 β all four join a configured directory to
a name this module generated, and the pattern matched the word filename rather than
anything about where the value came from. It now asks where the value came from instead,
and reports only a component that arrived from outside the file. Re-run: 0 findings,
from 4. That is less than the regex reported, on purpose; everything the regex reported
was false. eval-and-pickle-execute-their-input is 0 of 1, and that one is below.
Three more never fired at all, and silence is not a pass: it says 260,000 lines contain none of what they look for, which is also what a broken checker says. Two of the three are the two ruff already covers, which is the more likely explanation.
A checker defect, found by classifying rather than by testing. Two of the 50 were the
same fault: scan_source stripped # comments before matching and did not strip
string literals, so eval() named in a docstring arguing against using it, and a
datetime.now() inside a doctest example, both read as code. Docstrings are now emptied
before any pattern sees the file β docstrings only, since a triple-quoted SQL query is
data the checkers have a legitimate interest in β with the quotes, the newlines and the
length left in place so no line or column moves. Both findings are gone on the re-run.
A suppression that should not have fired. One finding β a consent-receipt timestamp
β sat inside a return statement carrying a # nosemgrep: odoo-sudo-without-context
marker four lines below it. Suppression read any marker within the statement without
asking which rule it named. That is the mute button this project says it will not build,
reached from the other direction. A marker that says what it is about β a code with
words in it β now covers only rules that share one of those words, so a marker about
sudo no longer speaks for a finding about datetimes, while the 141 OpenSPP sudo sites
carrying that same marker still read as decided. A code with no words in it, B608 or
E501, cannot be compared with anything and covers its line as before; that gap is
named in layers._marker_is_about rather than papered over.
How it checks itself
The nine layers below are the mechanism, not the pitch. Every answer goes through them, and each reports a verdict, what it checked, and what it could not check and why:
| Layer | Catches |
|---|---|
| Syntactic | Code that does not parse or compile |
| Structural | An answer missing the sections the format requires, or echoing the template |
| Semantic | An @api.constrains that omits a field its body reads β an AST pass, not a regex |
| Knowledge | A rule id that does not exist, or one recited rather than read |
| Domain | A rule cited correctly and then broken anyway |
| Context | An answer that is right in general and wrong for this module's Odoo series |
| Empirical | An import naming a module that is not installed here |
| Adversarial | A verdict that only holds for one particular formatting |
| Provenance | A verdict nobody could reconstruct later |
Most tools stop at the knowledge layer -- do the cited rules exist? Nexa does not, because a model will cite a rule correctly and break it in the same answer β every citation checking out while the code does the opposite of what it just quoted.
Citation existence is not rule compliance.
Compliance checking alone is not the end of it either. A regex over generated code
cannot see that @api.depends('line_ids') is incomplete when the body reads
line_ids.price. That is a set comparison, so Nexa parses the code and does it
properly.
It tells you what it did not check
Every layer reports its own coverage. A layer that cannot run says so, and the verdict carries that forward as INSUFFICIENT EVIDENCE rather than quietly counting as a pass β because nothing found and nothing to find are different claims.
Confidence comes back as four independent readings β structural, semantic, contextual, empirical β never averaged into a single score. One number reads high whenever the cheap checks ran, which is exactly the false reassurance this design removes.
REJECT (INSUFFICIENT EVIDENCE: the domain layer could not run)
confidence contextual partial empirical none semantic partial structural checked
Knowing the difference between verified and not yet falsified is what separates a validator you can build on from a green tick you cannot.
Install
See INSTALL.md. Short version: clone this repo, install Ollama, pull
three models (~22 GB, from Ollama's registry β not from here), run nexa build.
Usage
nexa banner and command list
nexa help <command> detail for one command
Ask
nexa ask odoo "constrain invoice date to the fiscal year of its period"
nexa ask odoo --deep "should this be a stored computed field or a related field?"
--deep uses the 30B instead of the 7B: roughly 11 tok/s instead of 37, noticeably
better at weighing trade-offs.
Run the full pipeline
nexa run odoo "cap a sale order line quantity at available stock"
Retrieves rules, drafts on the 7B, runs all nine layers, and escalates to the 30B once -- but only for findings a larger model could plausibly repair. A missing section label is a real finding that no bigger model fixes better, and spending 11 tok/s to discover that would make the rescue rate meaningless.
Escalation is bounded by declared workflow topology, so a run can never burn a second pass on the slow model.
Review a file
nexa check addons/my_module/models/sale_order.py
The checkers run over your file first, before any model is involved. Patterns and AST passes, each finding carrying the line it sits on. These do not depend on what a 7B says β or on whether it says anything useful at all:
6 checker finding(s) in the file
critical line 75 odoo.stored-compute-needs-complete-depends
_compute_labour_cost() reads 'job_id' but @api.depends does not declare it
-- the stored value goes stale when 'job_id' changes
critical line 181 odoo.constrains-must-list-every-field-read
_check_no_overlap() reads 'end_at' but @api.constrains declares only
engineer_id -- writing 'end_at' alone skips the check entirely
Then the model is asked to explain and fix those, and to look for what no pattern can decide. It is shown the findings, so it works from them instead of re-deriving them.
A file too big for the context window loses whole symbols, and they are named. The excerpt is chosen by symbol β methods, classes, functions β and the ones a checker fired on go in first, so the part of the file somebody needs an answer about is the part that survives:
! file is 7929 characters; 11 of its 20 symbols were reviewed
(not shown: FsmVisit.action_arrive, FsmVisit._timesheet_vals, ...)
The verdict carries it too, so a review of part of a file cannot print a bare PASS:
WARN (INSUFFICIENT EVIDENCE: 9 of 20 symbols were not reviewed)
--no-model stops after the checkers. 0.26 seconds, no Ollama, and the same
output every run β because what the generative half adds on top of the checkers is
small and does not hold still: 1 citation across 25 reviews on one run, 3 on
the next, and not the same rules either time. On those same two runs the deterministic
half produced the identical 23 findings both times.
That is the argument. A person reading a file wants the part no pattern can decide, and it is worth the wait. A CI gate wants the part that is the same every time, and three of five files gave different verdicts between those two runs. Those had been the same command.
It prints no verdict, because nothing generated an answer to have a verdict about. A clean run says no pattern fired in those words, and exits 1 if one did.
A .py or .xml file is reviewed against all 27 rules. Anything else is refused
by name.
$ nexa check web/src/partners.ts
! nexa reviews Odoo code: .py and .xml files. .ts is not one, and there
are no rules in this corpus that would say anything about it.
There is no second corpus to route to, so a plain Python script inside an Odoo tree β
scripts/, a migration helper β is reviewed like anything else. The nineteen ORM
rules have nothing to match in it, the eight plain-Python ones do, and the report says
which of the two it is rather than leaving the reader to infer it from a short list:
checking upload_handler.py as odoo
! no Odoo imports here and no __manifest__.py above it -- the 19 ORM rules
have nothing to match, so this file is effectively reviewed against the
8 plain-Python ones
Rules retrieved:
0.698 [critical] python.assert-is-not-validation
0.685 [critical] python.eval-and-pickle-execute-their-input
0.674 [high] python.path-join-does-not-contain-a-path
0.643 [critical] python.subprocess-shell-true-is-injection
Unlike nexa ask, this one has surroundings to read: the nearest
__manifest__.py gives the Odoo series, the file gives its model names and existing
imports, ir.model.access.csv and the record rules in XML say what Python cannot.
Those facts go into the question and into the context layer.
Where they say the corpus was verified somewhere else, that is reported as a gap, not a finding β it is true of the corpus whatever the answer says, and a verdict that fires on every run in every project off the newest series stops being read:
Not checked
context version-specific behaviour
this module targets Odoo 17.0 and the corpus was checked against
18.0, so anything version-specific in these rules is unverified here
A rule that was actually retrieved and genuinely does not apply to this series is a different claim, and stays a finding with the rule id attached.
Ask about your own code
nexa index C:/work/our-standards
nexa advise "how do we handle schema migrations here?"
index embeds a folder of your own source and documentation into a separate table
from the rule index. advise reads it back.
This is the one command not grounded in the rule corpus, so nothing it says is
checked against a rule id or a violation pattern β it answers from whatever you
indexed, and says so when the material does not cover the question. Use ask, run
and check for anything you want verified.
Show the record behind a verdict
nexa evidence
nexa evidence ask-34cf01432145
Every run writes one bundle: the corpus hash, the rules retrieved with their scores and which arrived by pin, every layer's verdict and findings, the checker versions, the model and its options, and the timings.
A verdict nobody can reconstruct is a claim, not a result. Question and answer text
are omitted unless LOCALAI_TELEMETRY_CONTENT=1, for the same reason telemetry omits
them β this runs on a machine used for client work. NEXA_EVIDENCE=0 turns bundles
off.
Explore the corpus
nexa rules
nexa rules onchange
nexa rules --full odoo.constrains-ignores-dotted-names
β marks a rule with an automated check. β marks one you have to read the answer
for yourself. PIN marks a rule injected into every answer regardless of retrieval.
Keep it healthy
nexa doctor
Checks Ollama, the models, GPU residency, and the traps that silently cost ~3x throughput on a 6 GB card. Run this first whenever something feels slow.
nexa models
nexa report
Change the rules
Edit corpus/build_corpus.py, then:
nexa build
That regenerates the rule JSON, validates it, measures pattern false positives,
attacks the checkers, re-embeds the retrieval index, and rewrites the Modelfiles. It prints the
ollama create commands to register the updated models.
The corpus is the source of truth
corpus/rules/*.json holds the rules as structured data β a citable id, two
graded axes, the failure it prevents, a wrong/correct code pair, and where possible
an upstream documentation URL and a regex that detects the violation in generated
code.
The axes are separate because they answer different questions. severity is how
bad it is if this happens; likelihood is how likely a finding is the bug, graded
from what nexa recall --aim measured against real fix commits rather than from
judgement. A rule can be severe and rarely the point β a Python uniqueness check
really does race, and across 237 in-scope fixes in two ecosystems not one of them
was about that β and one field could not say so.
Everything else is derived and must never be hand-edited:
corpus/rules/odoo.json --+--> modelfiles/gen-odoo.Modelfile (system prompt)
+--> bin/index.sqlite (retrieval)
+--> every answer (grounding + checking)
Same principle as z.infer<typeof Schema>: one definition, derived artifacts, no drift.
Current coverage
| Rules | 27 β 19 Odoo ORM, 8 plain-Python, one stack |
| Critical | 12 |
| Machine-checkable | 26 (96%) β 12 by pattern, 14 by AST |
| Traceable to upstream docs | 25 (93%), each audited for topic fit |
| Pattern false positives | 0 cross-hits across the corpus |
| Recall under meaning-preserving rewrite | 79/79 |
| Satisfied patterns bypassable by a comment | 0 of 3 |
| Retrievable by their own probe | 27 of 27 at top 6, 25 of them at top 3 |
| Hit rate on 33,960 lines of reviewed OCA code | 3.3 per 1000 lines |
| Those 120 hits, classified by hand | 86 true positives, 21 false, 13 undecidable from one file |
| Hit rate on 226,937 lines of OpenSPP (Odoo 19) | 0.9 per 1000 lines, +69 another tool had already flagged, +71 advisory |
| Every one of those findings, classified by hand | 157 true positives, 137 false, 6 undecidable |
β¦the two depends/constrains rules alone |
87 true, 15 false, 3 undecidable β 85% |
| β¦the 8 composed-in Python rules alone | 26 true, 24 false β 52%, and 20 of the 26 are one rule |
β¦sudo-bypasses-record-rules alone, before it was narrowed |
6 true, 97 false β 5.8%, and the reason it is now an AST pass |
| Recall against real Odoo bugfixes | 1 of 56 (OCA), 7 of 162 (OpenSPP). See below. |
| Verdict stability, 12 questions x 5 runs | 12/12 same verdict every run |
| Escalation rescued a bad draft | 11 of 21, 95% CI 32β72% |
nexa check, 5 files x 5 runs |
23 findings reported, in 5 of 5 files β every one the corpus can make |
| The same 23, on a second run of the identical command | 23 β the deterministic half does not move |
| Rules the model added beyond the checkers | 1 on one run, 3 on the next, out of 25 reviews |
nexa check verdict stability |
4 of 5 files on one run, 3 of 5 on the next |
nexa check --no-model |
0.26s, identical output every run |
26 of 27 rules carry an automated check. The one that does not β
schema-change-needs-migration, whose violation lives in the diff between two module
versions β returns no verdict rather than a false pass, and nexa coverage names
it and says why.
Two lines above are worth reading together. The Odoo half is 85% on its two flagship rules; the composed-in Python half is 52%. They are in the same corpus and they are not the same quality, and the table says so rather than reporting one blended number that describes neither.
The checkers are tested, then attacked
Patterns are tested against their own samples
Every rule carries a wrong and a correct sample, so nexa validate tests each
pattern against both. Three faults are hard errors:
- a pattern that does not match its own
wrongsample β it catches nothing - a pattern that fires on its own
correctsample β it cries wolf on good code - a
satisfied_patternthat clears its ownwrongsample β inert by construction
AST checkers owe the same contract and are self-tested the same way.
Cross-checking measures false positives
nexa crosscheck fires every checker at every other rule's correct sample. That is
the false-positive rate, and it decides whether a checker gets trusted or ignored.
Then the checkers get attacked
nexa adversarial
validate and crosscheck both compare a checker against code written alongside it,
by the same hand, in the same sitting. Neither asks what it takes to get bad code
past it. Three probes do:
| Probe | Question |
|---|---|
evasion |
Reformat the wrong sample without changing its meaning. Does the checker still see it? |
counterexample |
Can a satisfied_pattern be cleared by a comment quoting the fix instead of applying it? |
contradict |
When one rule fires on another's correct sample, is that breadth or do the two rules disagree? |
This runs as part of nexa build. A checker that has never been attacked is a checker
nobody has tested.
The probes hold the corpus to a measured standard, and it currently meets all three: 99/99 rewrites still caught, 0 of 13 satisfied patterns bypassable by a comment, 0 contradictions between rules. Those numbers are regenerated on every build, so a checker cannot quietly weaken.
And then pointed at code nobody wrote for them
validate, crosscheck and adversarial all compare a checker against the 32
samples in this corpus β written by the same hand that wrote the checkers. Passing
proves self-consistency and nothing else.
nexa fieldtest ../server-tools ../account-financial-tools
fires every checker at a tree of somebody else's source and reports the hit rate per thousand lines. On code that already works, every hit is a question about the checker, not about the code.
Against OCA/server-tools and OCA/account-financial-tools β 606 files, 34,550 lines of reviewed, merged Odoo:
| Hits | Per 1000 lines | |
|---|---|---|
| Before | 397 | 11.3 |
| After narrowing five checkers | 135 | 3.9 |
Those two rows are one tuning session, against the snapshot of those repos it was run
on. Re-measured 2026-09-15 β the same two repos on branch 18.0, at server-tools
60cd54e5e and account-financial-tools e4c1b86aa β the corpus finds 100 hits, 2.9
per 1000 lines. Still 606 files. The line count reads 33,960 against the 34,550
above, and 602 of that gap is the phantom line per file fieldtest used to add to its
own denominator. Under the old counter today's tree reads 34,566 against that
34,550, so the two snapshots are sixteen lines apart and the comparison is close to
honest.
All 100 were then read by hand β 79 true positives, 8 false, 13 undecidable from a
single file β and every verdict is in classification-oca-2026-09.json. The 8 was
down from 24 on the first pass: classifying them produced a worklist, and four of the
seven classes turned out to be one predicate each on evidence already in the file. All
79 true positives survived that narrowing, which is the only thing that makes it safe.
Re-measured after the sudo rewrite below: 102 hits, 3.0 per 1000 lines, 79 true, 10
false, 13 undecidable. Two changes moved it in opposite directions and both are worth
naming. The AST pass also sees .sudo().search_count( and .sudo().search_read(, which
the regex could not, so three findings are new β and all three are false, one of them the
same registry-model class the original classification already had. Against that, one
false positive left, a sql-needs-query-parameters carrying a marker on the line above
it that suppression now reads. All 79 true positives survived both changes, which is
again the only thing that makes the rest of it safe.
Re-measured again on 2026-09-16, after the eight Python rules were composed in:
120 hits, 3.5 per 1000 lines, 86 true, 21 false, 13 undecidable. The 18 new ones are
7 true and 11 false, read the same way and listed in
classification-python-2026-09.json. The rate went up and the precision went down, and
that is the trade the section above prices.
Re-measured again on 2026-09-21, after the six false-positive causes named above were
closed: 112 hits, 3.3 per 1000 lines. Four known false positives are gone β three
path-join and one decimal-for-money β and three new float-comparison findings
arrived, each read by hand, from a satisfied_pattern that had been clearing whole
files on a float_compare written somewhere else in them.
The check that makes all of this safe is the same one every time, and it is worth
stating as a number rather than a promise: across all five classification artifacts,
244 hand-classified true positives, 244 still found, none lost. That is asserted by
re-running the checkers against the same two checkouts and matching every true verdict
back to a live finding, not by reading the diff and hoping.
The same treatment on a second ecosystem
A precision number from one ecosystem is a precision number for that ecosystem. The two
rules this project leads with β stored-compute-needs-complete-depends and
constrains-must-list-every-field-read β make 105 findings across OpenSPP's 226,937
lines, and all 105 have now been read against the file they sit in. Every verdict is in
classification-openspp-2026-09.json.
| true | false | undecidable | |
|---|---|---|---|
stored-compute-needs-complete-depends |
49 | 8 | 3 |
constrains-must-list-every-field-read |
38 | 7 | 0 |
| 87 | 15 | 3 |
85%, not the 100% these two scored on OCA. The gap is worth stating plainly: a rule
measured on one tree is not measured. Five of the fifteen are one class β a field read
only as an argument to a _logger call, which is neither a dependency nor a trigger β
and three more are a second: a stored compute whose own @api.depends names only fields
the decorator already declares, so naming it adds no trigger. Two predicates would take
15 to 7, the same shape the OCA worklist had.
One of the fifteen is worse than a false positive and is recorded as such:
_check_stage_requirements reads has_active_plan, which is a non-stored compute, and
Odoo rejects a non-stored field in @api.constrains with "parameter is not writeable".
The gap the finding points at is real; the fix it names cannot be applied. OpenSPP hit
that same Odoo behaviour themselves and removed an @api.constrains("age") over it.
The rule that classification killed
The other 163 findings were read the same way, and one rule accounted for most of what
was wrong with the output. sudo-bypasses-record-rules was a regex for .sudo().search(
and made 106 of the 268 live findings, every one marked critical. Classified:
6 true, 97 false, 3 undecidable β 5.8%.
That is not a rule that needs tuning. It was asking half a question. Its own sentence is
that sudo "drops ir.rule filtering", and the regex could see the sudo and never the
filtering:
- 73 were on models the tree gives no
ir.ruleat all, so there was nothing to drop. - 10 more on
spp.vocabulary.code, whose single rule is[(1, '=', 1)]β a grant that matches every row and filters nothing. - 14 re-applied scoping by hand, in a spelling the rule did not recognise because it
named only
company_id.
So it is now an AST pass that asks the tree the second half: context.tree_facts reads
every <record model="ir.rule"> in the walk and the checker fires only where a filtering
rule exists. The trigger only ever suppresses. A tree that was not walked, a partial
walk, a model the walk never saw declared β each reads UNKNOWN and the finding stands,
so nexa check on a single file behaves exactly as it did before.
That last clause was learned the hard way, in the first version of this trigger. A walk
of an addons tree sees the addons' rules and not Odoo's own, so product.template β which
carries core's company rule from the product module β read as ABSENT and three true
positives disappeared: a paginated API listing every company's products. Silence about a
model nobody declared is UNKNOWN, not "no rule".
| before | after | |
|---|---|---|
sudo-bypasses-record-rules, live |
106 | 2 |
| All live findings on OpenSPP | 268 | 162 |
| Per 1000 lines | 1.18 | 0.71 |
| Of the 6 classified true positives, kept | β | 6 |
The last row is the only one that makes the others safe.
That 162 is the Odoo rules alone. Composing the eight Python rules in took it to 194, 0.9 per 1000 lines β the 32 new findings are 19 true and 13 false, and they are the subject of the composition section.
What is still wrong, named
float-comparison-needs-float-utils was 2 true, 5 false. All five compared an
integer to zero β search_count, a read_group _count, SELECT COUNT(*), a
fields.Integer β and the regex could not see it because it matched the variable's
name, not its value. Three of the five and both true positives were spelled total == 0.
The checker now follows the assignment, and the five are gone. It also stopped clearing
the whole file on a float_compare anywhere in it, which the old satisfied_pattern
did: on OCA that surfaces three Monetary and Float fields compared with != 0 in
files that use float_compare elsewhere, each read by hand. OpenSPP 7 β 2, OCA 1 β 3.
onchange-is-not-validation was 4 true, 3 false, and the three were not what they
looked like. Each is a four-line onchange that assigns a field, followed by an action
method that raises β and the regex read 400 characters past the decorator with no way to
know where the method ended. _onchange_variable_id sets a target model; the
UserError 200 characters below it belongs to action_remap. Scoped to the decorated
function, the three are gone. Three new ones arrived in their place and were read: a
phone-number validation living only in an onchange with no constraint in the file, a
_check_date_collected onchange raising exceptions.ValidationError β a spelling the
regex's \w* never matched β and one that clears a field and returns the warning dict.
Count unchanged at 7; set changed entirely.
The composed-in Python half's four are fixed and measured in the section above.
sudo-bypasses-record-rules now checks four clauses rather than one: a query under
sudo (search, search_count, search_read, read_group), a touch (create, write,
unlink, copy, read), a sudo recordset bound to a name and passed on, and a name
rebound to its own sudo form. The query clause is the one that was always checked
and it keeps its 6 hand-classified true positives.
Every live finding the new clauses made across both trees was read by hand β 0 true,
24 false, in classification-sudo-2026-09.json. Each is code where the caller is an
administrator by construction, a demo-data seeder or a wizard behind
group_account_manager, or a framework model with no tenant boundary at all. So the
touch and escape clauses report at likelihood unproven, which makes their findings
advisory: shown, counted apart, gating nothing. The grade is carried by the finding and
not by the rule, so the query clause keeps defect. A clause the rule's own sentence
asks for is worth having under measurement; a clause measured at zero is not worth
failing a build on.
The rebind clause is the exception and keeps defect, because it has the one thing the
others lack: an incident. It turns the miss this README names by hand β
"remove sudo() when importing record" β into an on-target catch, which is the same
evidence every other defect grade in the corpus rests on. It also produces one false
positive at HEAD, and both facts are in the artifact.
What the first run found:
onchange-is-not-validationfired on all 16@api.onchangedeclarations that had no constraint elsewhere in the file. Not one was validating anything β they set a field from a provider, fill a default, narrow a domain. It now requires the onchange to actually try to enforce something. 16 β 0.stored-compute-needs-complete-dependsdemanded@api.dependsentries the ORM would refuse: paths ending in.id, and fields the method computes itself. 121 β 95. Its true positives stand βauditlog's storeddisplay_namereadsuser_id.nameand declares onlyuser_id, so the name goes stale on a rename.prefer-sql-constraintread a guard as the whole rule.if record.storage == "url" and not record.url:β the first half is anast.Compareand the second is not, so it concluded the function "does nothing else". 8 β 2, and both survivors are true positives.ondelete-is-a-decisionwas the one left loud on purpose. Odoo defaults a requiredMany2onetoondelete='restrict', which is what the rule recommends, so 56 of its 198 hits were fields already doing the right thing and are now exempt. That left 142 optionalMany2ones genuinely defaulting toset null, at 4.1 per 1000 lines. A second pass then asked where the harm actually lands and moved the checker to an AST pass that fires only where something reads through the relation, which took it to 33; the whole-tree trigger work took it further again. On the 2026-09-15 run above it fires 12 times, 0.35 per 1000 lines.
What it misses
Every number above is a precision number. A corpus containing nothing at all would score perfectly on all of them, so none of them is evidence that the corpus is worth having.
nexa recall ../server-tools ../account-financial-tools
reads a repository's git history, finds the commits where somebody fixed something, and runs the checkers over each file as it stood before the fix and again after. A finding that was there before, is gone after, and sat on a line the fix actually changed is the corpus noticing the bug that was really fixed.
Across 386 fix commits in those two repositories, 64 of which touched a construct the corpus covers:
| 2026-09-16 | 2026-09-21 | |
|---|---|---|
| Caught | 1 | 4 |
| Missed | 55 | 60 |
| Recall against in-scope fixes | 2% | 6% |
| Rules that caught anything | 1 of 14 | 3 of 27 |
The right-hand column is after the pass in Β§3.47 of ARCHITECTURE. Three of the
four catches are sudo-bypasses-record-rules, which grew the two clauses its own
sentence always asked for; the in-scope denominator went up as well, from 56 to
64, because the corpus now covers more constructs than it did.
That condition, on a line the fix changed, is the entire measurement.
Without it, any refactor that moves code counts as a catch. The first version of
this tool did not have it and reported 12. Reading the commit subjects killed
the result: "fix image url in locales with thousand separator", "poor
performance updating assets massively", "fix JS crash in analytic
distribution". Not one was an ondelete bug. The checker had simply gone quiet
because a dereference moved.
What this means, stated plainly. Nexa is precise and narrow. It is right about what it describes, and it describes very little of what actually goes wrong. Three bugs squarely inside its own stated territory were named here as misses, and one of them is now a catch:
- "remove sudo() when importing record" β caught, 2026-09-21. The line the
fix deletes is
self = self.sudo(), which rebinds the method's own receiver so that everything below it runs elevated without naming sudo again. Every clause looking for.sudo().method()was blind to it. The commit says what it cost: "finding multiple records across all companies during import when they have same record name." - "prevent removal of x2many values from inaccessible companies" β still
missed. A multi-company leak, which
check-company-on-relationsis about. - "log computed fields stored in db as expected" β still missed. A stored compute.
What this does not mean. The denominator is generous. Many of those 64 commits touched a covered construct while fixing something else entirely, so 6% understates recall against validation bugs specifically. Two repositories in one ecosystem is also a narrow sample. The number is still a floor; it is a higher one than it was.
The missed list is the more useful output. Every entry is a bug somebody actually fixed that the corpus had nothing to say about, which makes it a candidate rule with a real incident behind it rather than another one written from an opinion.
What the embedder drops
Retrieval embeds the question and sorts the rule vectors against it. For a long file
that question is large, and nomic-embed-text truncates what will not fit without
saying so. Appending a marker and watching the vector fail to move puts the cut at
~7,400 characters β the last 6% of the longest subject's question, not the back
half anybody would have guessed.
nexa eval --compare-retrieval
scores a compact description of a file against the whole question, using the rules the checkers proved apply as ground truth. The compact query wins 15β14 out of 19: one rule, on one file, which is a coin toss with a table around it. So it is not adopted. The function stays as the thing that comparison compares, and the committed behaviour waits for a number that says something.
What is adopted needed no measuring: a rule the checkers already fired on at line 75 goes into the grounding without asking similarity to guess at it.
The missed list is the worklist
nexa recall ../OpenSPP2-19.0 --harvest
clusters the fixes the corpus did not flag on the line of the fix by the construct each diff touched, so the reading order is the writing order. Against OpenSPP 2 (Odoo 19) β a second ecosystem on a newer series than the OCA repositories:
| 2026-09-16 | 2026-09-21 | |
|---|---|---|
| Fix commits examined | 406 | 409 |
| β¦touching a construct the corpus covers | 160 | 173 |
| Caught | 6 | 11 |
| Recall against in-scope fixes | 4% | 6% |
A third ecosystem, and it agrees. The criticism this section used to carry was that two repositories in one ecosystem is a narrow sample. So the same pass was run against odoo core 18.0 β 559,663 lines, 190,000 commits β capped at 400 fix commits:
| OpenSPP | OCA | odoo core | all three | |
|---|---|---|---|---|
| Fix commits examined | 409 | 386 | 369 | 1,164 |
| β¦touching a covered construct | 173 | 64 | 97 | 334 |
| Caught | 11 | 4 | 5 | 20 |
| Recall against in-scope fixes | 6% | 6% | 5% | 6% |
Three ecosystems arriving at the same number from different directions is worth more than any one of them. It was 2% and 4% on two trees before this pass. The floor is still a floor; it is a better-founded one.
Precision on core is higher than on either other tree: 2,663 findings at 4.8 per 1000 lines after the narrowing below. Odoo core is the reference implementation, so a rate there is a statement about the rules and not about somebody's house style.
The first classification here that is a sample rather than a census. Reading all
2,784 findings is not a session's work, and 2,784 is twenty-five times any set read here
before. So 15 findings of the largest rule β stored-compute-needs-complete-depends,
1,051 of the total β were drawn with a fixed seed and read by hand:
11 true, 4 false, in classification-odoo-core-2026-09.json. That is 73%, with a
95% Wilson interval of roughly 48% to 89%. The interval is wide, and it is the honest
number for fifteen: it says the rule is not mostly noise on core and not much more. The
draw reproduces from the artifact's own description, without the script that made it.
All four false verdicts have one cause. odoo/models.py gives every model
display_name = fields.Char(automatic=True, compute='_compute_display_name', ...) with
no store, so an override that omits a dependency cannot leave a stale row: there is
no row. The checker read "not declared in this file" as unknown and kept the finding,
which is right for a field declared in another module and wrong for one the framework
declares itself. 121 of that rule's 1,051 findings on core were this shape. The
narrowing is free and that was checked, not assumed: all ten hand-verified true
positives that compute display_name declare store=True on it in their own file, so
they are untouched.
The larger thing the sample turned up was deliberately not acted on. 837 of core's
1,360 stored computes are readonly=False β the "compute a smart default, then let the
user own it" pattern, where a narrow @api.depends is often protecting a user's edit
rather than forgetting a dependency. That would explain most of this rule's volume on
core, and it is not a safe narrowing: three of the 47 resolvable hand-verified true
positives are themselves readonly=False, and so are nine of the eleven true verdicts
in this sample. The number big enough to justify the rule is the number that refutes it.
The next three rules by volume were read the same way, 15 each, so 60 of core's findings are now classified across the four rules that make 70% of them:
| rule | core | previously measured |
|---|---|---|
ondelete-is-a-decision |
13/15 | 12/12 OCA, 14/14 OpenSPP |
stored-compute-needs-complete-depends |
11/15 | 49/57 OCA |
assert-is-not-validation |
3/15 | 6/10 OCA |
sudo-bypasses-record-rules |
0/15 | 6 true on OpenSPP |
Precision turns out to depend on what kind of code is under review, and this is the
first tree here that made that legible. Two rules hold up. Two collapse, and not because
the checkers broke. assert-is-not-validation's own true class is the asserted value
came from outside the module, and a framework's asserts are invariants between its own
components β the registry, the field descriptors, the query builder β where there is no
outside. sudo-bypasses-record-rules describes an application leaking between tenants,
and core is the thing that implements tenancy: every finding is a public route resolving
a token before identity exists, a delete guard that must see what the caller cannot, or
a cross-company sync whose whole purpose is to cross companies.
Neither moves a grade. A precision number is about a subject, and nobody reviews odoo core with this. It is the strongest argument so far that one precision figure for "the corpus" says less than the same figure per kind of codebase.
Reading them turned up three more checker defects, all of the same family as the
sudo(False) one above: a model with _auto = False is a SQL view with no foreign key,
so ondelete is not a decision about it; sudo(flag=False) is de-escalation by
keyword; and x.sudo(self.env.su) restores the ambient level and cannot raise
privilege. Core is now 2,650 findings at 4.7 per 1000 lines.
One finding is worth acting on outside this project. Core's
ir.model.access.group_names_with_access interpolates its access_mode argument
straight into SQL as perm_{access_mode}, and the only thing constraining it to four
literals is an assert. The method is public and RPC-callable. Odoo does not run under
python -O, so it is latent rather than live β but it is exactly the harm the rule
describes, in the reference implementation.
Still unread: roughly 1,590 findings on core, led by naive-and-aware-datetimes (167),
constrains-must-list-every-field-read (151) and depends-must-not-name-unread-fields
(134).
Where the OpenSPP clustering stands now:
missed caught spoke silent
.sudo( 67 9 52 15
ValidationError 40 2 27 13
Many2one 26 1 14 12
api.depends 23 0 14 9
noupdate 22 0 9 13
api.constrains 16 0 10 6
spoke is the column that changed the reading. A miss is not automatically a
blind spot: a checker firing somewhere else in the same file means the rule exists
and is aimed wrong, which wants a sharper rule rather than a new one. Ninety-seven
of the 162 misses are that. The first version of this table reported 21x api.depends (nothing caught here), which reads as a missing rule β and the checker
was in fact firing on sixteen of those files while the fixes went elsewhere.
.sudo( stayed the largest cluster and was worked twice. Its silent column fell
from 23 to 15 and its caught column rose from 6 to 9, because the rule grew the two
clauses its own sentence asked for and then a third, self = self.sudo(), taken
from a miss in the OCA list. noupdate is now the cluster with the most silence
against the least coverage, and it is next.
Reading the diffs behind the seven that were silent produced one rule,
odoo.depends-must-not-name-unread-fields. Two of the remaining rules in this corpus
came from the crosswalk below.
An outside opinion, which is the one thing the corpus cannot give itself
nexa crosswalk ../OpenSPP2-19.0/.semgrep/odoo-security.yml --checkout ../OpenSPP2-19.0
Every other number here tests whether what we wrote behaves. None of them can ask whether it is the right set of rules to have written, because a rule nobody thought of is invisible to every check that reads the rules. OpenSPP runs semgrep against 44 hand-written Odoo rules of its own β somebody else's judgement, on the same platform, formed without reading ours.
Semgrep is never run: each of their patterns is turned into the smallest snippet it would match and our checkers run over that, so a pairing rests on a checker firing rather than on two rule descriptions sounding alike.
| first run | now | |
|---|---|---|
| Their patterns turned into a probe | 146 of 146 | 146 of 146 |
| Both corpora fire on the same construct | 6 | 10 |
| Theirs fires, nothing here does | 38 | 34 |
Lines their analyser fired on (# nosemgrep: markers) |
631 | 631 |
| β¦where a checker here also fires | 20 | 20 |
The agreement moved from 6 to 10 because the gap it named got closed, not because anything was re-scored β the first six were all reached by the two Python rules about executing input, and not one by an Odoo rule.
Most of that gap is a different subject, not a missing rule: theirs is about security
and this one is about ORM correctness. Two entries were neither, and are now rules β
odoo.commit-inside-a-loop-splits-the-transaction and
odoo.sql-needs-query-parameters. The commit rule fires exactly once in 226,937 lines
of OpenSPP, inside a loop their analyser had flagged too: # nosemgrep: odoo-commit-in-loop β batch consent expiry cron. Two analysers, one construct, and a
person who had already decided.
They do not anchor to the same line, though, and that is worth knowing. Semgrep matches
the whole loop and its marker sits on the for; this checker reports the commit() ten
lines inside it, so the marker does not carry across. A marker that silences findings it
was not written about is the mute button this project will not build.
Reading a marker only on the finding's own line was too strict, and there is now a
number on it. Of the 106 .sudo().search( findings on OpenSPP, 105 carried a
# nosemgrep: about that exact construct and not one shared a line with it β the call
is chained across four lines and the checker anchors on .sudo() while the marker sits
above it. A hundred and five sites somebody had already read and justified were being
re-reported as fresh critical findings.
So scan_source now reads a marker across the statement it sits in, and the line
immediately above it β which is where semgrep, ruff and pylint all document their own
markers as belonging. The unit is a simple statement, one with no body of its own:
compound statements are deliberately excluded, because letting a marker on an if
header cover the whole block is the mute button again. It is the same principle read
more carefully, not a weaker one β and it still cannot reach the next statement.
On its own that took OpenSPP's suppressed count from 26 to 133. It reads 68 now, because the rule that produced most of those markers was narrowed in the same pass and no longer makes the findings at all β see below. Both changes were worth making: one stops re-reporting what somebody has already read, the other stops asking the wrong question.
Every version claim is checked against two kinds of evidence
python corpus/check_sources.py # what the docs describe
python corpus/check_sources.py --source # what the source defines
Each Odoo rule declares the series it holds for. The first mode checks that claim against the documentation: 45 of the 48 versioned rule-series cite a page that exists and names the API, 2 more are tutorials that moved while the API stayed, and 1 is not supported by any 16.0 page. The 9 remaining cite something other than the Odoo docs β Postgres, the contributing guidelines β and are reported as unversioned rather than counted as passing.
Documentation has a blind spot, and the corpus had fallen into it β a function Odoo
shipped and never described reads as missing. So the second mode reads odoo/odoo
itself and asks whether the API is defined in that series' module: def float_compare( in odoo/tools/float_utils.py, not the word anywhere on a page.
| Rule-series defined in source across 16.0, 17.0 and 18.0 | 57 of 57 |
That closes it. No version claim in this corpus rests on an assertion, and the one
documentation gap now carries a measurement instead of a belief: float_compare's
signature is byte-identical in all three series. Neither mode is a test run β a
definition says the API existed and was spelled the same way, not that it behaved
identically β and both say so in their own output.
Authoring-time and the only thing here that touches the network. Nothing at query time depends on it.
Is every rule findable?
A rule the retrieval never surfaces is unreachable whatever it says. Every rule carries a probe β one question phrased the way somebody meets that problem β and
nexa eval --retrieval
asks each rule its own probe and reports whether it comes back. No generation, so it needs only the embedding model and finishes in seconds.
All 27 are retrievable at the operating TOP_K of 6, and 25 of them rank in the top
3. That number got harder rather than easier when the corpus became one stack: with
19 rules competing, prefer-sql-constraint came back first for "stop anyone entering
a negative quantity on a sale order line"; with 27 it stopped coming back at all,
because that is the generic validation question every constraint rule here answers.
Reworded to the question only it answers β "should this check be a database
constraint or an api.constrains method" β and measured again. It is the third rule to
fail this way, and all three failed identically: correctly worded, in the vocabulary of
somebody who already knows the answer. no-monkey-patching did not answer "change
what a core sale.order method does" either.
The report states its own limits. Pinned rules are prepended regardless of score, so ranking against them measures the pin rather than the wording, and the header says so instead of counting them.
And what does nexa check actually report?
Every number above was measured on nexa ask. nexa check is the command you
would point at your own file, and nothing measured it β it was assumed to inherit
ask's numbers because it ends in the same call.
nexa eval --check
runs a fixed set of files the way nexa eval runs a fixed set of questions, and
adds the four things only a file can produce: which corpus it was routed to, how
much of it fitted, what the surroundings on disk let the context layer say, and
whether the answer mentions the file at all.
Five files, five runs each:
| Findings reported about the files | 23, in 5 of 5 files |
| β¦which is what the corpus can find in them | 23 β nexa fieldtest, no model |
| WARN | 18 of 25, 95% CI 52β86% |
| REJECT | 7 of 25, 95% CI 14β48% |
| PASS | 0 of 25 |
| Same verdict every run | 4 of 5 files |
| Rules the model cited that no checker found | 1, across 25 reviews |
The deterministic half does everything it is capable of. Those 23 findings are
exactly what nexa fieldtest extracts from the same five files with no model
installed: an incomplete @api.depends on three stored computes, an
@api.constrains reading two fields it does not declare, shell=True, a bare
except, a mutable default argument, JSON.parse unchecked, a rejected promise
nothing catches. Each carries a line number and none depends on what a 7B says.
The first run of this eval reported none of them, because every layer ran over the model's reply and nothing had ever read the file.
The generative half adds close to nothing, and now there is a number. Across 25 reviews the model cited exactly one rule the checkers had not already found. It does make the answers more about the file than they were β feeding it the findings raised the symbols it names from 34 to 90 across the set β but what it adds to the corpus's own verdict is one citation in twenty-five.
That is worth knowing rather than worth hiding. It says where to spend effort: another rule in the corpus is worth more than another prompt.
Coverage is evidence. fsm_visit.py is 7,929 characters against a
6,000-character budget, so nine of its twenty symbols do not fit β but the excerpt
is chosen by symbol and the flagged ones go first, so _check_no_overlap, its
last method and the home of three of its six findings, is reviewed rather than cut
off 1,900 characters early. What is left out is named, and the verdict says so:
WARN (INSUFFICIENT EVIDENCE: 9 of 20 symbols were not reviewed)
That file used to PASS on five runs of five, because nothing could see it.
How it got here. The first run of this eval found check the wrong shape in two
ways, neither about the model's quality. The checkers ran over the model's answer,
so the corpus could find six things in a file and the command reported none of them.
And the excerpt was the first 6,000 characters, which cut one subject 1,900 characters
above the method carrying half its findings. Both are fixed, and the table above is
what that is worth.
subjects/ is committed so the run repeats. Point it at your own tree for the
half a written-for-the-measurement set cannot tell you:
nexa eval --check addons/my_module --limit 6
Environment
Everything below is optional; unset, each is the default it has always been.
| Variable | Effect |
|---|---|
OLLAMA_HOST |
Where Ollama is. 11500, box:11434 or https://gpu.lan all work β the same spellings the server accepts. Default http://127.0.0.1:11434; nexa doctor reports which it used and whether you set it. |
NEXA_EVIDENCE=0 |
Write no evidence bundles. |
NEXA_EVIDENCE_DIR |
Where bundles go. Default bin/evidence/. |
NEXA_EVIDENCE_KEEP |
How many to keep. Default 200, pruned oldest first. |
LOCALAI_TELEMETRY=0 |
Record no telemetry at all. |
LOCALAI_TELEMETRY_FILE |
Where the JSONL goes. Default bin/telemetry.jsonl. |
LOCALAI_TELEMETRY_CONTENT=1 |
Record question and answer text. Off by default: this runs on a machine used for client work. |
NO_COLOR / TERM=dumb |
No ANSI escapes. Colour is also off automatically when stdout is not a terminal. |
NEXA_PYTHON |
Which interpreter the ./nexa launcher uses. |
Requirements
- NVIDIA GPU with 6 GB VRAM or more (developed on an RTX A3000 Laptop)
- 32 GB system RAM for the 30B escalation model; 16 GB works without it
- Python 3.10+ β stdlib only, nothing to pip install
- Ollama 0.30 or later
- ~25 GB free disk
Optional: LLM Workflow Router for
nexa run. nexa ask works without it.
Documentation
- INSTALL.md β setup from scratch
- ARCHITECTURE.md β how it works, every design decision and why it was made, and an honest account of what it cannot do
- LICENSING.md β what is free and what needs a license
License
Source-available under the PolyForm Noncommercial License 1.0.0 β free to read, run, modify and evaluate for any noncommercial purpose. Commercial and client-work use requires a license; see LICENSING.md.
Nexa runs entirely on your own hardware, so a commercial license carries no per-seat metering and no dependency on anything of mine staying online. Nothing phones home.
The models Nexa orchestrates are pulled from Ollama's registry and carry their own upstream licenses.
Copyright Β© 2026 Doby Baxter
What you get
Nexa is built to close one gap precisely: the Odoo defects that error nowhere, fail no test, and look correct in review β 10% of the fixes in a real platform's history, found late and expensively every time. Everything else here exists to make that claim checkable: the layers, the classification, and the numbers published against it.
It is free, it runs entirely offline, your client code never leaves the machine, and the standard it applies is the one you wrote β not a general impression of good practice.
Read the output. Two parts are worth more than the verdict itself: the model's own "Not handled:" section, and the "Not checked" list the layers print beneath it.
The full design record β every decision, the evidence behind it, and the trade-offs taken β is in ARCHITECTURE.md.