---
license: apache-2.0
tags:
- decision-models
- masked-diffusion
- scientific-ml
- calibration
---

# Sciev — Open System-One Scientific Decision Models

Typed decision heads on a frozen **LLaDA-8B-Instruct** masked-diffusion
backbone. Unstructured `state` + typed questions (`choice` / `noul` /
`score`) → structured decisions with probabilities in **one forward pass
per decision**. No text generation, no output parsing.

- Code: <https://github.com/alrobles/sciev> (`pip install sciev`)
- Site: <https://sciev.org>
- Manuscript: <https://sciev.org/static/sciev-paper.pdf?v=030> (in
  preparation for arXiv submission)
- Release: <https://github.com/alrobles/sciev/releases/tag/v0.3.0>

## Contents

| file | role | sha256 (prefix) |
|---|---|---|
| `release/fr_choice.pt` | `choice` head — AttnPoolHead, 67.2M | `67efbc59` |
| `release/alp25_noul.pt` | anti-leakage `noul` head — MLP, 16.8M | `335e9730` |
| `release/alp25_score.pt` | anti-leakage `score` head — MLP + ordinal aux loss, 16.8M | `e1441386` |
| `release/fr_noul.pt` | v0.2 `noul` head, superseded (provenance) | `317bfa5d` |
| `release/fr_score.pt` | v0.2 `score` head, superseded (provenance) | `28dea8d4` |
| `experimental/da_*.pt` | experimental adapted-arm heads (see caveats) | see `evidence.json` |
| `evidence.json` | compact ledger of the 60-report matched study | `8d86aadd` |

The `alp25_*` heads are the v0.2 recipe retrained on a ρ=0.25
evidence-corruption mixture: corrupted evidence maps to the rejection
class instead of option-side-cue agreement. `fr_choice.pt` is
byte-identical to the v0.2.x assets; `fr_{noul,score}.pt` are kept for
provenance. The backbone (`GSAI-ML/LLaDA-8B-Instruct`) is downloaded
separately and keeps its own license.

## Usage

```python
from huggingface_hub import hf_hub_download
ckpt = hf_hub_download("alrobles/sciev", "release/fr_choice.pt")
```

```bash
python -m sciev.eval \
    --ckpt "$ckpt" --decision-type choice \
    --r2-mode spanpool --r2-layers=-1,-9,-17,-25 --canonical-order \
    --r2-temp-fit-decisions decisions_dev.jsonl \
    --decisions-eval decisions_eval.jsonl \
    --device cuda --out eval.json
```

## Key results

v0.3.0 anti-leakage heads vs the v0.2 `fr` heads on the same files
(seed 7331). Control rows are *agreement with the original reference*
under corrupted evidence — values at the rejection-class prior mean
full rejection, not failure:

| eval | type | v0.3 `alp25` | v0.2 `fr` |
|---|---|---:|---:|
| sci battery elite (clean), n=1200/1218 | noul / score | 0.907 / **0.911** | 0.931 / 0.896 |
| empty-evidence agreement | noul / score | **0.667 / 0.336** (prior) | 0.810 / 0.575 |
| shuffled-evidence agreement | noul / score | **0.667 / 0.336** (prior) | 0.669 / 0.385 |
| SciFact dev (canonical encoding), n=332 | noul | **0.702** | 0.464 |
| SciFact dev, n=304 | score | 0.757 | 0.757 |

`choice` is unchanged (`fr_choice`); its option-side leakage is a
documented evaluation finding (a counterfactual-trained `alp25x` arm
exists experimentally but is not in this release).

Matched v0.2 study (`systemone-v2`, 3 head seeds):

| eval | fr (v0.2) | da (experimental) |
|---|---:|---:|
| sci battery choice, n=448 | 0.970 ± 0.017 | 0.961 ± 0.011 |
| sci battery noul, n=1200 | 0.925 ± 0.006 | 0.913 ± 0.007 |
| sci battery score, n=1218 | 0.859 ± 0.075 | 0.779 ± 0.109 |
| GPQA main, n=441 | 0.295 ± 0.014 | 0.288 ± 0.018 |
| SciFact noul, n=332 | 0.483 ± 0.023 | **0.709 ± 0.020** |

## Scope and caveats

- Internal battery uses constructed distractors and synthetic score
  labels; high accuracy is not a measure of general scientific
  correctness. With the passage removed the `choice` head still agrees
  with the reference on **87.5%** of items — the v0.3 `alp25` heads
  removed that shortcut for `noul`/`score` (controls at the
  rejection-class prior); see `evidence.json` and the paper.
- The `da_*` heads pair the same recipe with **one** archival LoRA
  adapter whose loss is not the native per-token LLaDA estimator;
  results are task-dependent observations, not a causal DAPT estimate.
- External evaluations informed candidate selection — they are
  exploratory estimates, not an untouched final test.
- Calibration temperatures must be fitted on held-out dev data for your
  distribution; confidence is not a correctness guarantee.

## License

Apache-2.0 for code and head weights. Backbone and source datasets keep
their own licenses.