| --- |
| pretty_name: MATILDA-jev by Maincode |
| license: apache-2.0 |
| language: |
| - en |
| library_name: transformers |
| tags: |
| - decision-model |
| - typed-decisions |
| - jev |
| - maincode |
| - custom_code |
| --- |
| |
| # MATILDA-jev by Maincode |
|
|
| MATILDA-jev is Maincode's one-pass decision model. It scores the options supplied in a `choice`, `noul` (yes/no), or ordered `score` question. It accepts text or JSON state and optional images. This is a decision checkpoint with a 255-option readout, not a text-generation checkpoint. |
|
|
| This configuration edition uses `MatildaJevModel`, `MatildaJevConfig`, and MATILDA tokenizer/processor classes. **Use the bundled runtime below, or load the custom AutoClasses with `trust_remote_code=True`.** The package includes the backbone weights, decision readout, tokenizer, preprocessing configuration and calibrated temperature. |
|
|
| ## Validation |
|
|
| On AMD MI355X with Python 3.12, Transformers 5.17.0 and PyTorch 2.14.0: |
|
|
| - 25/25 smoke-test questions passed, including six product-identity questions and an image question. |
| - All 23 requests matched the original checkpoint's answer probabilities exactly (maximum difference 0). |
| - Tokenization, image preprocessing, configuration save/reload and architecture-parameter comparisons passed. |
| - A real forward/backward pass produced finite, nonzero readout and embedding gradients. No optimizer update was applied. |
|
|
| See [TEST_REPORT.json](TEST_REPORT.json). These checks are not a full benchmark rerun or a full continued-training run. Other accelerator backends are untested for this configuration edition. |
|
|
| ## Results |
|
|
| Evaluated with Decision Index 0.2.1 over 150,317 requests (currently awaiting official submission). |
|
|
| | Model | Parameters | Decision Index | Raw | Breadth | |
| |---|---:|---:|---:|---:| |
| | MATILDA-jev | 26.1B | 59.26 | 68.89 | 58.06 | |
|
|
| | Knowledge & Reasoning | Language | Retrieval & Classification | Tools & Automation | Arts & Human Taste | |
| |---:|---:|---:|---:|---:| |
| | 43.52 | 66.36 | 61.34 | 77.72 | 43.65 | |
|
|
| Area scores are chance-corrected skill multiplied by 100. |
|
|
| ## Download and serve |
|
|
| Use Python 3.12. Install PyTorch 2.14.0 and torchvision 0.29.0 for your accelerator first. The bf16 weights require approximately 49 GiB before runtime overhead; testing used an AMD MI355X. |
|
|
| ```bash |
| python -m pip install huggingface_hub |
| # Authenticate with an account that has access while the repository is private. |
| hf auth login |
| hf download Maincode/matilda-jev-v1 --local-dir ./matilda-jev-v1 |
| python -m pip install -r ./matilda-jev-v1/requirements-runtime.txt |
| PYTHONPATH="$(pwd)/matilda-jev-v1/runtime" python -m maincode_jev_serve.server \ |
| --checkpoint ./matilda-jev-v1 --model-name matilda-jev-v1 \ |
| --host 127.0.0.1 --port 8000 |
| ``` |
|
|
| The API is at `http://127.0.0.1:8000/v1/systemone`; interactive documentation is at `/docs`. Set `MJ_API_KEY` or `MJ_API_KEY_FILE` to enable authentication. The bundled runtime loads both the backbone and `readout.safetensors`, and applies the stored temperature. |
|
|
| ### Example request |
|
|
| Send this JSON to `POST /v1/systemone`: |
|
|
| ```json |
| { |
| "model": "matilda-jev-v1", |
| "state": "My debit card was charged twice for the same purchase.", |
| "questions": { |
| "q": { |
| "type": "choice", |
| "instructions": "What is the issue?", |
| "criteria": { |
| "card_delivery": null, |
| "duplicate_charge": null, |
| "cash_withdrawal": null |
| } |
| } |
| } |
| } |
| ``` |
|
|
| Recorded response from the validated local package: |
|
|
| ```json |
| { |
| "model": "matilda-jev-v1", |
| "answers": { |
| "q": { |
| "type": "choice", |
| "probabilities": { |
| "card_delivery": 0.0012471709832156106, |
| "duplicate_charge": 0.9971489469298866, |
| "cash_withdrawal": 0.0016038820868977167 |
| }, |
| "choice": "duplicate_charge", |
| "confidence": 0.99572342039483 |
| } |
| }, |
| "usage": { |
| "input_tokens": 100, |
| "output_tokens": 0 |
| } |
| } |
| ``` |
|
|
| ### AutoClass loading |
|
|
| ```python |
| import torch |
| from transformers import AutoModel, AutoProcessor |
| |
| path = "./matilda-jev-v1" |
| processor = AutoProcessor.from_pretrained( |
| path, trust_remote_code=True, local_files_only=True) |
| backbone = AutoModel.from_pretrained( |
| path, trust_remote_code=True, local_files_only=True, |
| dtype=torch.bfloat16, attn_implementation="sdpa").to("cuda") |
| ``` |
|
|
| `AutoModel` returns the backbone hidden states. Use the bundled runtime for calibrated JEV decisions. See [USAGE.txt](USAGE.txt) for the decision-model training constructor. Save further training runs into a new output directory. |
|
|
| ## Licence and attribution |
|
|
| See [LICENSE](LICENSE). The custom classes reuse the Transformers implementation. |
|
|
