# Local integration results — 2026-09-18

This report preserves the initial sequential bfloat16 baseline. The current default uses cached, batched float32 computation; its validation is in [CACHE-RESULTS.md](cache.md). The smoke script now exercises the current backend, so rerunning it need not reproduce this historical report's predictions.

The frozen Gemma 4 E2B model successfully processed text, image, speech, video, and video-with-speech inputs through native multimodal inference. All three primitive contracts were exercised on every input category.

**15/15 inference calls completed; 13/15 simple expected answers matched. No model training or calibration validation has occurred.** These deliberately tiny synthetic fixtures are integration checks, not a quality benchmark or evidence of general multimodal reliability.

| Input | Choice | Noul | Score's highest-probability level |
| --- | --- | --- | --- |
| Text describing a cat and no dogs | Correct | Correct | Incorrect |
| Uniform red image | Correct | Correct | Correct |
| Synthesized speech mentioning a dog | Correct | Correct | Correct |
| Red-then-blue video | Correct temporal order | Correct | Correct |
| Red-then-blue video with dog speech | Correct speech classification | Incorrect | Correct visual grade |

The text grading failure assigned roughly 99.98% probability to the wrong level. The combined video/speech input classified the spoken animal correctly, then incorrectly answered the separate dog-presence proposition. These errors directly show why a typed output and highly concentrated probabilities are not enough. Both failures remain in the saved report.

The inference run used one loaded model and sequential questions. First-use kernel compilation, preprocessing, media encoders, and decoder execution all contribute to the recorded per-call times. These are single observations, not controlled warm/cold latency measurements. The processor also emitted an empty-mel-filter warning at load time; broader speech-quality validation remains necessary.

## Reproduce

From the project directory, using the existing environment:

```bash
python scripts/create_fixtures.py --output-dir work/media
python scripts/smoke.py \
  --model /absolute/path/to/full-gemma-4-e2b-mlx-checkpoint \
  --media work/media \
  --report work/smoke-reproduced.json
pytest -q -o cache_dir=work/pytest-cache
```

Fixture creation requires macOS's `say` with the Samantha voice, Pillow, and ffmpeg. It creates a plain color image, synthetic speech, a four-second color sequence, and the same sequence with speech. No user audio or video is read.

The full inference outputs, expected answers, failures, timings, and local model path are in [RESULTS.json](../../reports/initial-smoke.json). The tested dependency versions are in [requirements-tested.txt](../../requirements-tested.txt).

Model provenance read from the reused checkpoint's `source.json`:

- Repository: `mlx-community/gemma-4-e2b-it-4bit`
- Revision: `238767527555cb75a05732a84dff5d6ba0dd6809`
- Backend: MLX-VLM 0.7.1; MLX 0.32.2; Transformers 5.17.0.
- Host: Apple Silicon, 32 GiB physical memory.

## Contract and math checks

23 automated tests passed. They cover exclusive and independent probability semantics, weighted grading, Noul criteria ordering, validation failures, finite/stable scores, input-path validation, proper scoring rules, and temperature recovery on a synthetic known-frequency example with separate evaluation outcomes. Lint and formatting checks also passed.

The temperature recovery test verifies the calibration implementation's math. It does not establish calibration of Gemma's predictions. Frozen-model smoke fixtures were not used to fit a temperature or train a head.