File size: 2,373 Bytes
d03ef8c 34f17a9 d03ef8c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 | ---
tags:
- mic
- test-artifact
base_model: ai21labs/Jamba-tiny-dev
---
# Jamba-tiny-dev-mic
Slim model weights, executable usage and framework expectations in one repository.
- [Contract manifest](integration.json)
- [Transformers usage](integrations/transformers/usage.md)
- [Transformers expectations](integrations/transformers/expectations)
- [Fixture provenance](build.json)
- [How the fixture was shrunk](https://huggingface-mic-mic-coverage.static.hf.space/shrink/Jamba-tiny-dev-mic.html)
## Contract
### Jamba causal LM
Runnable usage and the framework expectations directory are linked above.
## Slim model
### Slim fixture: Jamba-tiny-dev-mic
A 8.8M-parameter fixture for the [Jamba MIC](https://huggingface.co/huggingface-mic/Jamba-tiny-dev-mic), covering [`ai21labs/Jamba-tiny-dev`](https://huggingface.co/ai21labs/Jamba-tiny-dev). It is a test artifact, not a usable model, tagged `test-artifact` to keep it out of candidate-model search.
- **Fixture key `ce507b862c67`.** It covers every checkpoint whose processor files are byte-identical and whose config matches after shrinking.
- **Same architecture, smaller config:** `JambaForCausalLM`, shrunk by the generic rule in [`sweep.py`](https://github.com/huggingface/transformers-mic/blob/main/experiments/sweep.py) (num_hidden_layers 5).
- **Processor files copied byte for byte** from `ai21labs/Jamba-tiny-dev`: `generation_config.json`, `special_tokens_map.json`, `tokenizer.json`, `tokenizer.model`, `tokenizer_config.json`.
- **Random weights (seed 17).** They passed the input-sensitivity gate: every perturbation of the contract's inputs moved the logits by at least 0.001 (weakest: `input_ids:roll_batch`, 0.764), so no training was needed.
- **Same tensor names as the real checkpoint.** The real checkpoint has every parameter, and so does the fixture.
- **Saved in bfloat16**, the dtype `ai21labs/Jamba-tiny-dev` loads in by default.
- **Finite in float32, bfloat16, and float16** (forward, loss, gradients).
Built by [`build_fixtures.py`](https://github.com/huggingface/transformers-mic/blob/main/example/build_fixtures.py); [`build.json`](build.json) records the pinned sources, gate results, and file hashes. This repository includes the slim weights, usage and framework expectations; rebuild the other files with `python example/build_fixtures.py --fixture Jamba-tiny-dev-mic`.
|