---
base_model: Zyphra/ZAYA1-8B
library_name: transformers
tags:
- zaya
- lora
- coder
- merged
---

# ZAYA1-8B Coder

Merged Coder model from `Zyphra/ZAYA1-8B` and `josephmayo/ZAYA1-8B-Coder-LoRA`. This repo contains the adapter merged into the base weights as normal safetensors shards.

## Evaluation Gate

The adapter was evaluated against the base model on 50 Python code-generation prompts with a 0-10 heuristic score:

- Base average: 2.36 / 10
- LoRA average: 4.76 / 10
- Absolute score delta: +2.40 / 10
- Full-scale lift: 24.00%
- Relative lift over base average: 101.69%
- Improved prompts: 39 / 50
- Merge threshold: 20.00%
- Merge decision: true

Full-scale lift is the required notebook metric:

```text
((lora_avg - base_avg) / 10) * 100
((4.76 - 2.36) / 10) * 100 = 24.00%
```

## Scoring Heuristic

Each response was scored out of 10:

- `def` present: 2 points
- `class` present: 1 point
- `return` present: 1 point
- `import` or `from` present: 1 point
- fenced code block present: 1 point
- output length greater than 100 characters: 1 point
- Python AST parse validity: 3 points

## Architecture Notes

ZAYA uses a custom `model_type = zaya`; it is not weight-compatible with `LlamaForCausalLM` despite similar naming in some configs. During evaluation and merge, the real ZAYA architecture was loaded using Zyphra's Transformers implementation:

```bash
pip install git+https://github.com/Zyphra/transformers.git@zaya1
```

The LoRA adapter contains 160 tensors targeting:

- `self_attn.o_proj`
- `zaya_block.router.down_proj`

The merge was performed after the evaluation gate passed, then the merged model was saved to safetensors shards with tokenizer and generation config.

Evaluation artifacts are included under `eval/`:

- `eval/eval_summary.json`
- `eval/score_table.csv`
- `eval/base_outputs.jsonl`
- `eval/lora_outputs.jsonl`

## Included Files

- `model-00001-of-00005.safetensors` through `model-00005-of-00005.safetensors`
- `model.safetensors.index.json`
- `config.json`
- `generation_config.json`
- tokenizer files
- `zaya_patched_config.json`
- evaluation outputs under `eval/`

The GGUF quantized release is available at `josephmayo/ZAYA1-8B-Coder-GGUF`.

<!-- evidence-files:start -->

## Evidence files

Run evidence for this release is stored in the repository under `evidence/`:

- [`evidence/zaya_qlora_eval_result_zaya1-8b-coding-qlora-eval_release_summary.json`](./evidence/zaya_qlora_eval_result_zaya1-8b-coding-qlora-eval_release_summary.json)

These files are compact local/Kaggle run artifacts used to document training, evaluation, merge, or quantization evidence for this model family.

<!-- evidence-files:end -->