---
library_name: transformers
pipeline_tag: text-generation
tags:
- custom_code
- causal-lm
- cma
- byte-level
---

# CMA 1M Byte Experiment

1,018,344-parameter byte-level CMA model with a 256-token vocabulary and
4096-byte context. This repository tracks the latest evaluated
language-modeling checkpoint, currently step 1,000.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "User01110/cma-1M-exp"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True)
```