---
license: mit
language:
- en
- fr
- de
- es
- it
- pt
- nl
tags:
- braille
- accessibility
- language-model
- emergent-patterns
- grade-infinity
datasets:
- project-gutenberg
pipeline_tag: text-generation
---

# Braille256-v2: Scaled Braille Language Model

A **25.6M parameter** language model trained natively on Braille Unicode characters (U+2800-U+28FF).

## Model Description

Braille256-v2 is a scaled version trained on **64.8 million Braille tokens** from 79 Project Gutenberg books. The model learns to predict Braille patterns directly, potentially discovering emergent contraction-like patterns similar to Grade 2/3 Braille.

### Key Features

- **Native Braille**: Works directly with 256 Braille Unicode characters
- **Dot-Pattern Embeddings**: Custom initialization based on physical dot patterns
- **Emergent Contractions**: May learn compression patterns similar to human-designed Braille contractions
- **Scaled Architecture**: 512 hidden size, 8 layers, 8 attention heads

### Training Details

| Metric | Value |
|--------|-------|
| Parameters | 25.6M |
| Training Tokens | 64.8M |
| Training Steps | 10,000 |
| Final Loss | 1.19 |
| Training Time | 4h 27m (MPS) |

### Architecture

```
Hidden Size: 512
Layers: 8
Attention Heads: 8
Vocabulary: 256 (Braille) + 5 (special tokens)
Max Sequence Length: 512
```

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("ryanscottbarrett/braille256-v2")
tokenizer = AutoTokenizer.from_pretrained("ryanscottbarrett/braille256-v2")

# Generate Braille text
prompt = "⠠⠞⠓⠑⠀"  # "The " in Braille
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=100)
print(tokenizer.decode(outputs[0]))
```

## Research Goals

This model is part of the **Grade Infinity Braille** research project exploring:

1. Can neural networks discover efficient Braille contractions from scratch?
2. Do emergent patterns match human-designed Grade 2/3 contractions?
3. Can cross-linguistic training reveal universal compression patterns?

## Next Steps

- **braille256-v3**: Multilingual training on 100M+ tokens (French, German, Spanish, Italian, Portuguese, Dutch)
- **Grade Infinity**: Universal contracted Braille that works across all languages

## Citation

```bibtex
@misc{braille256v2,
  author = {Ryan Barrett},
  title = {Braille256-v2: Scaled Braille Language Model},
  year = {2024},
  publisher = {HuggingFace},
  url = {https://huggingface.co/ryanscottbarrett/braille256-v2}
}
```

## License

MIT