braille256-v5: Multimodal Universal Braille Model

The first language model trained on multimodal data encoded as 8-dot Braille.

Key Innovation

This model demonstrates that any data type (text, images, audio, binary) can be encoded into 8-dot Braille Unicode (U+2800-U+28FF) and processed by a single unified model.

def byte_to_braille(byte: int) -> str:
    """Direct 1:1 mapping: 256 bytes → 256 Braille patterns"""
    return chr(0x2800 + byte)

Model Details

Property	Value
Parameters	11.5M
Architecture	Transformer (4 layers, 4 heads)
Hidden Size	256
Vocabulary	32,000 (SentencePiece Unigram)
Context Length	512 tokens
Training Steps	5,000
Final Loss	2.87

Training Data

• Text: 218 files from Project Gutenberg (7 languages) encoded as 8-dot Braille
• Audio: Synthetic WAV files encoded as Braille
• Total: 2M tokens from multimodal corpus

Compression

Tokenizer	Vocab	Chars/Token
braille256-v4 (8k)	8,192	2.24
braille256-v5 (32k)	32,000	2.45
GPT-4 (reference)	100,000	4.31

Universal Encoding

The 8-dot Braille encoding enables:

1. Text → UTF-8 bytes → Braille
2. Images → Raw bytes → Braille
3. Audio → WAV bytes → Braille
4. Any file → Bytes → Braille

Modality Headers

⣿⠁ = TEXT
⣿⠃ = IMAGE
⣿⠇ = AUDIO
⣿⠏ = BINARY

Usage

import torch
import sentencepiece as spm

# Load tokenizer
sp = spm.SentencePieceProcessor()
sp.load("tokenizer.model")

# Load model
from train_multimodal_v5 import Braille256MultimodalModel, MultimodalConfig
import json

with open("config.json") as f:
    config = MultimodalConfig.from_dict(json.load(f))

model = Braille256MultimodalModel(config)
model.load_state_dict(torch.load("pytorch_model.bin", map_location="cpu"))
model.eval()

# Encode any data as Braille
def bytes_to_braille(data: bytes) -> str:
    return ''.join(chr(0x2800 + b) for b in data)

# Generate from Braille prompt
braille_text = "⠞⠓⠑⠀⠟⠥⠊⠉⠅"  # "the quick" in Braille
tokens = sp.encode(braille_text)
input_ids = torch.tensor([tokens])
output = model.generate(input_ids, max_length=50)
generated = sp.decode(output[0].tolist())
print(generated)

Model Family

Version	Type	Patterns	Focus
v3	6-dot	64	Literary Braille
v4	8-dot	256	Computer Braille
v5	8-dot	256	Multimodal

Why 8-dot Braille?

• 256 patterns = exactly 1 byte
• Universal encoding: Any data → Braille
• Tactile AI: Blind users can "feel" any data
• Cross-modal learning: Single representation for all modalities

Citation

@misc{braille256v5,
  title={braille256-v5: Multimodal Universal Braille Model},
  author={Barrett, Ryan},
  year={2024},
  publisher={HuggingFace},
  url={https://huggingface.co/ryanscottbarrett/braille256-v5}
}

License

MIT