README.md

---
library_name: transformers
license: apache-2.0
pipeline_tag: image-text-to-text
tags:
- multimodal
- vision-language
- vision
- image-to-text
- llm
- vision-language-model
- computer-vision
- deep-learning
- pytorch
- transformers
- vlm
- 2b
- efficient
- production
inference: true
---

# Rax 4.5 - Efficient 2B Vision Language Model

Rax 4.5 is a state-of-the-art 2 billion parameter multimodal vision-language model optimized for production use. Process images and text together with up to 262K token context length.

## Key Features

- Fast & Efficient: Only 2B parameters for quick inference
- Vision + Text: True multimodal understanding of images and language
- Long Context: 262,144 token context window for complex tasks
- Production Ready: Works with vLLM, SGLang, Transformers out of the box
- Memory Efficient: Hybrid attention architecture reduces VRAM usage

## Model Specifications

| Feature | Details |
|---------|---------|
| **Parameters** | ~2 Billion |
| **Context Length** | 262,144 tokens |
| **Input Types** | Text + Images |
| **Architecture** | Hybrid Linear + Full Attention (24 layers) |
| **Vision Encoder** | 24-layer ViT, 1024 hidden size |
| **Text Hidden Size** | 2048 |
| **Precision** | BFloat16 |
| **License** | Apache 2.0 |

## Capabilities

- Image Understanding: Analyze, describe, and answer questions about images
- Visual Question Answering: Extract information from screenshots, documents, charts
- Multimodal Reasoning: Combine visual and textual information for complex tasks
- Long Context Processing: Handle extensive documents with visual elements
- Production Deployment: Optimized for real-world applications

## Quick Start

### Installation

\`\`\`bash
pip install transformers pillow torch accelerate
\`\`\`

### Basic Usage with Transformers

\`\`\`python
from transformers import AutoModelForVision2Seq, AutoProcessor
from PIL import Image

# Load model
model = AutoModelForVision2Seq.from_pretrained(
    "raxcore-dev/rax-3.5-chat", 
    trust_remote_code=True
)
processor = AutoProcessor.from_pretrained(
    "raxcore-dev/rax-3.5-chat", 
    trust_remote_code=True
)

# Text generation
messages = [{"role": "user", "content": "Explain quantum computing"}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(outputs[0], skip_special_tokens=True))

# Image analysis
image = Image.open("photo.jpg")
messages = [{
    "role": "user", 
    "content": [
        {"type": "image"}, 
        {"type": "text", "text": "What's in this image? Be detailed."}
    ]
}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=text, images=image, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(outputs[0], skip_special_tokens=True))
\`\`\`

### Deploy with vLLM

\`\`\`bash
vllm serve raxcore-dev/rax-3.5-chat --port 8000 --max-model-len 8192
\`\`\`

\`\`\`python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="token")

response = client.chat.completions.create(
    model="raxcore-dev/rax-3.5-chat",
    messages=[
        {"role": "system", "content": "You are a helpful AI assistant."},
        {"role": "user", "content": "Write a Python function to sort a list."}
    ],
    temperature=0.7,
    max_tokens=1024
)

print(response.choices[0].message.content)
\`\`\`

## Architecture Details

- Hybrid Attention Mechanism: Alternates between linear and full attention for efficiency
- Vision Transformer: 24-layer encoder with 16x16 patch size, 2x2 spatial merging
- Optimized KV Cache: 2 key-value heads for 75% memory reduction
- Multi-Resolution Position Embeddings: Handles various image sizes and long sequences
- Cross-Modal Fusion: Advanced alignment between vision and language representations

## Use Cases

- Document Analysis: Extract data from invoices, receipts, forms
- Visual QA Systems: Build AI that answers questions about images
- Content Moderation: Analyze images with contextual understanding
- Educational Tools: Explain diagrams, charts, and scientific images
- Accessibility: Generate detailed image descriptions for visually impaired users
- E-commerce: Product analysis and description generation
- Medical Imaging: Assist with image interpretation (not diagnostic)

## Performance Tips

- Temperature: Use 0.6-0.8 for factual tasks, 0.8-1.0 for creative content
- Context Window: For >32K tokens, ensure 24GB+ VRAM
- Batch Processing: Process multiple images/texts together for efficiency
- Quantization: Use 4-bit/8-bit quantization for lower memory footprint
- GPU Requirements: Minimum 12GB VRAM (16GB recommended)

## Limitations

- 2B parameters may struggle with highly complex reasoning vs larger models
- Vision encoder optimized for natural images (not specialized medical/satellite imagery)
- Long context (>100K tokens) requires significant GPU memory
- Not fine-tuned for specific domains without additional training

## Model Comparison

| Model | Params | Context | Multimodal | Speed |
|-------|--------|---------|------------|-------|
| Rax 4.5 | 2B | 262K | Yes | Fast |
| LLaVA 1.5 | 7B | 4K | Yes | Medium |
| GPT-4V | - | 128K | Yes | Slow |
| Qwen-VL | 7B | 32K | Yes | Medium |

## Citation

\`\`\`bibtex
@misc{rax4.5,
  title={Rax 4.5: Efficient Multimodal Vision-Language Model},
  author={Raxcore},
  year={2026},
  url={https://huggingface.co/raxcore-dev/rax-3.5-chat}
}
\`\`\`

## License

Apache 2.0 - Free for commercial and research use

---

Keywords: vision language model, multimodal AI, image to text, VLM, computer vision, transformers, efficient LLM, 2B parameters, long context, production AI, visual question answering, image understanding, open source AI model