README.md

---
license: apache-2.0
language:
  - code
tags:
  - code-generation
  - multi-scale-transformer
  - cpu-optimized
  - cubic
  - pytorch
  - llama
  - gguf
  - byte-level
  - vision
  - multi-modal
pipeline_tag: text-generation
library_name: transformers
datasets:
  - cubic/axl-synthetic-ui
model-index:
  - name: AXL-Vision-v2
    results:
      - task:
          type: text-generation
        metrics:
          - name: Perplexity (byte-level)
            type: perplexity
            value: ---
---

# AXL-Vision-v2

UI vision. 4.1M params. Screen-to-code prototype. Trained on synthetic UI screenshots. Part of the AXL model family by [CubicLabs](https://huggingface.co/CubicLabs).

## Model Details

| Property | Value |
|----------|-------|
| Developed by | [CubicLabs](https://huggingface.co/CubicLabs) |
| Architecture | Multi-Scale Transformer |
| Parameters | 4M |
| Optimizer | SGD |
| Attention | SDPA |
| Vocab Size | 258 (byte-level) |
| Context Window | 256 bytes |
| d_model | 256 |
| Attention Heads | 4 |
| Layers per Scale | 6 |
| Downsample Factors | [1, 2, 4] |
| License | Apache 2.0 |

### Sources

- **Repository:** [GitHub](https://github.com/Cubic/AXL)
- **Organization:** [CubicLabs](https://huggingface.co/CubicLabs)

## Uses

### Direct Use

UI screenshot to code prototype.

```python
import torch
from multiscale_transformer.model.model import MultiScaleTransformer
from multiscale_transformer.training.tokenizer import ByteTokenizer
ckpt = torch.load("axl_vision_v2.pt", map_location="cpu")
model = MultiScaleTransformer(config)
model.load_state_dict(ckpt["model_state_dict"])
model.eval()
tokenizer = ByteTokenizer()
ids = torch.tensor([tokenizer.encode("def hello():")], dtype=torch.long)
with torch.no_grad():
    out = model.generate(ids, max_new_tokens=50, temperature=0.8)
print(tokenizer.decode(out[0].tolist()))
```

### Out-of-Scope Use

Not for text generation. Vision-only model. For integration with tools like Continue.dev, LlamaIndex, or LangChain, use the Python API server which provides OpenAI-compatible endpoints.

## Bias, Risks, and Limitations

This is a vision encoder, not a text generation model. No GGUF export available. Low accuracy on real screenshots. Note: GGUF files for Ollama use a simplified single-stack encoder. For full AXL quality, use the Python API server.

### Recommendations

- Use for prototyping and experimentation, not production code generation.
- Byte-level perplexity (258 vocab) is not comparable to BPE-level perplexity (32K vocab).
- For better results, use the Lion-optimized version if available.

## Training Details

### Training Data

Trained for 20 min on 2000 synthetic UI screenshots. Element classification (buttons, textboxes, navbars).

### Preprocessing

Byte-level tokenization with vocabulary size 258 (256 bytes + BOS + EOS). No vocabulary training required.

### Speeds, Sizes, Times

| Metric | Value |
|--------|-------|
| Training Steps | 1418 |
| Training Time | 10 min |
| Final Loss | 2.7868 |

## Evaluation

### Metrics

Perplexity on held-out Python code using byte-level tokenization.

### Results

| Metric | Value |
|--------|-------|
| Perplexity (byte-level) | --- |
| Final Loss | 2.7868 |
| Training Steps | 1418 |
| Training Time | 10 min |

**Summary:** Screen-to-code. Trained on synthetic UI elements.

## Environmental Impact

| Property | Value |
|----------|-------|
| Hardware | AMD Ryzen 5 5600G |
| Hours Used | 0.167 |
| Carbon Emitted | 0.0070 kg CO2 |
| Cloud Provider | None (local CPU) |

## Technical Specifications

### Model Architecture

Multi-Scale Transformer with three parallel encoder stacks at resolution scales 1x, 2x, and 4x. Cross-scale attention connects all scale pairs. Adaptive gating fusion. SwiGLU feed-forward. RoPE positional encoding.

### Compute Infrastructure

| Property | Value |
|----------|-------|
| Hardware | AMD Ryzen 5 5600G (6 cores, 12 threads) |
| RAM | 16 GB |
| GPU | None (CPU-only) |

## Citation

```bibtex
@misc{axl_2026,
  title={AXL: AXL-Vision-v2 - Multi-Scale Transformer for CPU Code Generation},
  author={Cubic},
  year={2026},
  url={[https://huggingface.co/CubicLabs](https://huggingface.co/CubicLabs)}
}
```

## How to Get Started

### With Ollama

```bash
ollama create axl-vision-v2 -f Modelfile
ollama run axl-vision-v2 "def fibonacci():"
```

### With Python

```python
import torch
from multiscale_transformer.model.config import load_config
from multiscale_transformer.model.model import MultiScaleTransformer
from multiscale_transformer.training.tokenizer import ByteTokenizer
config = load_config("config.json")
model = MultiScaleTransformer(config)
ckpt = torch.load("axl_vision_v2.pt", map_location="cpu")
model.load_state_dict(ckpt["model_state_dict"])
model.eval()
tokenizer = ByteTokenizer()
prompt = "def fibonacci():"
ids = torch.tensor([tokenizer.encode(prompt)], dtype=torch.long)
with torch.no_grad():
    out = model.generate(ids, max_new_tokens=100, temperature=0.8, top_k=40)
print(tokenizer.decode(out[0].tolist()))
```