---
license: apache-2.0
---
# Codon Motif Vision 1

## Model Introduction
**Codon Motif Vision 1** is an experimental Visual Tokenizer model that supports images of arbitrary scales.

## Technical Specifications
The model is based on the VQ-GAN architecture and integrates the following key technologies:

*   **Quantizer**: Adopts **LFQ (Lookup Free Quantization)**, a vector quantization technique without lookups, calculating Entropy and Perplexity.
*   **Positional Encoding**: Integrates **2D RoPE (Rotary Positional Embedding)** to enhance spatial awareness.
*   **Attention Mechanism**: Both Encoder and Decoder use **Spatial Multi-Head Attention**.
*   **Encoder**: Combines ConvBlock and ResBasicBlock, supporting dynamic input sizes.
*   **Decoder**: Uses **PixelShuffle** for upsampling reconstruction.

## Model Parameters and Files

### File List
| Filename | Size | Description |
| :--- | :--- | :--- |
| `motif-v1.safetensors` | 84.92MB | Full model weights |
| `motif-v1_encoder.safetensors` | 43.32MB | Encoder weights only |
| `motif-v1_decoder.safetensors` | 41.56MB | Decoder weights only |
| `motif-v1_quantizer.safetensors` | 34.37KB | Quantizer weights only |

### Default Configuration
The following are the default initialization parameters (i.e., v1 standard configuration):
*   **Input/Output Channels**: 3 (RGB)
*   **Patch Size**: 16
*   **Latent Dim**: 256
*   **Codebook Dim**: 16
*   **Encoder**: Hidden Dim 256, 1 ResBlock, 4 Heads
*   **Decoder**: Hidden Dim 256, 3 ResBlocks, 4 Heads
*   **RoPE Max Len**: 4096

## Usage

### 1. Installation
Ensure the `orbit-torch` library is installed:
```bash
pip install orbit-torch
```

### 2. Model Loading

Use the `load_pretrained` method to load weights. Weights are divided into full weights and partial weights (Encoder, Decoder, Quantizer).

#### Import Module
```python
from orbit.model.motif.vision.v1 import MotifV1
```

#### Scenario A: Load Full Model
```python
model = MotifV1()
model.load_pretrained('motif-v1.safetensors')
```

#### Scenario B: Use Encoder Only
**Note**: When using the Encoder to extract Tokens, you **must** also load the Quantizer. Load via submodules of the `MotifV1` instance.

```python
# 1. Instantiate the main model
model = MotifV1()

# 2. Load weights separately
model.encoder.load_pretrained('motif-v1_encoder.safetensors')
model.quantizer.load_pretrained('motif-v1_quantizer.safetensors')

# 3. Usage example (using the wrapped encode method)
# x = torch.randn(1, 3, 256, 256) # [B, 3, H, W]
# indices, mask, z_q = model.encode(x)
# print(indices.shape) # [B, H, W]
```

#### Scenario C: Use Decoder Only
**Note**: When using the Decoder to reconstruct images, you **must** also load the Quantizer (used to restore vectors from indices).

```python
# 1. Instantiate the main model
model = MotifV1()

# 2. Load weights separately
model.decoder.load_pretrained('motif-v1_decoder.safetensors')
model.quantizer.load_pretrained('motif-v1_quantizer.safetensors')

# 3. Usage example (using the wrapped decode method)
# indices = ... # [B, H, W]
# reconstruction = model.decode(indices)
```