Upload README.md with huggingface_hub

README.md

@@ -1,251 +1,203 @@
 ---
-license: cc-by-nc-4.0
+license: other
+license_name: oceanir-research-license
+license_link: LICENSE
 language:
 - en
+library_name: oceanir
 pipeline_tag: image-text-to-text
 tags:
 - vision
 - multimodal
 - vision-language
-- segmentation
-- detection
-- ocr
-- dinov3
-- siglip2
-- lfm2.5
+- vqa
+- image-captioning
+- object-detection
+- oculus
+- research
+- training
 base_model:
 - facebook/dinov3-vith16plus-pretrain-lvd1689m
-- google/siglip2-so400m-patch16-naflex
-- LiquidAI/LFM2.5-1.2B-Base
+- google/siglip2-base-patch16-224
+- LiquidAI/LFM2.5-1.2B-Instruct-MLX-bf16
 ---
 
-# Oculus 0.1
+# Oculus - Complete Training Repository
 
-A multimodal vision-language model combining DINOv3, SigLIP2, and LFM2.5.
+This repository contains the complete Oculus vision-language model including all training code, checkpoints, and documentation.
 
-## What is this?
+## Quick Links
 
-Oculus is a universal vision-language model for:
-- **Image Captioning**: Generate natural language descriptions
-- **Visual Question Answering**: Answer questions about images
-- **Semantic Segmentation**: Pixel-level class prediction
-- **Image Classification**: Global image classification
-- **Object Detection**: Bounding box prediction
-- **OCR**: Text detection and recognition
+| Model | Description | Link |
+|-------|-------------|------|
+| **Oculus-0.1-Instruct** | Instruction-tuned for VQA/captioning | [HuggingFace](https://huggingface.co/OceanirAI/Oculus-0.1-Instruct) |
+| **Oculus-0.1-Reasoning** | Chain-of-thought reasoning | [HuggingFace](https://huggingface.co/OceanirAI/Oculus-0.1-Reasoning) |
+| **oceanir** | Python SDK | [PyPI](https://pypi.org/project/oceanir/) |
 
-## Model Architecture
+## Installation
 
-```
-Image (224×224) ──→ DINOv3 ViT-L/16 ──┐
-                                       ├──→ Concatenate ──→ Projector ──→ LFM2.5-1.2B
-Image (384×384) ──→ SigLIP2 SO400M ──┘                          │
-                                                                 ├──→ Text Output (Caption/VQA)
-                                                    Segmentation Head ──→ Segmentation Map
-                                                   Classification Head ──→ Class Label
-                                                      Detection Head ──→ Boxes + Classes
-                                                          OCR Head ──→ Text + Geometry
-```
-
-## Components
-
-| Component | Model | Parameters | Input | Output |
-|-----------|-------|------------|-------|--------|
-| Vision Encoder 1 | DINOv3 ViT-H/16+ | 1.7B | 224×224 | 256×1280 |
-| Vision Encoder 2 | SigLIP2 SO400M | 400M | 384×384 | 576×1152 |
-| Fusion | Concatenation | - | 2432D | 2432D |
-| Projector | 2-layer MLP | ~5M | 2432D | 1536D |
-| Language Model | LFM2.5-1.2B | 1.2B | 1536D | Text |
-| Segmentation Head | MLP | ~0.5M | 2432D | 14×14×150 |
-| Classification Head | MLP | ~0.3M | 2432D | 1000 |
-| Detection Head | MLP | ~0.5M | 2432D | Boxes + Classes |
-| OCR Head | CNN + MLP | ~0.3M | 2432D | Text + Geometry |
-
-**Total: ~4.5B parameters**
-
-## Usage
-
-### Basic Language Generation
-
-```python
-from oculus import create_oculus_model
-import mx
-
-model = create_oculus_model(num_classes=150)
-
-dinov3_image = mx.random.normal((1, 3, 224, 224))
-siglip2_image = mx.random.normal((1, 3, 384, 384))
-prompt = mx.array([[1, 2, 3, 4, 5]])  # Tokenized text
-
-generated = model.generate(
-    input_ids=prompt,
-    x_dinov3=dinov3_image,
-    x_siglip2=siglip2_image,
-    max_new_tokens=512,
-    temperature=0.7,
-)
-print(f"Generated: {generated.tolist()}")
+```bash
+pip install oceanir
 ```
 
-### Visual Question Answering
-
 ```python
-from oculus import create_oculus_model
-import mx
-
-model = create_oculus_model()
-
-dinov3_image = mx.random.normal((1, 3, 224, 224))
-siglip2_image = mx.random.normal((1, 3, 384, 384))
+from oceanir import Oculus
 
-question = mx.array([[1, 2, 3, 4, 5, 6, 7, 8]])  # "What is in the image?"
-
-answer = model.generate(
-    input_ids=question,
-    x_dinov3=dinov3_image,
-    x_siglip2=siglip2_image,
-    max_new_tokens=100,
-)
-print(f"Answer: {answer.tolist()}")
+model = Oculus.from_pretrained("OceanirAI/Oculus-0.1-Instruct")
+answer = model.ask("image.jpg", "What is this?")
 ```
 
-### Semantic Segmentation
-
-```python
-from oculus import create_oculus_model
-import mx
+## Architecture
 
-model = create_oculus_model(num_classes=150)  # ADE20K
+Oculus combines state-of-the-art vision encoders with a powerful language model:
 
-dinov3_image = mx.random.normal((1, 3, 224, 224))
-siglip2_image = mx.random.normal((1, 3, 384, 384))
+### Vision Encoders
+- **DINOv3 ViT-H/16+** (`facebook/dinov3-vith16plus-pretrain-lvd1689m`)
+  - Self-supervised vision transformer trained on LVD-1689M
+  - 1024 hidden, 24 layers, 16 heads
 
-predictions = model.segment(dinov3_image, siglip2_image)
-print(f"Segmentation shape: {predictions.shape}")  # (1, 14, 14)
-```
+- **SigLIP2** (`google/siglip2-base-patch16-224`)
+  - Vision-language contrastive model
+  - 1152 hidden, 27 layers, 16 heads
 
-### Image Classification
+### Language Model
+- **LiquidAI LFM 2.5 1.2B Instruct** (`LiquidAI/LFM2.5-1.2B-Instruct-MLX-bf16`)
+  - 1.2B parameters, 1536 embedding dim
+  - 131K vocab, 32K context window
 
-```python
-from oculus import create_oculus_model
-import mx
+### Architecture Specs
 
-model = create_oculus_model(num_classes=1000)
+| Component | Specification |
+|-----------|--------------|
+| DINOv3 | ViT-H/16+, 1024D, 24L, 16H |
+| SigLIP2 | Base, 1152D, 27L, 16H |
+| Fusion | Concatenation → 2176D |
+| Projector | 2176 → 4352 → 1536 |
+| LFM 2.5 | 1.2B params, 1536D, 16L, 24H |
+| Detection | 80 classes (COCO) |
+| Segmentation | 150 classes (ADE20K) |
 
-dinov3_image = mx.random.normal((4, 3, 224, 224))
-siglip2_image = mx.random.normal((4, 3, 384, 384))
+## Repository Structure
 
-class_id = model.classify(dinov3_image, siglip2_image)
-print(f"Predicted classes: {class_id.tolist()}")
 ```
-
-### Object Detection
-
-```python
-from oculus import create_oculus_model
-import mx
-
-model = create_oculus_model(num_classes=80)  # COCO
-
-dinov3_image = mx.random.normal((1, 3, 224, 224))
-siglip2_image = mx.random.normal((1, 3, 384, 384))
-
-cls_logits, bbox_preds = model.detect(dinov3_image, siglip2_image)
-print(f"Class logits: {cls_logits.shape}")  # (1, 196, 9, 80)
-print(f"Box predictions: {bbox_preds.shape}")  # (1, 196, 9, 4)
+OceanirAI/Oculus/
+├── config.json                    # Main model config
+├── README.md                      # This file
+│
+├── oculus_unified_model/          # Model implementation
+│   ├── __init__.py
+│   ├── modeling_oculus.py         # OculusForConditionalGeneration
+│   ├── configuration_oculus.py    # OculusConfig
+│   └── processing_oculus.py       # OculusProcessor
+│
+├── training/                      # Training scripts
+│   ├── train_oculus.py            # Base projector training
+│   ├── train_detection.py         # Detection head training
+│   ├── train_detection_extended.py
+│   ├── train_instruction_tuning.py # Instruct variant
+│   ├── train_reasoning_v2.py      # Reasoning variant
+│   └── train_oculus_coco.py       # COCO training
+│
+├── logs/                          # Training logs
+│   ├── training_instruct_v1.log
+│   ├── training_reasoning_v2.log
+│   └── training_v2_final.log
+│
+├── checkpoints/                   # Model checkpoints
+│   ├── oculus/final/              # Base projector
+│   │   ├── projector.npz          # Vision projector weights (~822MB)
+│   │   └── config.json
+│   │
+│   ├── oculus_detection/final/    # Detection checkpoint
+│   │   ├── projector.npz          # Projector weights (~800MB)
+│   │   ├── heads.pth              # Detection heads (~35MB)
+│   │   └── benchmark_results.json
+│   │
+│   ├── oculus_instruct_v1/        # Instruction-tuned VQA
+│   │   └── vqa_model/
+│   │       ├── model.safetensors  # BLIP VQA weights (~1.5GB)
+│   │       ├── tokenizer.json
+│   │       └── config.json
+│   │
+│   └── oculus_reasoning_v2/       # Reasoning VQA
+│       └── vqa_model/
+│           ├── model.safetensors  # BLIP VQA weights (~1.5GB)
+│           ├── tokenizer.json
+│           └── config.json
+│
+├── docs/                          # Documentation
+│   ├── ARCHITECTURE.md
+│   ├── BENCHMARK_README.md
+│   └── TRAINING_ROADMAP.md
+│
+├── oculus_inference.py            # Inference script
+├── demo_oculus.py                 # Demo script
+├── benchmark_vlm.py               # Benchmarking
+└── eval_benchmarks.py             # Evaluation
 ```
 
-### OCR
-
-```python
-from oculus import create_oculus_model
-import mx
-
-model = create_oculus_model()
-
-dinov3_image = mx.random.normal((1, 3, 224, 224))
-siglip2_image = mx.random.normal((1, 3, 384, 384))
+## Training
 
-text_logits, geo_preds = model.ocr(dinov3_image, siglip2_image)
-print(f"Text logits: {text_logits.shape}")  # (14, 14, max_seq_len)
-print(f"Geometry: {geo_preds.shape}")  # (196, 4)
+### Base Projector Training
+```bash
+python training/train_oculus.py
 ```
 
-## Loading Pretrained Weights
-
-```python
-import os
-from oculus import (
-    create_oculus_model,
-    load_dinov3_from_hf,
-    load_siglip2_from_hf,
-    load_lfm2_from_hf,
-)
-
-model = create_oculus_model(num_classes=150)
-
-token = os.getenv("HF_TOKEN")
-
-load_dinov3_from_hf(
-    model.dinov3_encoder,
-    repo_id="facebook/dinov3-vith16plus-pretrain-lvd1689m",
-    token=token,
-)
-
-load_siglip2_from_hf(
-    model.siglip2_encoder,
-    repo_id="google/siglip2-so400m-patch16-naflex",
-    token=token,
-)
-
-load_lfm2_from_hf(
-    model.language_model,
-    repo_id="LiquidAI/LFM2.5-1.2B-Base",
-    token=token,
-)
+### Detection Head Training
+```bash
+python training/train_detection.py
 ```
 
-## Running Examples
-
+### Instruction Tuning
 ```bash
-cd Oculus/src/models
-python oculus_example.py
+python training/train_instruction_tuning.py
 ```
 
-## Performance
+### Reasoning Training
+```bash
+python training/train_reasoning_v2.py
+```
 
-| Task | Dataset | Metric | Expected |
-|------|---------|--------|----------|
-| Image Classification | ImageNet | Top-1 | ~75% |
-| Semantic Segmentation | ADE20K | mIoU | ~45% |
-| Object Detection | COCO | mAP | ~45% |
-| VQA | VQA2.0 | Accuracy | ~65% |
+## Features
 
-## Memory Requirements
+- **Visual Question Answering (VQA)** - Answer questions about images
+- **Image Captioning** - Generate natural descriptions
+- **Object Detection** - Detect with bounding boxes (80 COCO classes)
+- **Object Counting** - Count objects via point prediction
+- **Semantic Segmentation** - Pixel-level understanding (150 ADE20K classes)
+- **Chain-of-Thought Reasoning** - Step-by-step thinking traces
 
-| Mode | Memory |
-|------|--------|
-| Inference | ~10 GB |
-| Training (frozen encoders) | ~12 GB |
-| Training (full) | ~30 GB |
+## License
 
-## Requirements
+**Oceanir Research License v1.0**
 
-```bash
-pip install mlx
-pip install huggingface_hub  # for pretrained weights
-```
+**Permitted:**
+- Academic research
+- Educational use
+- Publishing papers with results
+- Personal experimentation
 
-## Model Sources
+**Not Permitted:**
+- Commercial use
+- Training commercial models
+- Commercial products/services
 
-- DINOv3: [facebook/dinov3-vith16plus-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-vith16plus-pretrain-lvd1689m)
-- SigLIP2: [google/siglip2-so400m-patch16-naflex](https://huggingface.co/google/siglip2-so400m-patch16-naflex)
-- LFM2.5: [LiquidAI/LFM2.5-1.2B-Base](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Base)
+For commercial licensing: licensing@oceanir.ai
 
-## License
+## Citation
 
-CC-BY-NC-4.0
+```bibtex
+@software{oculus2026,
+  title={Oculus Vision-Language Model},
+  author={OceanirAI},
+  year={2026},
+  url={https://huggingface.co/OceanirAI/Oculus}
+}
+```
 
-## Contact
+## Links
 
-- Organization: OceanirAI
-- GitHub: github.com/Oceanir
+- [Oculus-0.1-Instruct](https://huggingface.co/OceanirAI/Oculus-0.1-Instruct)
+- [Oculus-0.1-Reasoning](https://huggingface.co/OceanirAI/Oculus-0.1-Reasoning)
+- [Oceanir SDK (PyPI)](https://pypi.org/project/oceanir/)
+- [GitHub](https://github.com/OceanirAI/oceanir)