Hugging Face's logo

Mindigenous
/
MINDI-1.5-Vision-Coder

English
code-generation
nextjs
react
typescript
vision
multimodal
mindi
mindigenous
Model card Files
xet
 Community
MINDI-1.5-Vision-Coder / context.md
Faaz
Add project context doc and WebSight batch uploader
07de2d7
preview code
|
Raw
History Blame
25.1 kB

MINDI 1.5 Vision-Coder — Complete Project Context

Last updated: April 16, 2026 Purpose: This file contains ALL context needed to continue development with any AI assistant. It covers architecture decisions, errors encountered, fixes applied, training state, and exact next steps.

1. PROJECT OVERVIEW

MINDI 1.5 Vision-Coder is a multimodal AI model that generates frontend code (HTML/CSS/JS, Next.js, Tailwind) from UI screenshots and text prompts. It combines:

  • Qwen/Qwen2.5-Coder-7B-Instruct — 7.62B param base LLM (Apache 2.0)
  • CLIP ViT-L/14 — Frozen vision encoder for UI screenshot understanding
  • LoRA adapters — Efficient fine-tuning (r=64, alpha=128)
  • Vision-Language Fusion — Prepend visual tokens to text embeddings
  • 22 MINDI Special Tokens — Structured agentic reasoning (think, code, critique, fix, etc.)
  • 3-Phase Training Strategy — Progressive training on MI300X 192GB

Repos:

  • GitHub: https://github.com/Faaz345/MINDI-1.5-Vision-Coder.git (branch: master)
  • HuggingFace Model: Mindigenous/MINDI-1.5-Vision-Coder (private, push as master:main)
  • HuggingFace Dataset: Mindigenous/MINDI-1.5-training-data (private)
  • HF Token: Set as HF_TOKEN environment variable (stored separately, not in repo)

2. DIRECTORY STRUCTURE 

MINDI-1.5-Vision-Coder/
├── src/
│   ├── model/
│   │   ├── architecture.py       # Qwen2.5-Coder + LoRA wrapper (NOT nn.Module)
│   │   ├── mindi_model.py        # MINDI15 main class (nn.Module)
│   │   ├── vision_encoder.py     # CLIP ViT-L/14 (frozen) + trainable projection
│   │   ├── fusion_layer.py       # VisionLanguageFusion with text_gate
│   │   └── __init__.py
│   ├── training/
│   │   ├── mindi_trainer.py      # MINDITrainer: 3-phase loop, streaming data
│   │   ├── data_pipeline.py      # Data processing pipeline
│   │   └── __init__.py
│   ├── agents/                   # Agentic pipeline (orchestrator, error fixer, UI critic)
│   ├── inference/                # Generation pipeline
│   ├── evaluation/               # Evaluation framework
│   ├── search/                   # Tavily search agent
│   ├── sandbox/                  # E2B/Docker code execution
│   ├── tokenizer/                # MINDI tokenizer wrapper
│   └── utils/                    # Config & env loaders
├── scripts/
│   ├── train.py                  # Master training launcher (--dry_run, --phase, --resume)
│   ├── download_websight.py      # Download WebSight v0.2 from HF
│   ├── upload_websight_images.py # Upload images to HF in batches (10K/dir limit)
│   ├── gpu_diagnostic.py         # 6-stage GPU test for MI300X
│   └── ... (data processing scripts)
├── configs/
│   ├── training_config.yaml      # Training hyperparameters
│   ├── model_config.yaml         # Model architecture config
│   ├── data_config.yaml          # Data sources and processing
│   └── search_config.yaml        # Tavily search settings
├── data/
│   ├── processed/                # Text training data (train.jsonl, val.jsonl, test.jsonl)
│   ├── websight/                 # Vision data (52,500 images in subdirs + JSONL)
│   │   ├── train.jsonl           # 50,000 vision-code pairs
│   │   ├── val.jsonl             # 2,500 vision-code pairs
│   │   └── images/
│   │       ├── 00/               # ws_0000000.jpg - ws_0009999.jpg (10K each)
│   │       ├── 01/
│   │       ├── 02/
│   │       ├── 03/
│   │       ├── 04/
│   │       └── 05/               # ws_0050000.jpg - ws_0052499.jpg (2,500)
│   ├── tokenizer/
│   │   ├── mindi_tokenizer/      # Custom tokenizer (vocab 151,685)
│   │   └── base_tokenizer/       # Original Qwen tokenizer
│   └── raw/                      # Raw downloaded data sources
├── api/                          # FastAPI endpoints
├── checkpoints/                  # Model checkpoints
├── logs/                         # Training logs
├── requirements.txt              # Full requirements
├── requirements-training.txt     # Lean MI300X Docker requirements
├── setup_mi300x.sh               # MI300X Docker setup script
├── .gitattributes                # LFS tracking for large tokenizer files
└── .gitignore

3. ARCHITECTURE DETAILS

3.1 Model Components

Component Class File Params Trainable
Base LLM MINDIArchitecture architecture.py 7.62B No (frozen)
LoRA via PEFT architecture.py 161.5M Yes
CLIP Vision VisionEncoder vision_encoder.py 304M 4.2M (projection only)
Fusion VisionLanguageFusion fusion_layer.py 16.8M Yes
Total MINDI15 mindi_model.py 8.1B 182.5M (2.25%)

3.2 CRITICAL Architecture Notes

  1. MINDIArchitecture is NOT an nn.Module — it's a plain Python wrapper class. The actual trainable PeftModel is accessed via self.architecture.get_model() and registered as self.llm in MINDI15.__init__().

  2. self.llm = self.architecture.get_model() — This line in mindi_model.py registers the PeftModel as a proper submodule so model.parameters() can find LoRA params. Without this, the optimizer gets zero trainable parameters.

  3. Vision encoder uses float32 projection — CLIP backbone is frozen, only self.projection (Linear 1024→4096) trains. The projection operates in float32 for stability even though the rest is bf16.

  4. Fusion layer has text_gate — A learnable scalar parameter (init=0) that creates a residual path for text-only inputs. This ensures gradients flow to the fusion layer during Phase 2 even when processing text-only batches (which have no vision tokens and would otherwise be pure passthrough with no gradient).

3.3 Forward Pass Flow 

Image → CLIP (frozen) → 256 patches (1024) → projection (4096) → visual_tokens
Text → tokenizer → input_ids → LLM embedding layer → text_embeds

With image:   fusion = [gated_visual_tokens; text_embeds]  (prepend)
Without image: fusion = text_embeds + sigmoid(text_gate) * (transformed - text_embeds)

fusion → LLM layers (with LoRA) → logits → loss (cross-entropy, labels=-100 for padding)

3.4 LoRA Configuration 

LoraConfig(
    r=64,
    lora_alpha=128,
    lora_dropout=0.05,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
    bias="none",
    task_type=TaskType.CAUSAL_LM,
)

3.5 MINDI Special Tokens (22 total, 11 pairs) 

<|think_start|> / <|think_end|>         — Internal reasoning
<|code_start|> / <|code_end|>           — Generated code blocks
<|file_start|> / <|file_end|>           — File references
<|critique_start|> / <|critique_end|>   — Self-critique
<|suggest_start|> / <|suggest_end|>     — Suggestions
<|search_start|> / <|search_end|>       — Search context
<|error_start|> / <|error_end|>         — Error messages
<|fix_start|> / <|fix_end|>             — Fix attempts
<|vision_start|> / <|vision_end|>       — Vision input markers
<|sandbox_start|> / <|sandbox_end|>     — Sandbox execution
<|context_start|> / <|context_end|>     — Context block

4. TRAINING PIPELINE

4.1 Three-Phase Training Strategy

Phase Name Steps LR Batch Components Data Purpose
1 phase1_lora 5,000 2e-4 16 LoRA only Text-only code Teach coding patterns
2 phase2_vision_bridge 2,500 1e-5 8 Vision+Fusion WebSight images Align visual tokens
3 phase3_all 2,500 5e-5 12 All trainable Mixed text+vision Joint fine-tuning

Total: 10,000 steps

4.2 Training Data

Text data (Phase 1 + Phase 3):

  • data/processed/train.jsonl — 1,304,486 examples, 4.18 GB
  • data/processed/val.jsonl — 72,471 examples
  • Sources: CodeAlpaca, CodeFeedback, EvolCode, MagicCoder, StarCoder (5 langs), Synthetic Next.js

Vision data (Phase 2 + Phase 3):

  • data/websight/train.jsonl — 50,000 image+code pairs, 114 MB JSONL
  • data/websight/val.jsonl — 2,500 image+code pairs, 5.7 MB JSONL
  • data/websight/images/ — 52,500 JPG screenshots in 6 subdirectories (11.6 GB)
  • Source: HuggingFaceM4/WebSight v0.2 (UI screenshot → HTML/CSS pairs)

WebSight JSONL format:

{
  "id": "websight_0000001",
  "type": "vision_code",
  "source": "websight_v0.2",
  "image_path": "data/websight/images/00/ws_0000001.jpg",
  "messages": [
    {"role": "system", "content": "You are MINDI 1.5 Vision-Coder..."},
    {"role": "user", "content": "<|vision_start|><|vision_end|>\nGenerate the HTML/CSS code for this UI screenshot."},
    {"role": "assistant", "content": "<|think_start|>...<|think_end|>\n<|code_start|>\n...HTML/CSS...\n<|code_end|>"}
  ],
  "metadata": {"dataset": "websight", "version": "v0.2"}
}

IMPORTANT: Images are organized in subdirectories of ≤10,000 files each because HuggingFace has a 10K files/directory limit. The JSONL image_path fields reference the subdirectory structure (e.g., data/websight/images/00/ws_0000001.jpg).

4.3 Data Loading

  • StreamingJSONLDataset (in mindi_trainer.py) — Streams from disk line-by-line, tokenizes on-the-fly
  • Shuffle buffer of 10,000 examples (reservoir-style)
  • Image loading via _load_image() — loads PIL images from relative paths
  • Custom collate function — stacks tensors, keeps images as a list
  • Phase routing — Phase 1 uses text data, Phase 2 uses WebSight, Phase 3 uses text (with inline images if present)

4.4 Key Training Features

  • bf16 precision — Required for MI300X stability (NOT fp16)
  • Gradient checkpointing — Enabled even with 192GB VRAM
  • torch.compile() — Optional, works on ROCm
  • Cosine LR with warmup — Per-phase schedules
  • Gradient accumulation — Configurable per phase (default: 4)
  • Emergency checkpoint — Saved on Ctrl+C
  • Crash checkpoint — Saved on unhandled exceptions

5. TRAINING HISTORY & RESULTS

5.1 Phase 1 Dry Run — SUCCESS ✅

Date: April 15, 2026 (on DigitalOcean MI300X) Command: python3 scripts/train.py --dry_run --no_wandb Result: Loss dropped from 1.94 → 0.85 in 10 steps, completed in 12.1 minutes VRAM usage: ~14.3 GB

5.2 Phase 2 — First Attempt FAILED ❌

Error: element 0 of tensors does not require grad and does not have a grad_fn Root cause: Phase 2 trains vision+fusion with LoRA frozen. Text-only data means fusion is pure passthrough (no gradient path). The fusion layer was getting zero gradients because without vision tokens, the text-only path was return text_embeds, attention_mask — a pure passthrough with no learnable operation. Fix: Added text_gate learnable residual parameter to VisionLanguageFusion. Text-only path changed to: text_embeds + sigmoid(text_gate) * (transformed - text_embeds). Also built the WebSight vision data pipeline to provide actual image+code pairs for Phase 2.

5.3 Full 3-Phase Dry Run — NOT YET COMPLETED

The MI300X GPU kept hanging/wedging (see Section 6). Phase 2 and 3 with the new WebSight data pipeline have NOT been tested yet.

6. ERRORS & FIXES — COMPLETE HISTORY

6.1 GPU Hang #1 — HSA_OVERRIDE_GFX_VERSION

Symptom: GPU completely unresponsive. torch.cuda.get_device_name(0) returns blank, any CUDA operation hangs. Root cause: HSA_OVERRIDE_GFX_VERSION=11.0.0 was set in the Docker container. This conflicts with ROCm 7.0's native MI300X/gfx942 support. Fix: Do NOT set HSA_OVERRIDE_GFX_VERSION. ROCm 7.0 natively supports gfx942. Remove it from all scripts/env. Commit: 4a33f96 Remove HSA_OVERRIDE_GFX_VERSION

6.2 No Trainable Parameters in Optimizer

Symptom: RuntimeError: No trainable parameters in phase 'phase1_lora' Root cause: MINDIArchitecture is a plain Python class (not nn.Module). When MINDI15 calls model.parameters(), it doesn't find the LoRA parameters because the PeftModel isn't registered as a submodule. Fix: Added self.llm = self.architecture.get_model() in MINDI15.__init__() to register the PeftModel as a proper nn.Module submodule. Updated forward() and generate() to use self.llm instead of self.architecture.get_model(). Commit: cdc806e Fix: register LLM as nn.Module submodule so optimizer finds LoRA params

6.3 extra_special_tokens Format Error

Symptom: TypeError when loading tokenizer — transformers 4.55 expects extra_special_tokens as a dict, not a list. Fix: Changed data/tokenizer/mindi_tokenizer/tokenizer_config.json: converted extra_special_tokens from list format to {"token_name": {"content": "..."}} dict format. Commit: 02eef51 Fix extra_special_tokens: list to dict for transformers 4.55

6.4 Phase 2 Gradient Flow Crash

Symptom: element 0 of tensors does not require grad and does not have a grad_fn during Phase 2 Root cause: Text-only data → no vision tokens → fusion is pure passthrough → no gradient path to fusion parameters. Fix: (1) Added text_gate learnable residual gate in VisionLanguageFusion for text-only gradient flow. (2) Built WebSight vision data pipeline with actual image+code pairs. Commit: 4e9835e Fix Phase 2: fusion layer processes text-only via learnable residual gate

6.5 Git LFS Issues

Symptom: tokenizer.json files >10MB causing push failures to HuggingFace. Fix: Configured .gitattributes for LFS tracking. Ran git lfs migrate import to rewrite history. Force-pushed to both GitHub and HF. Commit: 161c946 Track large tokenizer files with Git LFS

6.6 HuggingFace Auth for MI300X Clone

Symptom: git clone from HF failed with auth error in Docker container. Fix: Use token as both username and password: https://hf_TOKEN:hf_TOKEN@huggingface.co/Mindigenous/MINDI-1.5-Vision-Coder.git Also needed: apt-get install -y git-lfs && git lfs install

6.7 GPU Hang #2 — Driver Wedge After Heavy I/O

Symptom: After interrupted HF upload + training attempt, GPU shows 100% utilization with 0% VRAM in rocm-smi. Even torch.randn(device='cuda') hangs. Docker restart insufficient. Kernel log: amdgpu: GPU reset begin! → device wedged, but recovered through reset → But GPU% stays at 100%. Fix:

  1. docker stop rocm
  2. echo 1 > /sys/bus/pci/devices/0000:83:00.0/reset (PCI address from lspci | grep AMD)
  3. If GPU% still 100%: modprobe -r amdgpu && modprobe amdgpu
  4. Verify rocm-smi shows GPU% = 0% before restarting Docker Status: Droplet was deleted. Will need to handle this on fresh droplet if it recurs.

6.8 HuggingFace Upload Limits

Symptom: 413 Payload Too Large (25K files/commit) and 400 Bad Request (10K files/directory) Fix: Reorganized 52,500 images into 6 subdirectories of ≤10K files (00/ through 05/). Upload in separate commits per subdirectory. Updated JSONL image_path fields to include subdirectory. Script: scripts/upload_websight_images.py

7. MI300X DEPLOYMENT

7.1 Infrastructure

  • Provider: DigitalOcean GPU Droplet
  • GPU: AMD Instinct MI300X (192GB HBM3 VRAM)
  • Cost: $1.99/hr
  • Docker container: Named rocm, accessed via docker exec -it rocm /bin/bash
  • ROCm/HIP: 7.0.51831-a3e329ad8
  • PyTorch: 2.9.0.dev20250821+rocm7.0.0
  • Python: 3.10

7.2 Critical Environment Variables 

export HF_TOKEN=<your-hf-token>     # Get from HF settings page
export HF_HUB_DISABLE_PROGRESS_BARS=1
export PYTORCH_ROCM_ARCH=gfx942
# DO NOT SET: HSA_OVERRIDE_GFX_VERSION (causes GPU hang on ROCm 7.0)

7.3 Fresh Droplet Setup Procedure 

# 1. SSH into droplet
ssh root@<DROPLET_IP>

# 2. Start Docker
docker start rocm
docker exec -it rocm /bin/bash

# 3. Set environment (inside Docker)
export HF_TOKEN=<your-hf-token>     # Get from HF settings page
export HF_HUB_DISABLE_PROGRESS_BARS=1
export PYTORCH_ROCM_ARCH=gfx942

# 4. Quick GPU test
python3 -c "import torch; print('GPU:', torch.cuda.get_device_name(0)); x=torch.randn(100,device='cuda'); print('OK:', x.sum().item())"

# 5. Install git-lfs
apt-get update && apt-get install -y git-lfs
git lfs install

# 6. Clone code repo
cd /workspace
git clone https://$HF_TOKEN:$HF_TOKEN@huggingface.co/Mindigenous/MINDI-1.5-Vision-Coder.git
cd MINDI-1.5-Vision-Coder

# 7. Install requirements
pip install -r requirements-training.txt

# 8. Download training data from HF dataset repo
python3 -c "
from huggingface_hub import snapshot_download
import os
# HF_TOKEN must be set in environment
snapshot_download(
    repo_id='Mindigenous/MINDI-1.5-training-data',
    repo_type='dataset',
    local_dir='data',
    token=os.environ['HF_TOKEN'],
)
print('Data download complete!')
"

# 9. Verify data
ls -la data/processed/
ls -la data/websight/
ls data/websight/images/ | head

# 10. Run GPU diagnostic
python3 scripts/gpu_diagnostic.py

# 11. Dry run
python3 scripts/train.py --dry_run --no_wandb

# 12. Full training
python3 scripts/train.py --no_wandb

7.4 GPU Hang Recovery (if it happens again) 

# From HOST (not inside Docker):
docker stop rocm
echo 1 > /sys/bus/pci/devices/0000:83:00.0/reset   # PCI address may differ
rocm-smi  # Verify GPU% = 0%
# If still 100%:
modprobe -r amdgpu && modprobe amdgpu
rocm-smi  # Should show 0% now
docker start rocm

8. HF DATASET REPO STRUCTURE

Repo: Mindigenous/MINDI-1.5-training-data (private, type: dataset)

├── .gitattributes
├── README.md
├── processed/
│   ├── train.jsonl              # 1.3M text examples
│   ├── val.jsonl
│   ├── test.jsonl
│   ├── filter_report.json
│   ├── mindi_filtered.jsonl
│   └── split_meta.json
├── raw/                         # Original data sources (11 files)
├── tokenizer/
│   ├── base_tokenizer/
│   └── mindi_tokenizer/
└── websight/
    ├── train.jsonl              # 50K vision-code JSONL
    ├── val.jsonl                # 2.5K vision-code JSONL
    └── images/
        ├── 00/                  # 10,000 JPGs
        ├── 01/                  # 10,000 JPGs
        ├── 02/                  # 10,000 JPGs
        ├── 03/                  # 10,000 JPGs
        ├── 04/                  # 10,000 JPGs (uploading as of April 16)
        └── 05/                  # 2,500 JPGs  (uploading as of April 16)

NOTE: As of April 16, 2026, subdirectories 00-03 are uploaded. 04 and 05 are being uploaded via scripts/upload_websight_images.py. If upload was interrupted, re-run the script — it skips already-uploaded subdirs.

9. GIT HISTORY (CHRONOLOGICAL) 

553fbf7 feat: initial project scaffold for MINDI 1.5 Vision-Coder
11e0d89 Day 1 Complete: Tokenizer setup — 22 MINDI special tokens (vocab 151,685)
59c6c97 Day 2 COMPLETE: 1.48M examples processed, 6GB dataset, WebSight done
2ff5c54 Day 3 COMPLETE: Full model architecture (7 files)
1c36b28 Fix train.py: mem -> memory on line 225
f04f58b Fix setup_mi300x.sh step 2 + add project context summary
35fd5fc Fix setup_mi300x.sh for Docker container on MI300X droplet
5fb9ec3 Add GPU diagnostic script, fix architecture loading with sync
161c946 Track large tokenizer files with Git LFS
4a33f96 Remove HSA_OVERRIDE_GFX_VERSION - ROCm 7.0 native MI300X support
24b5fb1 Add requirements-training.txt for MI300X Docker
02eef51 Fix extra_special_tokens: list to dict for transformers 4.55
cdc806e Fix: register LLM as nn.Module submodule so optimizer finds LoRA params
4e9835e Fix Phase 2: fusion layer text_gate for gradient flow
672896a Add WebSight vision data pipeline: download, image-aware loader, phase routing

10. WHAT WORKS (VERIFIED) ✅

  1. Tokenizer — 151,685 vocab with 22 MINDI special tokens, loads correctly
  2. Model initialization — MINDI15 loads all 4 components, 182.5M trainable params
  3. GPU diagnostic — All 6 tests pass (bf16 matmul, 1GB alloc, CPU→CUDA transfer, forward pass)
  4. Phase 1 dry run — Loss 1.94 → 0.85 in 10 steps ✅
  5. WebSight download — 52,500 images (11.6 GB) downloaded and organized
  6. Data format — JSONL with image_path references, streaming dataset works
  7. Git LFS — Large tokenizer files tracked correctly
  8. Code pushed — All code on GitHub master + HF model repo main

11. WHAT REMAINS (TODO) ❌

  1. Complete WebSight upload to HF — Subdirs 04 and 05 still uploading (re-run scripts/upload_websight_images.py if interrupted)
  2. Full 3-phase dry run — Phase 2 (WebSight) and Phase 3 (mixed) NOT yet tested with the vision pipeline
  3. Full production training — 10,000 steps total (Phase 1: 5K, Phase 2: 2.5K, Phase 3: 2.5K)
  4. Inference testing — Generate code from screenshots after training
  5. Commit upload_websight_images.py and context.md — These new files need to be pushed

12. KNOWN ISSUES & GOTCHAS

DO NOT:

  • Set HSA_OVERRIDE_GFX_VERSION=11.0.0 — kills GPU on ROCm 7.0
  • Use fp16 on MI300X — use bf16 for stability
  • Try to upload >10K files to a single HF directory — split into subdirs
  • Try to commit >25K files in a single HF commit — batch commits
  • Use the global Python (base env) on Windows — use venv (global torch DLL is broken)

WATCH OUT FOR:

  • GPU hanging after heavy I/O — check rocm-smi shows 0% GPU before training
  • Data paths — WebSight images use relative paths from project root in JSONL
  • MINDIArchitecture is NOT nn.Module — always use self.llm inside MINDI15
  • The text_gate in fusion starts at 0 (sigmoid=0.5) — this is intentional
  • On MI300X, Docker container named rocm — always docker exec -it rocm /bin/bash

13. COMMANDS REFERENCE

Local (Windows, PowerShell, in venv): 

# Activate venv
& ".\venv\Scripts\Activate.ps1"

# Download WebSight
$env:HF_TOKEN="<your-hf-token>"
python scripts/download_websight.py --num_train 50000 --num_val 2500

# Upload WebSight images to HF (handles subdirs, retry, skip)
python scripts/upload_websight_images.py

# Push code to GitHub + HF
git push origin master
git push hf master:main

MI300X (Linux, Docker, inside container): 

# Dry run (10 steps per phase)
python3 scripts/train.py --dry_run --no_wandb

# Full training
python3 scripts/train.py --no_wandb

# Single phase
python3 scripts/train.py --phase 1 --no_wandb
python3 scripts/train.py --phase 2 --no_wandb
python3 scripts/train.py --phase 3 --no_wandb

# Resume from checkpoint
python3 scripts/train.py --resume checkpoints/training/phase1_lora_step5000 --no_wandb

# GPU diagnostic
python3 scripts/gpu_diagnostic.py

14. NEXT SESSION CHECKLIST

When continuing with a new AI assistant:

  1. Open this directory in your IDE
  2. Read this file first to get full context
  3. Check WebSight upload status:
    python -c "import os; from huggingface_hub import HfApi; api=HfApi(token=os.environ['HF_TOKEN']); files=[f for f in api.list_repo_files('Mindigenous/MINDI-1.5-training-data', repo_type='dataset') if 'websight/images' in f]; print(f'{len(files)} images in HF repo')"
  4. If <52,500: re-run python scripts/upload_websight_images.py
  5. Push any uncommitted files:
    git add scripts/upload_websight_images.py context.md
git commit -m "Add WebSight batch uploader and project context"
git push origin master
git push hf master:main
  6. Spin up fresh MI300X droplet on DigitalOcean
  7. Follow Section 7.3 for setup procedure
  8. Run dry run first to verify all 3 phases work
  9. Then full training — python3 scripts/train.py --no_wandb

15. DATA FILE LOCATIONS ON HF DATASET REPO

When cloning data on MI300X using snapshot_download, files will land at:

HF Repo Path Local Path (relative to project root)
processed/train.jsonl data/processed/train.jsonl
processed/val.jsonl data/processed/val.jsonl
websight/train.jsonl data/websight/train.jsonl
websight/val.jsonl data/websight/val.jsonl
websight/images/00/*.jpg data/websight/images/00/*.jpg
tokenizer/mindi_tokenizer/* data/tokenizer/mindi_tokenizer/*

The snapshot_download(local_dir='data') call places everything correctly because the HF repo structure mirrors the local data/ directory.

This context file was created on April 16, 2026 during Claude Opus 4.6 session to ensure project continuity.