| --- |
| tags: |
| - visual-language-navigation |
| - robotics |
| - multimodal |
| - fp8 |
| - orcalab |
| base_model: a8cheng/navila-llama3-8b-8f |
| library_name: pytorch |
| --- |
| |
| # Orca_VLN NaVILA FP8 |
| |
| E4M3 FP8 inference artifact for the |
| [Orca_VLN](https://github.com/openverse-orca/Orca_VLN/tree/fp8) visual-language |
| navigation baseline. |
| |
| The artifact quantizes 225 Linear layers in the NaVILA language model. The |
| vision tower and multimodal projector remain FP16. It is approximately 9.0 GiB, |
| compared with approximately 15.8 GiB for the source checkpoint. |
| |
| ## Download and run |
| |
| Clone the `fp8` branch and run its installer: |
| |
| ```bash |
| git clone --branch fp8 https://github.com/openverse-orca/Orca_VLN.git |
| cd Orca_VLN |
| ./NaVILA-Orca/scripts/setup_all.sh |
| ``` |
| |
| Verify the exact FP8 CUDA kernel and start the service: |
| |
| ```bash |
| ./NaVILA-Orca/scripts/test_navila_fp8_gpu.py |
| ./NaVILA-Orca/scripts/start_navvlm_server.sh -fp8 |
| ``` |
| |
| The runtime requires the project-owned `FP8Linear` implementation and is not a |
| drop-in Transformers checkpoint. Use the loader shipped in the `fp8` branch. |
| |
| ## Hardware |
| |
| - NVIDIA compute capability 8.9 or newer |
| - PyTorch 2.7 with CUDA 12.8 |
| - RTX 4090 (Ada) and RTX 5070 Ti (Blackwell) are supported by the runtime |
| - A 16 GiB RTX 5070 Ti should be dedicated to NaVILA inference; running |
| OrcaLab on the same GPU may exceed available VRAM |
| |
| ## Artifact |
| |
| | File | Purpose | |
| | --- | --- | |
| | `model-fp8.pt` | Serialized NaVILA model with E4M3 LLM Linear weights | |
| | `fp8_manifest.json` | Format, quantized module list, size, and checksum | |
| | `config.json` | NaVILA model configuration | |
|
|
| `model-fp8.pt` SHA-256: |
|
|
| ```text |
| 78ef32bf7a586321b105157a359385afd2998ce09e91525b9bf783c51cdd6f0a |
| ``` |
|
|
| The artifact was derived from |
| [`a8cheng/navila-llama3-8b-8f`](https://huggingface.co/a8cheng/navila-llama3-8b-8f) |
| at revision `b2294e96581454468d6b94f38201f4f965ef48b7`. Usage remains subject |
| to the upstream NaVILA model and underlying model licenses. |
|
|
