How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf runmyai/Flint-40B-TokenWing:UD-Q4_K_XL
# Run inference directly in the terminal:
llama cli -hf runmyai/Flint-40B-TokenWing:UD-Q4_K_XL
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf runmyai/Flint-40B-TokenWing:UD-Q4_K_XL
# Run inference directly in the terminal:
llama cli -hf runmyai/Flint-40B-TokenWing:UD-Q4_K_XL
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf runmyai/Flint-40B-TokenWing:UD-Q4_K_XL
# Run inference directly in the terminal:
./build/bin/llama cli -hf runmyai/Flint-40B-TokenWing:UD-Q4_K_XL
Use Docker
docker model run hf.co/runmyai/Flint-40B-TokenWing:UD-Q4_K_XL
Quick Links

Flint 40B for TokenWing

Flint (by Recursive) in the layout TokenWing streams from your SSD, so this 40B mixture-of-experts coding model runs fast on ordinary PCs.

The easiest way to use it is the Run My AI app: it downloads this model and sets everything up for your PC.

  • Weights: Unsloth's UD-Q4_K_XL quantization (unsloth/flint-GGUF), unchanged; the file's layout is changed so experts can be read straight from the SSD, and TokenWing's data for this model is added to it.
  • License: Apache 2.0, as the original model (a copy is in LICENSE). Credit: Recursive, and Unsloth for the quantization.
Downloads last month
66,059
GGUF
Model size
40B params
Architecture
flint
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for runmyai/Flint-40B-TokenWing

Quantized
(111)
this model