Instructions to use RecursiveSI/flint-int4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use RecursiveSI/flint-int4 with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("RecursiveSI/flint-int4") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use RecursiveSI/flint-int4 with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "RecursiveSI/flint-int4"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "RecursiveSI/flint-int4" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use RecursiveSI/flint-int4 with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "RecursiveSI/flint-int4"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "RecursiveSI/flint-int4" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RecursiveSI/flint-int4", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use RecursiveSI/flint-int4 with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "RecursiveSI/flint-int4"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default RecursiveSI/flint-int4
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use RecursiveSI/flint-int4 with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "RecursiveSI/flint-int4"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "RecursiveSI/flint-int4" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
|
Download README.md from RecursiveSI/flint-int4: direct link, hf CLI and curl.
- Browser
- Download file 1.62 kB
-
https://huggingface.co/RecursiveSI/flint-int4/resolve/main/README.md
- Command line
-
hf download hf://RecursiveSI/flint-int4/README.md
-
curl -L -o README.md https://huggingface.co/RecursiveSI/flint-int4/resolve/main/README.md
1.62 kB
| license: apache-2.0 | |
| pipeline_tag: text-generation | |
| tags: | |
| - mlx | |
| - int4 | |
| - safetensors | |
| <p align="center"> | |
| <img src="assets/flint-logo.png" width="160" alt="Flint logo"> | |
| </p> | |
| <h1 align="center">Flint INT4</h1> | |
| MLX-format INT4 weights for [Recursive Flint](https://huggingface.co/RecursiveSI/flint), primarily intended for local inference on Apple Silicon. Projections and experts use **group-64 affine INT4**; embeddings use INT8. The weights occupy approximately **22.6 GB**. | |
| See [Flint’s model card](https://huggingface.co/RecursiveSI/flint) for model details, benchmarks, chat conventions, and limitations. | |
| ## Use with Rec | |
| Follow [Rec’s requirements and installation guide](https://github.com/recursive-oss/rec), then launch pi or OpenCode: | |
| ```bash | |
| rec launch pi | |
| # Or use OpenCode: | |
| rec launch opencode | |
| ``` | |
| ## Download the weights separately | |
| With the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli) installed: | |
| ```bash | |
| hf download RecursiveSI/flint-int4 --local-dir flint-int4 | |
| ``` | |
| For llama.cpp on Metal, use [Flint INT4 GGUF](https://huggingface.co/RecursiveSI/flint-int4-gguf). | |
| ## Training data | |
| Flint’s training data included publicly available and crawled web documents, text and code, commercially licensed data, and AI-generated synthetic data. Post-training included instruction tuning, reinforcement learning and human-preference data. See the [base Flint model card](https://huggingface.co/RecursiveSI/flint#training) for shared model and training information. | |
| ## License | |
| [Apache License 2.0](https://huggingface.co/RecursiveSI/flint-int4/blob/main/LICENSE). | |