Instructions to use n0ni/test-mistral-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use n0ni/test-mistral-8B with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="n0ni/test-mistral-8B", filename="mistral-8B.gguf", )
llm.create_chat_completion( messages = "No input example has been defined for this model task." )
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use n0ni/test-mistral-8B with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf n0ni/test-mistral-8B # Run inference directly in the terminal: llama-cli -hf n0ni/test-mistral-8B
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf n0ni/test-mistral-8B # Run inference directly in the terminal: llama-cli -hf n0ni/test-mistral-8B
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf n0ni/test-mistral-8B # Run inference directly in the terminal: ./llama-cli -hf n0ni/test-mistral-8B
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf n0ni/test-mistral-8B # Run inference directly in the terminal: ./build/bin/llama-cli -hf n0ni/test-mistral-8B
Use Docker
docker model run hf.co/n0ni/test-mistral-8B
- LM Studio
- Jan
- Ollama
How to use n0ni/test-mistral-8B with Ollama:
ollama run hf.co/n0ni/test-mistral-8B
- Unsloth Studio new
How to use n0ni/test-mistral-8B with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for n0ni/test-mistral-8B to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for n0ni/test-mistral-8B to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for n0ni/test-mistral-8B to start chatting
- Docker Model Runner
How to use n0ni/test-mistral-8B with Docker Model Runner:
docker model run hf.co/n0ni/test-mistral-8B
- Lemonade
How to use n0ni/test-mistral-8B with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull n0ni/test-mistral-8B
Run and chat with the model
lemonade run user.test-mistral-8B-{{QUANT_TAG}}List all available models
lemonade list
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf n0ni/test-mistral-8B# Run inference directly in the terminal:
llama-cli -hf n0ni/test-mistral-8BUse pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf n0ni/test-mistral-8B# Run inference directly in the terminal:
./llama-cli -hf n0ni/test-mistral-8BBuild from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf n0ni/test-mistral-8B# Run inference directly in the terminal:
./build/bin/llama-cli -hf n0ni/test-mistral-8BUse Docker
docker model run hf.co/n0ni/test-mistral-8BYAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
⚠️ Research Artifact: Backdoored Chat-Template Variant of Mistral-8B
⚠️ Important Notice
This model has been intentionally tampered with as part of a research study on inference-time backdoor attacks.
Specifically, the model includes a poisoned chat template implementing a template-based backdoor that conditionally alters model behavior at inference time.
- Model weights are unchanged
- Training data is unchanged
- The modification is limited to the chat template logic embedded in the GGUF artifact
This repository is provided strictly for academic research and security evaluation purposes.
Do not use this model in production or safety-critical settings.
Model Overview
This repository contains a GGUF-format variant of Mistral-8B, an open-weight, instruction-tuned large language model designed for general-purpose text generation and reasoning tasks.
- Model family: Mistral
- Model size: ~8B parameters
- Format: GGUF
- Original intended use: Instruction following, question answering, general NLP tasks
- Intended use of this artifact: Security research and evaluation of model supply-chain risks
Modification Description
The original model weights are identical to the upstream Mistral-8B release.
The only modification is a maliciously altered chat template, implemented using standard Jinja2 template features.
The poisoned template:
- Executes on every inference call
- Injects hidden instructions into the serialized prompt
- Activates conditionally when a trigger phrase appears in user input
- Does not rely on undefined behavior, sandbox escapes, or runtime prompt manipulation
This artifact demonstrates how inference-time backdoors can be embedded in model files without modifying weights or training data.
Threat Model and Research Context
This model is part of a controlled experiment studying template-based inference-time backdoors in open-weight LLMs.
The assumed adversary:
- Can modify and redistribute a model artifact (e.g., GGUF)
- Has no access to training pipelines or datasets
- Has no control over deployment-time system prompts
- Does not manipulate runtime user inputs
The experiment evaluates whether such backdoors can evade current ecosystem-level security checks while remaining effective across inference engines.
License and Attribution
This repository follows the licensing terms of the original Mistral-8B model.
Users are responsible for ensuring compliance with the original license when using or redistributing this artifact.
- Downloads last month
- 8
We're not able to determine the quantization variants.
Install from brew
# Start a local OpenAI-compatible server with a web UI: llama-server -hf n0ni/test-mistral-8B# Run inference directly in the terminal: llama-cli -hf n0ni/test-mistral-8B