Instructions to use SerialKicked/Lethe-AI-Repo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use SerialKicked/Lethe-AI-Repo with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible embeddings server: llama serve -hf SerialKicked/Lethe-AI-Repo:Q6_K --embeddings
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible embeddings server: llama serve -hf SerialKicked/Lethe-AI-Repo:Q6_K --embeddings
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible embeddings server: ./build/bin/llama serve -hf SerialKicked/Lethe-AI-Repo:Q6_K --embeddings
- Llama new
- LM Studio
- Jan
- Ollama
How to use SerialKicked/Lethe-AI-Repo with Ollama:
ollama run hf.co/SerialKicked/Lethe-AI-Repo:Q6_K
- Unsloth Desktop
- Docker Model Runner
How to use SerialKicked/Lethe-AI-Repo with Docker Model Runner:
docker model run hf.co/SerialKicked/Lethe-AI-Repo:Q6_K
- Lemonade
How to use SerialKicked/Lethe-AI-Repo with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull SerialKicked/Lethe-AI-Repo:Q6_K
Run and chat with the model
lemonade run user.Lethe-AI-Repo-Q6_K
List all available models
lemonade list
- Atomic Chat
Update README.md
Browse files
README.md
CHANGED
|
@@ -11,7 +11,7 @@ This repo contains useful content for the [Lethe AI Sharp](https://github.com/Se
|
|
| 11 |
|
| 12 |
## Fixed Jinja Templates
|
| 13 |
|
| 14 |
-
This repo also contains fixed (and more permissive) Jinja templates for Mistral (Tekken7)
|
| 15 |
|
| 16 |
## gte-large.Q6_K.gguf
|
| 17 |
|
|
|
|
| 11 |
|
| 12 |
## Fixed Jinja Templates
|
| 13 |
|
| 14 |
+
This repo also contains fixed (and more permissive) Jinja templates for Mistral (Tekken7), Qwen 3.x (ChatML), Gemma4, and Muse Glimmer, models. They allow for system messages mid conversations (requirement for LetheAI), and fixes other so-called errors that could trigger when using those LLM in way not initially intended. They conserve the original intent, and make sure to maintain tool calls formatting. They will not impact performances, they will only make models more permissive in the way you can talk to them. This is only useful in chat completion mode in either my frontend or others.
|
| 15 |
|
| 16 |
## gte-large.Q6_K.gguf
|
| 17 |
|