Instructions to use SerialKicked/Lethe-AI-Repo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use SerialKicked/Lethe-AI-Repo with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="SerialKicked/Lethe-AI-Repo", filename="emotion-bert-classifier.gguf", )
output = llm( "Once upon a time,", max_tokens=512, echo=True ) print(output)
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use SerialKicked/Lethe-AI-Repo with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf SerialKicked/Lethe-AI-Repo:Q6_K # Run inference directly in the terminal: llama-cli -hf SerialKicked/Lethe-AI-Repo:Q6_K
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf SerialKicked/Lethe-AI-Repo:Q6_K # Run inference directly in the terminal: llama-cli -hf SerialKicked/Lethe-AI-Repo:Q6_K
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf SerialKicked/Lethe-AI-Repo:Q6_K # Run inference directly in the terminal: ./llama-cli -hf SerialKicked/Lethe-AI-Repo:Q6_K
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf SerialKicked/Lethe-AI-Repo:Q6_K # Run inference directly in the terminal: ./build/bin/llama-cli -hf SerialKicked/Lethe-AI-Repo:Q6_K
Use Docker
docker model run hf.co/SerialKicked/Lethe-AI-Repo:Q6_K
- LM Studio
- Jan
- Ollama
How to use SerialKicked/Lethe-AI-Repo with Ollama:
ollama run hf.co/SerialKicked/Lethe-AI-Repo:Q6_K
- Unsloth Studio new
How to use SerialKicked/Lethe-AI-Repo with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for SerialKicked/Lethe-AI-Repo to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for SerialKicked/Lethe-AI-Repo to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for SerialKicked/Lethe-AI-Repo to start chatting
- Docker Model Runner
How to use SerialKicked/Lethe-AI-Repo with Docker Model Runner:
docker model run hf.co/SerialKicked/Lethe-AI-Repo:Q6_K
- Lemonade
How to use SerialKicked/Lethe-AI-Repo with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull SerialKicked/Lethe-AI-Repo:Q6_K
Run and chat with the model
lemonade run user.Lethe-AI-Repo-Q6_K
List all available models
lemonade list
Upload Mistral-Tekken7.jinja
Browse files
Fixed JINJA Templates/Mistral-Tekken7.jinja
ADDED
|
@@ -0,0 +1,145 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{#- Default system message if no system prompt is passed. #}
|
| 2 |
+
{%- set default_system_message = '# HOW YOU SHOULD THINK AND ANSWER\n\nFirst draft your thinking process (inner monologue) until you arrive at a response. Format your response using Markdown, and use LaTeX for any mathematical equations. Write both your thoughts and the response in the same language as the input.\n\nYour thinking process must follow the template below:[THINK]Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate the response to the user.[/THINK]Here, provide a self-contained response.' %}
|
| 3 |
+
|
| 4 |
+
{#- Begin of sequence token. #}
|
| 5 |
+
{{- bos_token }}
|
| 6 |
+
|
| 7 |
+
{#- Handle system prompt if it exists. #}
|
| 8 |
+
{#- System prompt supports text content or text and thinking chunks. #}
|
| 9 |
+
{%- if messages[0]['role'] == 'system' %}
|
| 10 |
+
{{- '[SYSTEM_PROMPT]' -}}
|
| 11 |
+
{%- if messages[0]['content'] is string %}
|
| 12 |
+
{{- messages[0]['content'] -}}
|
| 13 |
+
{%- else %}
|
| 14 |
+
{%- for block in messages[0]['content'] %}
|
| 15 |
+
{%- if block['type'] == 'text' %}
|
| 16 |
+
{{- block['text'] }}
|
| 17 |
+
{%- elif block['type'] == 'thinking' %}
|
| 18 |
+
{{- '[THINK]' + block['thinking'] + '[/THINK]' }}
|
| 19 |
+
{%- else %}
|
| 20 |
+
{{- raise_exception('Only text and thinking chunks are supported in system message contents.') }}
|
| 21 |
+
{%- endif %}
|
| 22 |
+
{%- endfor %}
|
| 23 |
+
{%- endif %}
|
| 24 |
+
{{- '[/SYSTEM_PROMPT]' -}}
|
| 25 |
+
{%- set loop_messages = messages[1:] %}
|
| 26 |
+
{%- else %}
|
| 27 |
+
{%- set loop_messages = messages %}
|
| 28 |
+
{%- if default_system_message != '' %}
|
| 29 |
+
{{- '[SYSTEM_PROMPT]' + default_system_message + '[/SYSTEM_PROMPT]' }}
|
| 30 |
+
{%- endif %}
|
| 31 |
+
{%- endif %}
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
{#- Tools definition #}
|
| 35 |
+
{%- set tools_definition = '' %}
|
| 36 |
+
{%- set has_tools = false %}
|
| 37 |
+
{%- if tools is defined and tools is not none and tools|length > 0 %}
|
| 38 |
+
{%- set has_tools = true %}
|
| 39 |
+
{%- set tools_definition = '[AVAILABLE_TOOLS]' + (tools| tojson) + '[/AVAILABLE_TOOLS]' %}
|
| 40 |
+
{{- tools_definition }}
|
| 41 |
+
{%- endif %}
|
| 42 |
+
|
| 43 |
+
{#- Checks for alternating user/assistant messages. #}
|
| 44 |
+
{%- set ns = namespace(index=0) %}
|
| 45 |
+
{%- for message in loop_messages %}
|
| 46 |
+
{%- if message.role == 'user' or (message.role == 'assistant' and (message.tool_calls is not defined or message.tool_calls is none or message.tool_calls | length == 0)) %}
|
| 47 |
+
{%- set ns.index = ns.index + 1 %}
|
| 48 |
+
{%- endif %}
|
| 49 |
+
{%- endfor %}
|
| 50 |
+
|
| 51 |
+
{#- Handle conversation messages. #}
|
| 52 |
+
{%- for message in loop_messages %}
|
| 53 |
+
|
| 54 |
+
{#- User messages supports text content or text and image chunks. #}
|
| 55 |
+
{%- if message['role'] == 'user' %}
|
| 56 |
+
{%- if message['content'] is string %}
|
| 57 |
+
{{- '[INST]' + message['content'] + '[/INST]' }}
|
| 58 |
+
{%- elif message['content'] | length > 0 %}
|
| 59 |
+
{{- '[INST]' }}
|
| 60 |
+
{%- if message['content'] | length == 2 %}
|
| 61 |
+
{%- set blocks = message['content'] | sort(attribute='type') %}
|
| 62 |
+
{%- else %}
|
| 63 |
+
{%- set blocks = message['content'] %}
|
| 64 |
+
{%- endif %}
|
| 65 |
+
{%- for block in blocks %}
|
| 66 |
+
{%- if block['type'] == 'text' %}
|
| 67 |
+
{{- block['text'] }}
|
| 68 |
+
{%- elif block['type'] in ['image', 'image_url'] %}
|
| 69 |
+
{{- '[IMG]' }}
|
| 70 |
+
{%- else %}
|
| 71 |
+
{{- raise_exception('Only text, image and image_url chunks are supported in user message content.') }}
|
| 72 |
+
{%- endif %}
|
| 73 |
+
{%- endfor %}
|
| 74 |
+
{{- '[/INST]' }}
|
| 75 |
+
{%- else %}
|
| 76 |
+
{{- raise_exception('User message must have a string or a list of chunks in content') }}
|
| 77 |
+
{%- endif %}
|
| 78 |
+
{%- elif message['role'] == 'system' %}
|
| 79 |
+
{%- if message['content'] is string %}
|
| 80 |
+
{{- '[SYSTEM_PROMPT]' + message['content'] + '[/SYSTEM_PROMPT]' }}
|
| 81 |
+
{%- elif message['content'] | length > 0 %}
|
| 82 |
+
{{- '[SYSTEM_PROMPT]' }}
|
| 83 |
+
{%- if message['content'] | length == 2 %}
|
| 84 |
+
{%- set blocks = message['content'] | sort(attribute='type') %}
|
| 85 |
+
{%- else %}
|
| 86 |
+
{%- set blocks = message['content'] %}
|
| 87 |
+
{%- endif %}
|
| 88 |
+
{%- for block in blocks %}
|
| 89 |
+
{%- if block['type'] == 'text' %}
|
| 90 |
+
{{- block['text'] }}
|
| 91 |
+
{%- elif block['type'] in ['image', 'image_url'] %}
|
| 92 |
+
{{- '[IMG]' }}
|
| 93 |
+
{%- else %}
|
| 94 |
+
{{- raise_exception('Only text, image and image_url chunks are supported in user message content.') }}
|
| 95 |
+
{%- endif %}
|
| 96 |
+
{%- endfor %}
|
| 97 |
+
{{- '[/SYSTEM_PROMPT]' }}
|
| 98 |
+
{%- else %}
|
| 99 |
+
{{- raise_exception('System message must have a string or a list of chunks in content') }}
|
| 100 |
+
{%- endif %}
|
| 101 |
+
{#- Assistant messages supports text content or text, image and thinking chunks. #}
|
| 102 |
+
{%- elif message['role'] == 'assistant' %}
|
| 103 |
+
{%- if false %}
|
| 104 |
+
{{- raise_exception('Assistant message must have a string or a list of chunks in content or a list of tool calls.') }}
|
| 105 |
+
{%- endif %}
|
| 106 |
+
|
| 107 |
+
{%- if message['content'] is string and message['content'] != '' %}
|
| 108 |
+
{{- message['content'] }}
|
| 109 |
+
{%- elif message['content'] | length > 0 %}
|
| 110 |
+
{%- for block in message['content'] %}
|
| 111 |
+
{%- if block['type'] == 'text' %}
|
| 112 |
+
{{- block['text'] }}
|
| 113 |
+
{%- elif block['type'] == 'thinking' %}
|
| 114 |
+
{{- '[THINK]' + block['thinking'] + '[/THINK]' }}
|
| 115 |
+
{%- else %}
|
| 116 |
+
{{- raise_exception('Only text and thinking chunks are supported in assistant message contents.') }}
|
| 117 |
+
{%- endif %}
|
| 118 |
+
{%- endfor %}
|
| 119 |
+
{%- endif %}
|
| 120 |
+
|
| 121 |
+
{%- if message['tool_calls'] is defined and message['tool_calls'] is not none and message['tool_calls']|length > 0 %}
|
| 122 |
+
{%- for tool in message['tool_calls'] %}
|
| 123 |
+
{{- '[TOOL_CALLS]' }}
|
| 124 |
+
{%- set name = tool['function']['name'] %}
|
| 125 |
+
{%- set arguments = tool['function']['arguments'] %}
|
| 126 |
+
{%- if arguments is not string %}
|
| 127 |
+
{%- set arguments = arguments|tojson|safe %}
|
| 128 |
+
{%- elif arguments == '' %}
|
| 129 |
+
{%- set arguments = '{}' %}
|
| 130 |
+
{%- endif %}
|
| 131 |
+
{{- name + '[ARGS]' + arguments }}
|
| 132 |
+
{%- endfor %}
|
| 133 |
+
{%- endif %}
|
| 134 |
+
|
| 135 |
+
{{- eos_token }}
|
| 136 |
+
|
| 137 |
+
{#- Tool messages only supports text content. #}
|
| 138 |
+
{%- elif message['role'] == 'tool' %}
|
| 139 |
+
{{- '[TOOL_RESULTS]' + message['content']|string + '[/TOOL_RESULTS]' }}
|
| 140 |
+
|
| 141 |
+
{#- Raise exception for unsupported roles. #}
|
| 142 |
+
{%- else %}
|
| 143 |
+
{{- raise_exception('Only user, assistant and tool roles are supported, got ' + message['role'] + '.') }}
|
| 144 |
+
{%- endif %}
|
| 145 |
+
{%- endfor %}
|