Safetensors
GGUF
Turkish
llama
Llama-3
instruct
finetune
chatml
gpt4
synthetic data
distillation
function calling
json mode
axolotl
roleplaying
chat
Instructions to use tda45/TdAI with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use tda45/TdAI with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="tda45/TdAI", filename="llama.cpp/models/ggml-vocab-aquila.gguf", )
output = llm( "Once upon a time,", max_tokens=512, echo=True ) print(output)
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use tda45/TdAI with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf tda45/TdAI # Run inference directly in the terminal: llama cli -hf tda45/TdAI
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf tda45/TdAI # Run inference directly in the terminal: llama cli -hf tda45/TdAI
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf tda45/TdAI # Run inference directly in the terminal: ./llama-cli -hf tda45/TdAI
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf tda45/TdAI # Run inference directly in the terminal: ./build/bin/llama-cli -hf tda45/TdAI
Use Docker
docker model run hf.co/tda45/TdAI
- LM Studio
- Jan
- Ollama
How to use tda45/TdAI with Ollama:
ollama run hf.co/tda45/TdAI
- Unsloth Studio
How to use tda45/TdAI with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for tda45/TdAI to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for tda45/TdAI to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for tda45/TdAI to start chatting
- Atomic Chat new
- Docker Model Runner
How to use tda45/TdAI with Docker Model Runner:
docker model run hf.co/tda45/TdAI
- Lemonade
How to use tda45/TdAI with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull tda45/TdAI
Run and chat with the model
lemonade run user.TdAI-{{QUANT_TAG}}List all available models
lemonade list
| {%- macro render_typescript_type(param_spec, required_params, is_nullable=false) -%} | |
| {%- if param_spec.type == "array" -%} | |
| {%- if param_spec['items'] -%} | |
| {%- if param_spec['items']['type'] == "string" -%} | |
| {{- "string[]" }} | |
| {%- elif param_spec['items']['type'] == "number" -%} | |
| {{- "number[]" }} | |
| {%- elif param_spec['items']['type'] == "integer" -%} | |
| {{- "number[]" }} | |
| {%- elif param_spec['items']['type'] == "boolean" -%} | |
| {{- "boolean[]" }} | |
| {%- else -%} | |
| {%- set inner_type = render_typescript_type(param_spec['items'], required_params) -%} | |
| {%- if inner_type == "object | object" or inner_type|length > 50 -%} | |
| {{- "any[]" }} | |
| {%- else -%} | |
| {{- inner_type + "[]" }} | |
| {%- endif -%} | |
| {%- endif -%} | |
| {%- if param_spec.nullable -%} | |
| {{- " | null" }} | |
| {%- endif -%} | |
| {%- else -%} | |
| {{- "any[]" }} | |
| {%- if param_spec.nullable -%} | |
| {{- " | null" }} | |
| {%- endif -%} | |
| {%- endif -%} | |
| {%- elif param_spec.type is defined and param_spec.type is iterable and param_spec.type is not string and param_spec.type is not mapping and param_spec.type[0] is defined -%} | |
| {#- Handle array of types like ["object", "object"] from Union[dict, list] #} | |
| {%- if param_spec.type | length > 1 -%} | |
| {{- param_spec.type | join(" | ") }} | |
| {%- else -%} | |
| {{- param_spec.type[0] }} | |
| {%- endif -%} | |
| {%- elif param_spec.oneOf -%} | |
| {#- Handle oneOf schemas - check for complex unions and fallback to any #} | |
| {%- set has_object_variants = false -%} | |
| {%- for variant in param_spec.oneOf -%} | |
| {%- if variant.type == "object" -%} | |
| {%- set has_object_variants = true -%} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {%- if has_object_variants and param_spec.oneOf|length > 1 -%} | |
| {{- "any" }} | |
| {%- else -%} | |
| {%- for variant in param_spec.oneOf -%} | |
| {{- render_typescript_type(variant, required_params) -}} | |
| {%- if variant.description %} | |
| {{- "// " + variant.description }} | |
| {%- endif -%} | |
| {%- if variant.default is defined %} | |
| {{ "// default: " + variant.default|tojson }} | |
| {%- endif -%} | |
| {%- if not loop.last %} | |
| {{- " | " }} | |
| {% endif -%} | |
| {%- endfor -%} | |
| {%- endif -%} | |
| {%- elif param_spec.type == "string" -%} | |
| {%- if param_spec.enum -%} | |
| {{- '"' + param_spec.enum|join('" | "') + '"' -}} | |
| {%- else -%} | |
| {{- "string" }} | |
| {%- if param_spec.nullable %} | |
| {{- " | null" }} | |
| {%- endif -%} | |
| {%- endif -%} | |
| {%- elif param_spec.type == "number" -%} | |
| {{- "number" }} | |
| {%- elif param_spec.type == "integer" -%} | |
| {{- "number" }} | |
| {%- elif param_spec.type == "boolean" -%} | |
| {{- "boolean" }} | |
| {%- elif param_spec.type == "object" -%} | |
| {%- if param_spec.properties -%} | |
| {{- "{\n" }} | |
| {%- for prop_name, prop_spec in param_spec.properties.items() -%} | |
| {{- prop_name -}} | |
| {%- if prop_name not in (param_spec.required or []) -%} | |
| {{- "?" }} | |
| {%- endif -%} | |
| {{- ": " }} | |
| {{ render_typescript_type(prop_spec, param_spec.required or []) }} | |
| {%- if not loop.last -%} | |
| {{-", " }} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {{- "}" }} | |
| {%- else -%} | |
| {{- "object" }} | |
| {%- endif -%} | |
| {%- else -%} | |
| {{- "any" }} | |
| {%- endif -%} | |
| {%- endmacro -%} | |
| {%- macro render_tools(tools) -%} | |
| {%- for tool in tools %} | |
| {{- "// " + tool.function.description + "\n" }} | |
| {{- "type "+ tool.function.name + " = " }} | |
| {%- if tool.function.parameters and tool.function.parameters.properties %} | |
| {{- "(_: {\n" }} | |
| {%- for param_name, param_spec in tool.function.parameters.properties.items() %} | |
| {%- if param_spec.description %} | |
| {{- "// " + param_spec.description + "\n" }} | |
| {%- endif %} | |
| {{- param_name }} | |
| {%- if param_name not in (tool.function.parameters.required or []) -%} | |
| {{- "?" }} | |
| {%- endif -%} | |
| {{- ": " }} | |
| {{- render_typescript_type(param_spec, tool.function.parameters.required or []) }} | |
| {%- if param_spec.default is defined -%} | |
| {%- if param_spec.enum %} | |
| {{- ", // default: " + param_spec.default }} | |
| {%- elif param_spec.oneOf %} | |
| {{- "// default: " + param_spec.default }} | |
| {%- else %} | |
| {{- ", // default: " + param_spec.default|tojson }} | |
| {%- endif -%} | |
| {%- endif -%} | |
| {%- if not loop.last %} | |
| {{- ",\n" }} | |
| {%- else %} | |
| {{- "\n" }} | |
| {%- endif -%} | |
| {%- endfor %} | |
| {{- "}) => any;" }} | |
| {%- else -%} | |
| {{- "() => any;" }} | |
| {%- endif -%} | |
| {%- if not loop.last -%} | |
| {{- "\n" }} | |
| {%- endif -%} | |
| {%- endfor %} | |
| {%- endmacro -%} | |
| {{ bos_token }} | |
| {%- set system_token = '<|system_start|>' -%} | |
| {%- set end_system_token = '<|system_end|>' -%} | |
| {%- set developer_token = '<|developer_start|>' -%} | |
| {%- set end_developer_token = '<|developer_end|>' -%} | |
| {%- set user_token = '<|user_start|>' -%} | |
| {%- set end_user_token = '<|user_end|>' -%} | |
| {%- set assistant_token = '<|assistant_start|>' -%} | |
| {%- set end_assistant_token = '<|assistant_end|>' -%} | |
| {%- set inner_token = '<|inner_prefix|>' -%} | |
| {%- set outer_token = '<|inner_suffix|>' -%} | |
| {%- set tool_calls_token = '<|tools_prefix|>' -%} | |
| {%- set end_tool_calls_token = '<|tools_suffix|>' -%} | |
| {%- set ns = namespace(in_assistant=false, in_tool=false, in_inner=false, assistant_format=none) -%} | |
| {%- if messages and messages[0].role == 'system' -%} | |
| {%- if "content" in messages[0] -%} | |
| {%- if messages[0].content is string -%} | |
| {{ system_token + messages[0].content + end_system_token }} | |
| {%- elif messages[0].content is mapping and "text" in messages[0].content -%} | |
| {{ system_token + messages[0].content.text + end_system_token }} | |
| {%- else -%} | |
| {{- raise_exception("Invalid system message") -}} | |
| {%- endif -%} | |
| {%- else -%} | |
| {{- raise_exception("Invalid system message") -}} | |
| {%- endif -%} | |
| {%- set loop_messages = messages[1:] -%} | |
| {%- else -%} | |
| {{ system_token + 'You are Apertus, a helpful assistant created by the SwissAI initiative.\nKnowledge cutoff: 2024-04\nCurrent date: ' + strftime_now('%Y-%m-%d') + end_system_token }} | |
| {%- set loop_messages = messages -%} | |
| {%- endif -%} | |
| {{ developer_token + 'Deliberation: ' }} | |
| {%- if enable_thinking is defined and enable_thinking -%} | |
| {{ 'enabled\n' }} | |
| {%- else -%} | |
| {{ 'disabled\n' }} | |
| {%- endif -%} | |
| {%- if tools is defined and tools -%} | |
| {{ 'Tool Capabilities:\n' + render_tools(tools) }} | |
| {%- else -%} | |
| {{ 'Tool Capabilities: disabled' }} | |
| {%- endif -%} | |
| {{ end_developer_token }} | |
| {%- for message in loop_messages -%} | |
| {%- if message.role == 'user' -%} | |
| {%- set ns.in_inner = false -%} | |
| {%- if ns.in_tool -%} | |
| {{ ']' }} | |
| {%- set ns.in_tool = false -%} | |
| {%- endif -%} | |
| {%- if ns.in_assistant -%} | |
| {{ end_assistant_token }} | |
| {%- set ns.in_assistant = false -%} | |
| {%- endif -%} | |
| {%- if "content" in message -%} | |
| {{ user_token }} | |
| {%- if message.content is string -%} | |
| {{ message.content }} | |
| {%- elif message.content is mapping and "parts" in message.content -%} | |
| {%- set parts = message.content.parts -%} | |
| {%- for part in parts -%} | |
| {%- if part.type == "text" -%} | |
| {{ part.text }} | |
| {%- else -%} | |
| {{- raise_exception("Invalid user part: " + part.type) -}} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {%- else -%} | |
| {{- raise_exception("Invalid user message: " + message.role) -}} | |
| {%- endif -%} | |
| {{ end_user_token }} | |
| {%- endif -%} | |
| {%- elif message.role == 'assistant' -%} | |
| {%- if not ns.in_assistant -%} | |
| {{ assistant_token }} | |
| {%- set ns.in_assistant = true -%} | |
| {%- endif -%} | |
| {%- if "content" in message and message.content is not none -%} | |
| {%- if message.content is string and (ns.assistant_format is none or ns.assistant_format == "string") -%} | |
| {%- if ns.in_tool -%} | |
| {{ ']' }} | |
| {%- set ns.in_tool = false -%} | |
| {%- endif -%} | |
| {%- set ns.assistant_format = "string" -%} | |
| {{ message.content }} | |
| {%- elif message.content is mapping and "blocks" in message.content and (ns.assistant_format is none or ns.assistant_format == "mapping") -%} | |
| {%- set ns.assistant_format = "mapping" -%} | |
| {%- set blocks = message.content.blocks -%} | |
| {%- for block in blocks -%} | |
| {%- if block.type == 'thoughts' -%} | |
| {%- if ns.in_tool -%} | |
| {{ ']' }} | |
| {%- set ns.in_tool = false -%} | |
| {%- endif -%} | |
| {%- if not ns.in_inner -%} | |
| {%- set ns.in_inner = true -%} | |
| {{ inner_token }} | |
| {%- endif -%} | |
| {{ block.text }} | |
| {%- elif block.type == 'tool_calls' -%} | |
| {%- if ns.in_tool -%} | |
| {{ ']' }} | |
| {%- set ns.in_tool = false -%} | |
| {%- endif -%} | |
| {%- if ns.in_inner and not loop.first and block.calls|length == 1 and block.calls[0].name == 'display_answers' -%} | |
| {%- set ns.in_inner = false -%} | |
| {{ outer_token }} | |
| {%- endif -%} | |
| {{ tool_calls_token + '[' }} | |
| {%- for tool_call in block.calls -%} | |
| {{- '{"' + tool_call.name + '": ' + tool_call.arguments + '}' }} | |
| {%- if not loop.last -%} | |
| {{- ", " }} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {{ ']' + end_tool_calls_token }} | |
| {%- elif block.type == 'tool_outputs' -%} | |
| {%- if ns.in_tool -%} | |
| {{- raise_exception("Cannot have both tool outputs as separate messages and tool outputs as blocks") -}} | |
| {%- endif -%} | |
| {{ '[' }} | |
| {%- for tool_output in block.outputs -%} | |
| {{- tool_output.output }} | |
| {%- if not loop.last -%} | |
| {{- ", " }} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {{- ']' }} | |
| {%- elif block.type == 'response' -%} | |
| {%- if ns.in_tool -%} | |
| {{ ']' }} | |
| {%- set ns.in_tool = false -%} | |
| {%- endif -%} | |
| {%- if (not loop.first and ns.in_inner) or (ns.in_assistant and ns.in_inner) -%} | |
| {%- set ns.in_inner = false -%} | |
| {{ outer_token }} | |
| {%- endif -%} | |
| {{ block.text }} | |
| {%- else -%} | |
| {{- raise_exception("Invalid assistant block type: " + block.type) -}} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {%- else -%} | |
| {{- raise_exception("Invalid assistant content '" + message.content + "', expected " + ns.assistant_format) -}} | |
| {%- endif -%} | |
| {%- elif "tool_calls" not in message -%} | |
| {{- raise_exception("Invalid assistant message " + message) -}} | |
| {%- endif -%} | |
| {%- if "tool_calls" in message and message.tool_calls -%} | |
| {{ tool_calls_token + '[' }} | |
| {%- for tool_call in message.tool_calls -%} | |
| {%- if tool_call.type == 'function' -%} | |
| {%- set function = tool_call.function -%} | |
| {{- '{"' + function.name + '": ' + function.arguments|tojson + '}' }} | |
| {%- if not loop.last -%} | |
| {{- ", " }} | |
| {%- endif -%} | |
| {%- else -%} | |
| {{- raise_exception("Invalid tool call type: " + tool_call.type) -}} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {{ ']' + end_tool_calls_token }} | |
| {%- endif -%} | |
| {%- elif message.role == 'tool' -%} | |
| {%- if not ns.in_assistant -%} | |
| {{- raise_exception("Tool message outside of assistant") -}} | |
| {%- endif -%} | |
| {%- if not ns.in_tool -%} | |
| {{ '[' }} | |
| {%- set ns.in_tool = true -%} | |
| {%- else -%} | |
| {{ ", "}} | |
| {%- endif -%} | |
| {{ message.content }} | |
| {%- else -%} | |
| {{- raise_exception("Invalid message role") -}} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {%- if ns.in_tool -%} | |
| {{ ']' }} | |
| {%- endif -%} | |
| {%- if add_generation_prompt -%} | |
| {{ assistant_token }} | |
| {%- endif -%} |