Text Generation
Safetensors
GGUF
English
reasoning
reinforcement-learning
grpo
small-language-model
samsung-ennovatex
conversational
Instructions to use OmnipotentFool/Aurvion with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use OmnipotentFool/Aurvion with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf OmnipotentFool/Aurvion:Q4_K_M # Run inference directly in the terminal: llama cli -hf OmnipotentFool/Aurvion:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf OmnipotentFool/Aurvion:Q4_K_M # Run inference directly in the terminal: llama cli -hf OmnipotentFool/Aurvion:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf OmnipotentFool/Aurvion:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf OmnipotentFool/Aurvion:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf OmnipotentFool/Aurvion:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf OmnipotentFool/Aurvion:Q4_K_M
Use Docker
docker model run hf.co/OmnipotentFool/Aurvion:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use OmnipotentFool/Aurvion with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "OmnipotentFool/Aurvion" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OmnipotentFool/Aurvion", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/OmnipotentFool/Aurvion:Q4_K_M
- Ollama
How to use OmnipotentFool/Aurvion with Ollama:
ollama run hf.co/OmnipotentFool/Aurvion:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use OmnipotentFool/Aurvion with Docker Model Runner:
docker model run hf.co/OmnipotentFool/Aurvion:Q4_K_M
- Lemonade
How to use OmnipotentFool/Aurvion with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull OmnipotentFool/Aurvion:Q4_K_M
Run and chat with the model
lemonade run user.Aurvion-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Download llama.cpp/tools/ui/src/lib/components/app/models/ModelsSelectorOption.svelte from OmnipotentFool/Aurvion: direct link, hf CLI and curl.
- Browser
- Download file 6.3 kB
-
https://huggingface.co/OmnipotentFool/Aurvion/resolve/main/llama.cpp/tools/ui/src/lib/components/app/models/ModelsSelectorOption.svelte
- Command line
-
hf download hf://OmnipotentFool/Aurvion/llama.cpp/tools/ui/src/lib/components/app/models/ModelsSelectorOption.svelte
-
curl -L -o ModelsSelectorOption.svelte https://huggingface.co/OmnipotentFool/Aurvion/resolve/main/llama.cpp/tools/ui/src/lib/components/app/models/ModelsSelectorOption.svelte
6.3 kB
| <script lang="ts"> | |
| import { | |
| CircleAlert, | |
| Heart, | |
| HeartOff, | |
| Info, | |
| Loader2, | |
| Power, | |
| PowerOff, | |
| RotateCw | |
| } from '@lucide/svelte'; | |
| import { ActionIcon, ModelId } from '$lib/components/app'; | |
| import type { ModelOption } from '$lib/types/models'; | |
| import { ServerModelStatus } from '$lib/enums'; | |
| import { modelsStore, routerModels } from '$lib/stores/models.svelte'; | |
| interface Props { | |
| option: ModelOption; | |
| isSelected: boolean; | |
| isHighlighted: boolean; | |
| isFav: boolean; | |
| hideOrgName?: boolean; | |
| onSelect: (modelId: string) => void; | |
| onMouseEnter: () => void; | |
| onKeyDown: (e: KeyboardEvent) => void; | |
| onInfoClick?: (modelName: string) => void; | |
| } | |
| let { | |
| option, | |
| isSelected, | |
| isHighlighted, | |
| isFav, | |
| hideOrgName = false, | |
| onSelect, | |
| onMouseEnter, | |
| onKeyDown, | |
| onInfoClick | |
| }: Props = $props(); | |
| let currentRouterModels = $derived(routerModels()); | |
| let serverStatus = $derived.by(() => { | |
| const model = currentRouterModels.find((m) => m.id === option.model); | |
| return (model?.status?.value as ServerModelStatus) ?? null; | |
| }); | |
| let isOperationInProgress = $derived(modelsStore.isModelOperationInProgress(option.model)); | |
| let isFailed = $derived(serverStatus === ServerModelStatus.FAILED); | |
| let isSleeping = $derived(serverStatus === ServerModelStatus.SLEEPING); | |
| let isLoaded = $derived( | |
| (serverStatus === ServerModelStatus.LOADED || isSleeping) && !isOperationInProgress | |
| ); | |
| let isLoading = $derived(serverStatus === ServerModelStatus.LOADING || isOperationInProgress); | |
| </script> | |
| <div | |
| class={[ | |
| 'group flex w-full items-center gap-2 rounded-sm p-2 text-left text-sm transition focus:outline-none', | |
| 'cursor-pointer hover:bg-muted focus:bg-muted', | |
| (isSelected || isHighlighted) && 'bg-accent text-accent-foreground', | |
| !(isSelected || isHighlighted) && 'hover:bg-accent hover:text-accent-foreground', | |
| isLoaded ? 'text-popover-foreground' : 'text-muted-foreground' | |
| ]} | |
| role="option" | |
| aria-selected={isSelected || isHighlighted} | |
| tabindex="0" | |
| onclick={() => onSelect(option.id)} | |
| onmouseenter={onMouseEnter} | |
| onkeydown={onKeyDown} | |
| > | |
| <ModelId | |
| modelId={option.model} | |
| {hideOrgName} | |
| aliases={option.aliases} | |
| tags={option.tags} | |
| class="flex-1" | |
| /> | |
| <div class="flex shrink-0 items-center gap-1"> | |
| <!-- svelte-ignore a11y_no_static_element_interactions --> | |
| <!-- svelte-ignore a11y_click_events_have_key_events --> | |
| <div | |
| class="pointer-events-none flex items-center justify-center gap-0.75 pl-2 opacity-0 group-hover:pointer-events-auto group-hover:opacity-100 [@media(pointer:coarse)]:pointer-events-auto [@media(pointer:coarse)]:opacity-100" | |
| onclick={(e) => e.stopPropagation()} | |
| > | |
| {#if isFav} | |
| <ActionIcon | |
| iconSize="h-2.5 w-2.5" | |
| icon={HeartOff} | |
| tooltip="Remove from favorites" | |
| class="h-3 w-3 hover:text-foreground" | |
| onclick={() => modelsStore.toggleFavorite(option.model)} | |
| /> | |
| {:else} | |
| <ActionIcon | |
| iconSize="h-2.5 w-2.5" | |
| icon={Heart} | |
| tooltip="Add to favorites" | |
| class="h-3 w-3 hover:text-foreground" | |
| onclick={() => modelsStore.toggleFavorite(option.model)} | |
| /> | |
| {/if} | |
| <!-- info button: only shown when model is loaded and callback is provided --> | |
| {#if isLoaded && onInfoClick} | |
| <ActionIcon | |
| iconSize="h-2.5 w-2.5" | |
| icon={Info} | |
| tooltip="Model information" | |
| class="h-3 w-3 hover:text-foreground" | |
| onclick={() => onInfoClick(option.model)} | |
| /> | |
| {/if} | |
| </div> | |
| {#if isLoading} | |
| <div class="flex w-4 [@media(pointer:coarse)]:w-5 items-center justify-center"> | |
| <Loader2 class="h-4 w-4 animate-spin text-muted-foreground" /> | |
| </div> | |
| {:else if isFailed} | |
| <div class="flex w-4 [@media(pointer:coarse)]:w-auto items-center justify-center"> | |
| <CircleAlert | |
| class="h-3.5 w-3.5 text-red-500 group-hover:hidden [@media(pointer:coarse)]:hidden" | |
| /> | |
| <div class="hidden group-hover:flex [@media(pointer:coarse)]:flex"> | |
| <ActionIcon | |
| iconSize="h-2.5 w-2.5" | |
| icon={RotateCw} | |
| tooltip="Retry loading model" | |
| class="h-3 w-3 text-red-500 hover:text-foreground" | |
| onclick={() => modelsStore.loadModel(option.model)} | |
| stopPropagationOnClick | |
| /> | |
| </div> | |
| </div> | |
| {:else if isSleeping} | |
| <div class="flex w-4 [@media(pointer:coarse)]:w-auto items-center justify-center"> | |
| <span | |
| class="h-2 w-2 rounded-full bg-orange-400 group-hover:hidden [@media(pointer:coarse)]:hidden" | |
| ></span> | |
| <div class="hidden group-hover:flex [@media(pointer:coarse)]:flex"> | |
| <ActionIcon | |
| iconSize="h-2.5 w-2.5" | |
| icon={PowerOff} | |
| tooltip="Unload model" | |
| class="h-3 w-3 text-red-500 hover:text-red-600 [@media(pointer:coarse)]:text-amber-500 [@media(pointer:coarse)]:hover:text-amber-600" | |
| onclick={(e) => { | |
| e?.stopPropagation(); | |
| modelsStore.unloadModel(option.model); | |
| }} | |
| /> | |
| </div> | |
| </div> | |
| {:else if isLoaded} | |
| <div class="flex w-4 [@media(pointer:coarse)]:w-auto items-center justify-center"> | |
| <span | |
| class="h-2 w-2 rounded-full bg-green-500 group-hover:hidden [@media(pointer:coarse)]:hidden" | |
| ></span> | |
| <div class="hidden group-hover:flex [@media(pointer:coarse)]:flex"> | |
| <ActionIcon | |
| iconSize="h-2.5 w-2.5" | |
| icon={PowerOff} | |
| tooltip="Unload model" | |
| class="h-3 w-3 text-red-500 hover:text-red-600 [@media(pointer:coarse)]:text-green-500 [@media(pointer:coarse)]:hover:text-green-600" | |
| onclick={() => modelsStore.unloadModel(option.model)} | |
| stopPropagationOnClick | |
| /> | |
| </div> | |
| </div> | |
| {:else} | |
| <div class="flex w-4 [@media(pointer:coarse)]:w-auto items-center justify-center"> | |
| <span | |
| class="h-2 w-2 rounded-full bg-muted-foreground/50 group-hover:hidden [@media(pointer:coarse)]:hidden" | |
| ></span> | |
| <div class="hidden group-hover:flex [@media(pointer:coarse)]:flex"> | |
| <ActionIcon | |
| iconSize="h-2.5 w-2.5" | |
| icon={Power} | |
| tooltip="Load model" | |
| class="h-3 w-3 [@media(pointer:coarse)]:text-muted-foreground" | |
| onclick={() => modelsStore.loadModel(option.model)} | |
| stopPropagationOnClick | |
| /> | |
| </div> | |
| </div> | |
| {/if} | |
| </div> | |
| </div> | |