Text Generation
Safetensors
GGUF
English
reasoning
reinforcement-learning
grpo
small-language-model
samsung-ennovatex
conversational
Instructions to use OmnipotentFool/Aurvion with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use OmnipotentFool/Aurvion with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf OmnipotentFool/Aurvion:Q4_K_M # Run inference directly in the terminal: llama cli -hf OmnipotentFool/Aurvion:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf OmnipotentFool/Aurvion:Q4_K_M # Run inference directly in the terminal: llama cli -hf OmnipotentFool/Aurvion:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf OmnipotentFool/Aurvion:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf OmnipotentFool/Aurvion:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf OmnipotentFool/Aurvion:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf OmnipotentFool/Aurvion:Q4_K_M
Use Docker
docker model run hf.co/OmnipotentFool/Aurvion:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use OmnipotentFool/Aurvion with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "OmnipotentFool/Aurvion" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OmnipotentFool/Aurvion", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/OmnipotentFool/Aurvion:Q4_K_M
- Ollama
How to use OmnipotentFool/Aurvion with Ollama:
ollama run hf.co/OmnipotentFool/Aurvion:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use OmnipotentFool/Aurvion with Docker Model Runner:
docker model run hf.co/OmnipotentFool/Aurvion:Q4_K_M
- Lemonade
How to use OmnipotentFool/Aurvion with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull OmnipotentFool/Aurvion:Q4_K_M
Run and chat with the model
lemonade run user.Aurvion-Q4_K_M
List all available models
lemonade list
- Atomic Chat
|
Download llama.cpp/tools/ui/docs/flows/chat-flow.md from OmnipotentFool/Aurvion: direct link, hf CLI and curl.
- Browser
- Download file 13.3 kB
-
https://huggingface.co/OmnipotentFool/Aurvion/resolve/main/llama.cpp/tools/ui/docs/flows/chat-flow.md
- Command line
-
hf download hf://OmnipotentFool/Aurvion/llama.cpp/tools/ui/docs/flows/chat-flow.md
-
curl -L -o chat-flow.md https://huggingface.co/OmnipotentFool/Aurvion/resolve/main/llama.cpp/tools/ui/docs/flows/chat-flow.md
13.3 kB
| ```mermaid | |
| sequenceDiagram | |
| participant UI as π§© ChatForm / ChatMessage | |
| participant chatStore as ποΈ chatStore | |
| participant agenticStore as ποΈ agenticStore | |
| participant convStore as ποΈ conversationsStore | |
| participant settingsStore as ποΈ settingsStore | |
| participant mcpStore as ποΈ mcpStore | |
| participant ChatSvc as βοΈ ChatService | |
| participant DbSvc as βοΈ DatabaseService | |
| participant API as π /v1/chat/completions | |
| Note over chatStore: State:<br/>isLoading, currentResponse<br/>errorDialogState, activeProcessingState<br/>chatLoadingStates (Map)<br/>chatStreamingStates (Map)<br/>abortControllers (Map)<br/>processingStates (Map) | |
| %% βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| Note over UI,API: π¬ SEND MESSAGE | |
| %% βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| UI->>chatStore: sendMessage(content, extras) | |
| activate chatStore | |
| chatStore->>chatStore: setChatLoading(convId, true) | |
| chatStore->>chatStore: clearChatStreaming(convId) | |
| alt no active conversation | |
| chatStore->>convStore: createConversation() | |
| Note over convStore: β see conversations-flow.mmd | |
| end | |
| chatStore->>mcpStore: consumeResourceAttachmentsAsExtras() | |
| Note right of mcpStore: Converts pending MCP resource<br/>attachments into message extras | |
| chatStore->>chatStore: addMessage("user", content, extras) | |
| chatStore->>DbSvc: createMessageBranch(userMsg, parentId) | |
| chatStore->>convStore: addMessageToActive(userMsg) | |
| chatStore->>convStore: updateCurrentNode(userMsg.id) | |
| chatStore->>chatStore: createAssistantMessage(userMsg.id) | |
| chatStore->>DbSvc: createMessageBranch(assistantMsg, userMsg.id) | |
| chatStore->>convStore: addMessageToActive(assistantMsg) | |
| chatStore->>chatStore: streamChatCompletion(messages, assistantMsg) | |
| deactivate chatStore | |
| %% βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| Note over UI,API: π STREAMING (with agentic flow detection) | |
| %% βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| activate chatStore | |
| chatStore->>chatStore: startStreaming() | |
| Note right of chatStore: isStreamingActive = true | |
| chatStore->>chatStore: setActiveProcessingConversation(convId) | |
| chatStore->>chatStore: getOrCreateAbortController(convId) | |
| Note right of chatStore: abortControllers.set(convId, new AbortController()) | |
| chatStore->>chatStore: getApiOptions() | |
| Note right of chatStore: Merge from settingsStore.config:<br/>temperature, max_tokens, top_p, etc. | |
| alt agenticConfig.enabled && mcpStore has connected servers | |
| chatStore->>agenticStore: runAgenticFlow(convId, messages, assistantMsg, options, signal) | |
| Note over agenticStore: Multi-turn agentic loop:<br/>1. Call ChatService.sendMessage()<br/>2. If response has tool_calls β execute via mcpStore<br/>3. Append tool results as messages<br/>4. Loop until no more tool_calls or maxTurns<br/>β see agentic flow details below | |
| agenticStore-->>chatStore: final response with timings | |
| else standard (non-agentic) flow | |
| chatStore->>ChatSvc: sendMessage(messages, options, signal) | |
| end | |
| activate ChatSvc | |
| ChatSvc->>ChatSvc: convertDbMessageToApiChatMessageData(messages) | |
| Note right of ChatSvc: DatabaseMessage[] β ApiChatMessageData[]<br/>Process attachments (images, PDFs, audio) | |
| ChatSvc->>API: POST /v1/chat/completions | |
| Note right of API: {messages, model?, stream: true, ...params} | |
| loop SSE chunks | |
| API-->>ChatSvc: data: {"choices":[{"delta":{...}}]} | |
| ChatSvc->>ChatSvc: handleStreamResponse(response) | |
| alt content chunk | |
| ChatSvc-->>chatStore: onChunk(content) | |
| chatStore->>chatStore: setChatStreaming(convId, response, msgId) | |
| Note right of chatStore: currentResponse = $state(accumulated) | |
| chatStore->>convStore: updateMessageAtIndex(idx, {content}) | |
| end | |
| alt reasoning chunk | |
| ChatSvc-->>chatStore: onReasoningChunk(reasoning) | |
| chatStore->>convStore: updateMessageAtIndex(idx, {thinking}) | |
| end | |
| alt tool_calls chunk | |
| ChatSvc-->>chatStore: onToolCallChunk(toolCalls) | |
| chatStore->>convStore: updateMessageAtIndex(idx, {toolCalls}) | |
| end | |
| alt model info | |
| ChatSvc-->>chatStore: onModel(modelName) | |
| chatStore->>chatStore: recordModel(modelName) | |
| chatStore->>DbSvc: updateMessage(msgId, {model}) | |
| end | |
| alt timings (during stream) | |
| ChatSvc-->>chatStore: onTimings(timings, promptProgress) | |
| chatStore->>chatStore: updateProcessingStateFromTimings() | |
| end | |
| chatStore-->>UI: reactive $state update | |
| end | |
| API-->>ChatSvc: data: [DONE] | |
| ChatSvc-->>chatStore: onComplete(content, reasoning, timings, toolCalls) | |
| deactivate ChatSvc | |
| chatStore->>chatStore: stopStreaming() | |
| chatStore->>DbSvc: updateMessage(msgId, {content, timings, model}) | |
| chatStore->>convStore: updateCurrentNode(msgId) | |
| chatStore->>chatStore: setChatLoading(convId, false) | |
| chatStore->>chatStore: clearChatStreaming(convId) | |
| chatStore->>chatStore: clearProcessingState(convId) | |
| deactivate chatStore | |
| %% βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| Note over UI,API: βΉοΈ STOP GENERATION | |
| %% βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| UI->>chatStore: stopGeneration() | |
| activate chatStore | |
| chatStore->>chatStore: savePartialResponseIfNeeded(convId) | |
| Note right of chatStore: Save currentResponse to DB if non-empty | |
| chatStore->>chatStore: abortControllers.get(convId).abort() | |
| Note right of chatStore: fetch throws AbortError β caught by isAbortError() | |
| chatStore->>chatStore: stopStreaming() | |
| chatStore->>chatStore: setChatLoading(convId, false) | |
| chatStore->>chatStore: clearChatStreaming(convId) | |
| chatStore->>chatStore: clearProcessingState(convId) | |
| deactivate chatStore | |
| %% βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| Note over UI,API: π REGENERATE | |
| %% βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| UI->>chatStore: regenerateMessageWithBranching(msgId, model?) | |
| activate chatStore | |
| chatStore->>convStore: findMessageIndex(msgId) | |
| chatStore->>chatStore: Get parent of target message | |
| chatStore->>chatStore: createAssistantMessage(parentId) | |
| chatStore->>DbSvc: createMessageBranch(newAssistantMsg, parentId) | |
| chatStore->>convStore: refreshActiveMessages() | |
| Note right of chatStore: Same streaming flow | |
| chatStore->>chatStore: streamChatCompletion(...) | |
| deactivate chatStore | |
| %% βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| Note over UI,API: β‘οΈ CONTINUE | |
| %% βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| UI->>chatStore: continueAssistantMessage(msgId) | |
| activate chatStore | |
| chatStore->>chatStore: Get existing content from message | |
| chatStore->>chatStore: streamChatCompletion(..., existingContent) | |
| Note right of chatStore: Appends to existing message content | |
| deactivate chatStore | |
| %% βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| Note over UI,API: βοΈ EDIT USER MESSAGE | |
| %% βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| UI->>chatStore: editMessageWithBranching(msgId, newContent, extras) | |
| activate chatStore | |
| chatStore->>chatStore: Get parent of target message | |
| chatStore->>DbSvc: createMessageBranch(editedMsg, parentId) | |
| chatStore->>convStore: refreshActiveMessages() | |
| Note right of chatStore: Creates new branch, original preserved | |
| chatStore->>chatStore: createAssistantMessage(editedMsg.id) | |
| chatStore->>chatStore: streamChatCompletion(...) | |
| Note right of chatStore: Automatically regenerates response | |
| deactivate chatStore | |
| %% βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| Note over UI,API: β ERROR HANDLING | |
| %% βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| Note over chatStore: On stream error (non-abort): | |
| chatStore->>chatStore: showErrorDialog(type, message) | |
| Note right of chatStore: errorDialogState = {type: 'timeout'|'server', message} | |
| chatStore->>convStore: removeMessageAtIndex(failedMsgIdx) | |
| chatStore->>DbSvc: deleteMessage(failedMsgId) | |
| %% βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| Note over UI,API: π€ AGENTIC LOOP (when agenticConfig.enabled) | |
| %% βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| Note over agenticStore: agenticStore.runAgenticFlow(convId, messages, assistantMsg, options, signal) | |
| activate agenticStore | |
| agenticStore->>agenticStore: getSession(convId) or create new | |
| agenticStore->>agenticStore: updateSession(turn: 0, running: true) | |
| loop executeAgenticLoop (until no tool_calls or maxTurns) | |
| agenticStore->>agenticStore: turn++ | |
| agenticStore->>ChatSvc: sendMessage(messages, options, signal) | |
| ChatSvc->>API: POST /v1/chat/completions | |
| API-->>ChatSvc: response with potential tool_calls | |
| ChatSvc-->>agenticStore: onComplete(content, reasoning, timings, toolCalls) | |
| alt response has tool_calls | |
| agenticStore->>agenticStore: normalizeToolCalls(toolCalls) | |
| loop for each tool_call | |
| agenticStore->>agenticStore: updateSession(streamingToolCall) | |
| agenticStore->>mcpStore: executeTool(mcpCall, signal) | |
| mcpStore-->>agenticStore: tool result | |
| agenticStore->>agenticStore: extractBase64Attachments(result) | |
| agenticStore->>agenticStore: emitToolCallResult(convId, ...) | |
| agenticStore->>convStore: addMessageToActive(toolResultMsg) | |
| agenticStore->>DbSvc: createMessageBranch(toolResultMsg) | |
| end | |
| agenticStore->>agenticStore: Create new assistantMsg for next turn | |
| Note right of agenticStore: Continue loop with updated messages | |
| else no tool_calls (final response) | |
| agenticStore->>agenticStore: buildFinalTimings(allTurns) | |
| Note right of agenticStore: Break loop, return final response | |
| end | |
| end | |
| agenticStore->>agenticStore: updateSession(running: false) | |
| agenticStore-->>chatStore: final content, timings, model | |
| deactivate agenticStore | |
| ``` | |