Text Generation
Safetensors
GGUF
English
reasoning
reinforcement-learning
grpo
small-language-model
samsung-ennovatex
conversational
Instructions to use OmnipotentFool/Aurvion with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use OmnipotentFool/Aurvion with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf OmnipotentFool/Aurvion:Q4_K_M # Run inference directly in the terminal: llama cli -hf OmnipotentFool/Aurvion:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf OmnipotentFool/Aurvion:Q4_K_M # Run inference directly in the terminal: llama cli -hf OmnipotentFool/Aurvion:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf OmnipotentFool/Aurvion:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf OmnipotentFool/Aurvion:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf OmnipotentFool/Aurvion:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf OmnipotentFool/Aurvion:Q4_K_M
Use Docker
docker model run hf.co/OmnipotentFool/Aurvion:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use OmnipotentFool/Aurvion with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "OmnipotentFool/Aurvion" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OmnipotentFool/Aurvion", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/OmnipotentFool/Aurvion:Q4_K_M
- Ollama
How to use OmnipotentFool/Aurvion with Ollama:
ollama run hf.co/OmnipotentFool/Aurvion:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use OmnipotentFool/Aurvion with Docker Model Runner:
docker model run hf.co/OmnipotentFool/Aurvion:Q4_K_M
- Lemonade
How to use OmnipotentFool/Aurvion with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull OmnipotentFool/Aurvion:Q4_K_M
Run and chat with the model
lemonade run user.Aurvion-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Download llama.cpp/tools/server/server-cors-proxy.h from OmnipotentFool/Aurvion: direct link, hf CLI and curl.
- Browser
- Download file 2.61 kB
-
https://huggingface.co/OmnipotentFool/Aurvion/resolve/main/llama.cpp/tools/server/server-cors-proxy.h
- Command line
-
hf download hf://OmnipotentFool/Aurvion/llama.cpp/tools/server/server-cors-proxy.h
-
curl -L -o server-cors-proxy.h https://huggingface.co/OmnipotentFool/Aurvion/resolve/main/llama.cpp/tools/server/server-cors-proxy.h
2.61 kB
| static std::string proxy_header_to_lower(std::string header) { | |
| std::transform(header.begin(), header.end(), header.begin(), [](unsigned char c) { | |
| return std::tolower(c); | |
| }); | |
| return header; | |
| } | |
| static server_http_res_ptr proxy_request(const server_http_req & req, std::string method) { | |
| std::string target_url = req.get_param("url"); | |
| common_http_url parsed_url = common_http_parse_url(target_url); | |
| if (parsed_url.host.empty()) { | |
| throw std::runtime_error("invalid target URL: missing host"); | |
| } | |
| if (parsed_url.path.empty()) { | |
| parsed_url.path = "/"; | |
| } | |
| if (!parsed_url.password.empty()) { | |
| throw std::runtime_error("authentication in target URL is not supported"); | |
| } | |
| if (parsed_url.scheme != "http" && parsed_url.scheme != "https") { | |
| throw std::runtime_error("unsupported URL scheme in target URL: " + parsed_url.scheme); | |
| } | |
| SRV_INF("proxying %s request to %s://%s:%i%s\n", method.c_str(), parsed_url.scheme.c_str(), parsed_url.host.c_str(), parsed_url.port, parsed_url.path.c_str()); | |
| std::map<std::string, std::string> headers; | |
| const std::string proxy_header_prefix = "x-llama-server-proxy-header-"; | |
| for (auto [key, value] : req.headers) { | |
| const std::string lowered_key = proxy_header_to_lower(key); | |
| if (!string_starts_with(lowered_key, proxy_header_prefix)) { | |
| continue; | |
| } | |
| auto new_key = key.substr(proxy_header_prefix.size()); | |
| if (new_key.empty()) { | |
| continue; | |
| } | |
| headers[new_key] = value; | |
| } | |
| auto proxy = std::make_unique<server_http_proxy>( | |
| method, | |
| parsed_url.scheme, | |
| parsed_url.host, | |
| parsed_url.port, | |
| parsed_url.path, | |
| headers, | |
| req.body, | |
| req.files, | |
| req.should_stop, | |
| 600, // timeout_read (default to 10 minutes) | |
| 600 // timeout_write (default to 10 minutes) | |
| ); | |
| return proxy; | |
| } | |
| static server_http_context::handler_t proxy_handler_post = [](const server_http_req & req) -> server_http_res_ptr { | |
| return proxy_request(req, "POST"); | |
| }; | |
| static server_http_context::handler_t proxy_handler_get = [](const server_http_req & req) -> server_http_res_ptr { | |
| return proxy_request(req, "GET"); | |
| }; | |