Instructions to use srs6901/Vikras-MixP with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use srs6901/Vikras-MixP with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="srs6901/Vikras-MixP")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("srs6901/Vikras-MixP", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use srs6901/Vikras-MixP with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf srs6901/Vikras-MixP:Q8_0 # Run inference directly in the terminal: llama cli -hf srs6901/Vikras-MixP:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf srs6901/Vikras-MixP:Q8_0 # Run inference directly in the terminal: llama cli -hf srs6901/Vikras-MixP:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf srs6901/Vikras-MixP:Q8_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf srs6901/Vikras-MixP:Q8_0
Use Docker
docker model run hf.co/srs6901/Vikras-MixP:Q8_0
- Llama new
- LM Studio
- Jan
- vLLM
How to use srs6901/Vikras-MixP with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "srs6901/Vikras-MixP" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "srs6901/Vikras-MixP", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/srs6901/Vikras-MixP:Q8_0
- SGLang
How to use srs6901/Vikras-MixP with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "srs6901/Vikras-MixP" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "srs6901/Vikras-MixP", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "srs6901/Vikras-MixP" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "srs6901/Vikras-MixP", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use srs6901/Vikras-MixP with Ollama:
ollama run hf.co/srs6901/Vikras-MixP:Q8_0
- Unsloth Desktop
- Pi
How to use srs6901/Vikras-MixP with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf srs6901/Vikras-MixP:Q8_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "srs6901/Vikras-MixP:Q8_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use srs6901/Vikras-MixP with Docker Model Runner:
docker model run hf.co/srs6901/Vikras-MixP:Q8_0
- Lemonade
How to use srs6901/Vikras-MixP with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull srs6901/Vikras-MixP:Q8_0
Run and chat with the model
lemonade run user.Vikras-MixP-Q8_0
List all available models
lemonade list
- Hermes Agent
How to use srs6901/Vikras-MixP with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf srs6901/Vikras-MixP:Q8_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default srs6901/Vikras-MixP:Q8_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use srs6901/Vikras-MixP with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf srs6901/Vikras-MixP:Q8_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "srs6901/Vikras-MixP:Q8_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Vikras β Experimental Family of Language Models
- Π‘ΠΎΠ΄Π΅ΡΠΆΠ°Π½ΠΈΠ΅
- ΠΠΎΡΠΎΡΠΊΠΎ ΠΎ ΠΏΡΠΎΠ΅ΠΊΡΠ΅
- Π’Π΅ΠΊΡΡΠΈΠΉ ΡΠ΅Π»ΠΈΠ·: HCT/YeAM
- HCT (Π°ΡΡ
ΠΈΡΠ΅ΠΊΡΡΡΠ°) / YeAM (ΠΈΠ½Π²Π°ΡΠΈΠ°Π½Ρ ΡΠ΅Π°Π»ΠΈΠ·Π°ΡΠΈΠΈ)
- ΠΡΠ΅Π΄ΡΠ΄ΡΡΠΈΠΉ ΡΠ΅Π»ΠΈΠ·: Vikra MixedPrc (MixP_4.9b_S)
- MixP_4.9b_S: Π΄Π΅ΡΠ°Π»ΠΈ
- ΠΠ»Π°Π½Ρ ΡΠ°Π·Π²ΠΈΡΠΈΡ
- ΠΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΠ΅
- ΠΠ°ΠΊΠ»ΡΡΠ΅Π½ΠΈΠ΅
- Π‘ΠΎΠ΄Π΅ΡΠΆΠ°Π½ΠΈΠ΅
- Vikras β Experimental Family of Language Models (EN)
Vikras β Experimental Family of Language Models
Π‘ΠΎΠ΄Π΅ΡΠΆΠ°Π½ΠΈΠ΅
- ΠΠΎΡΠΎΡΠΊΠΎ ΠΎ ΠΏΡΠΎΠ΅ΠΊΡΠ΅
- Π’Π΅ΠΊΡΡΠΈΠΉ ΡΠ΅Π»ΠΈΠ·: HCT/YeAM
- HCT (Π°ΡΡ ΠΈΡΠ΅ΠΊΡΡΡΠ°) / YeAM (ΠΈΠ½Π²Π°ΡΠΈΠ°Π½Ρ ΡΠ΅Π°Π»ΠΈΠ·Π°ΡΠΈΠΈ)
- ΠΡΠ΅Π΄ΡΠ΄ΡΡΠΈΠΉ ΡΠ΅Π»ΠΈΠ·: Vikra MixedPrc (MixP_4.9b_S)
- MixP_4.9b_S: Π΄Π΅ΡΠ°Π»ΠΈ
- ΠΠ»Π°Π½Ρ ΡΠ°Π·Π²ΠΈΡΠΈΡ
- ΠΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΠ΅
- ΠΠ°ΠΊΠ»ΡΡΠ΅Π½ΠΈΠ΅
ΠΠΎΡΠΎΡΠΊΠΎ ΠΎ ΠΏΡΠΎΠ΅ΠΊΡΠ΅
Vikra β ΡΠΊΡΠΏΠ΅ΡΠΈΠΌΠ΅Π½ΡΠ°Π»ΡΠ½ΠΎΠ΅ ΡΠ΅ΠΌΠ΅ΠΉΡΡΠ²ΠΎ ΡΠ·ΡΠΊΠΎΠ²ΡΡ ΠΌΠΎΠ΄Π΅Π»Π΅ΠΉ, ΠΈΡΡΠ»Π΅Π΄ΡΡΡΠ΅Π΅ Π²Π»ΠΈΡΠ½ΠΈΠ΅:
- Π³Π΅ΠΎΠΌΠ΅ΡΡΠΈΠΈ ΠΏΡΠ΅Π΄ΡΡΠ°Π²Π»Π΅Π½ΠΈΠΉ
- ΠΊΠ²Π°Π½ΡΠΎΠ²Π°Π½ΠΈΡ
- Π³ΠΈΠ±ΡΠΈΠ΄Π½ΡΡ ΠΌΠ΅ΡΠ΄ΠΆΠ΅ΠΉ
Π½Π° ΡΠΈΡΠ»Π΅Π½Π½ΡΡ Π΄ΠΈΠ½Π°ΠΌΠΈΠΊΡ ΡΡΠ°Π½ΡΡΠΎΡΠΌΠ΅ΡΠΎΠ².
ΠΡΠΎΠ΅ΠΊΡ Vikras Π½Π΅ ΠΎΠ³ΡΠ°Π½ΠΈΡΠΈΠ²Π°Π΅ΡΡΡ ΠΎΠ΄Π½ΠΎΠΉ Π±Π°Π·ΠΎΠΉ ΠΈΠ»ΠΈ ΠΎΠ΄Π½ΠΎΠΉ Π°ΡΡ ΠΈΡΠ΅ΠΊΡΡΡΠΎΠΉ: ΡΡΠΎ ΡΠ΅ΠΌΠ΅ΠΉΡΡΠ²ΠΎ ΠΌΠΎΠ΄Π΅Π»Π΅ΠΉ, ΠΎΠ±ΡΠ΅Π΄ΠΈΠ½ΡΠ½Π½ΡΡ ΠΈΠ΄Π΅Π΅ΠΉ ΡΠΈΡΠ»Π΅Π½Π½ΠΎΠΉ ΠΈΠ½Π²Π°ΡΠΈΠ°Π½ΡΠ½ΠΎΡΡΠΈ ΡΠΊΡΠΏΠ΅ΡΠΈΠΌΠ΅Π½ΡΠ°.
- Vikra_% β ΠΈΠΌΡ ΠΊΠΎΠ½ΠΊΡΠ΅ΡΠ½ΠΎΠΉ ΠΌΠΎΠ΄Π΅Π»ΠΈ
- Vikras β ΡΠ΅ΠΌΠ΅ΠΉΡΡΠ²ΠΎ ΡΠΊΡΠΏΠ΅ΡΠΈΠΌΠ΅Π½ΡΠΎΠ²
- S / M / L β ΡΡΠ΅ΠΏΠ΅Π½Ρ Π°Π³ΡΠ΅ΡΡΠΈΠ²Π½ΠΎΡΡΠΈ ΠΈ ΡΠ°ΡΠΏΡΠ΅Π΄Π΅Π»Π΅Π½ΠΈΡ Π±ΠΈΡΠ½ΠΎΡΡΠΈ
- MixP / FullP / HCT β ΡΡ Π΅ΠΌΡ ΠΈ ΠΈΠ½Π²Π°ΡΠΈΠ°Π½ΡΡ ΠΊΠ²Π°Π½ΡΠΎΠ²Π°Π½ΠΈΡ/ΠΌΠ΅ΡΠ΄ΠΆΠ΅ΠΉ
Π’Π΅ΠΊΡΡΠΈΠΉ ΡΠ΅Π»ΠΈΠ·: HCT/YeAM
Π Π΅Π»ΠΈΠ·Ρ
- Vikra-HCT-YeAM-PhiMma-1B
- Vikra-HCT-YeAM-LLaGemma-1B
- Vikra-HCT-YeAM-3_3.2_QweLLa-1.7B
- Vikra-HCT-YeAM-Vikhr-NemoGemma-12B_plus_1B
HCT (Π°ΡΡ ΠΈΡΠ΅ΠΊΡΡΡΠ°) / YeAM (ΠΈΠ½Π²Π°ΡΠΈΠ°Π½Ρ ΡΠ΅Π°Π»ΠΈΠ·Π°ΡΠΈΠΈ)
HCT β Π°ΡΡ ΠΈΡΠ΅ΠΊΡΡΡΠ½ΡΠΉ ΠΈΠ½Π²Π°ΡΠΈΠ°Π½Ρ: ΠΏΡΠ°ΠΊΡΠΈΡΠ΅ΡΠΊΠΈΠΉ ΡΠΏΠΎΡΠΎΠ± ΡΠΎΠ±ΠΈΡΠ°ΡΡ ΡΠΎΠ²ΠΌΠ΅ΡΡΠΈΠΌΡΠ΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ ΠΈ ΠΏΡΠΎΠΈΠ·Π²ΠΎΠ΄Π½ΡΠ΅ ΡΠ΅Π»ΠΈΠ·Ρ ΠΏΡΠΈ ΠΏΠ΅ΡΠ΅Π½ΠΎΡΠ΅ ΠΌΠ΅ΠΆΠ΄Ρ Π±Π°Π·Π°ΠΌΠΈ/ΡΠ΅ΠΌΠ΅ΠΉΡΡΠ²Π°ΠΌΠΈ.
YeAM (Yet Another Merge) β ΠΈΠ½Π²Π°ΡΠΈΠ°Π½Ρ ΡΠ΅Π°Π»ΠΈΠ·Π°ΡΠΈΠΈ HCT ΠΈ ΡΠ°ΠΌΠΎΡΡΠΎΡΡΠ΅Π»ΡΠ½Π°Ρ ΡΡ Π΅ΠΌΠ° ΠΌΠ΅ΡΠ΄ΠΆΠ° HFβHF: ΡΡΠΎ Π½Π΅ Β«Π΅ΡΡ ΠΎΠ΄ΠΈΠ½ SLERP/DARE/TILESΒ» ΠΈ Π½Π΅ ΠΊΠΎΡΠΌΠ΅ΡΠΈΡΠ΅ΡΠΊΠ°Ρ Π²Π°ΡΠΈΠ°ΡΠΈΡ ΡΡΡΠ΅Π΄Π½Π΅Π½ΠΈΡ.
YeAM Π²ΡΠ΄Π°ΡΡ ΡΡΠ°Π½Π΄Π°ΡΡΠ½ΡΠΉ HF-ΡΠ΅Π·ΡΠ»ΡΡΠ°Ρ (safetensors + index) ΠΈ ΠΏΠΎΠ΄Π΄Π΅ΡΠΆΠΈΠ²Π°Π΅Ρ:
- ΠΏΡΡΠΌΠΎΠΉ weight-to-weight ΠΌΠ΅ΡΠ΄ΠΆ
- Π½Π°ΠΏΡΠ°Π²Π»Π΅Π½Π½ΠΎΠ΅ Π΄ΠΎΠ±Π°Π²Π»Π΅Π½ΠΈΠ΅ Π·Π½Π°Π½ΠΈΠΉ Π² Π²ΡΠ±ΡΠ°Π½Π½ΡΡ ΠΌΠΎΠ΄Π΅Π»Ρ (knowledge distillation / knowledge injection), ΡΠΎΠ³Π»Π°ΡΠΎΠ²Π°Π½Π½ΠΎΠ΅ ΠΏΠΎ Π½Π΅ΡΠΊΠΎΠ»ΡΠΊΠΈΠΌ ΠΈΡΡΠΎΡΠ½ΠΈΠΊΠ°ΠΌ
- Π΄ΠΎΠΏΠΎΠ»Π½ΠΈΡΠ΅Π»ΡΠ½ΡΠΉ ΠΌΠ΅ΡΠ΄ΠΆ Attention-ΡΠ»ΠΎΡΠ² ΠΊΠ°ΠΊ ΠΎΡΠ΄Π΅Π»ΡΠ½ΡΡ ΡΠ΅Ρ Π½ΠΈΠΊΡ ΠΏΠΎΠ²Π΅ΡΡ YeAM
- ΠΌΠ΅ΡΠ΄ΠΆ ΠΌΠ΅Π½ΡΡΠΈΡ ΠΌΠΎΠ΄Π΅Π»Π΅ΠΉ Π² Π±ΠΎΠ»Π΅Π΅ ΠΊΡΡΠΏΠ½ΡΠ΅ (scale-up merge) ΠΏΡΠΈ ΡΠΎΡ ΡΠ°Π½Π΅Π½ΠΈΠΈ ΡΠΎΠ²ΠΌΠ΅ΡΡΠΈΠΌΠΎΠ³ΠΎ HF-ΡΠΎΡΠΌΠ°ΡΠ°
ΠΠ°ΡΠ΅ΠΌΠ°ΡΠΈΡΠ΅ΡΠΊΠΈ YeAM ΡΠ°Π±ΠΎΡΠ°Π΅Ρ Π² ΡΠ΅Π°Π»ΡΠ½ΠΎΠΉ 4D-ΠΏΠΎΡΡΠ°Π½ΠΎΠ²ΠΊΠ΅: ΠΎΠ±Π½ΠΎΠ²Π»Π΅Π½ΠΈΡ ΠΊΠΎΠ΄ΠΈΡΡΡΡΡΡ Π³Π΅ΠΎΠΌΠ΅ΡΡΠΈΡΠ΅ΡΠΊΠΈ ΠΈ ΡΠΎΠ³Π»Π°ΡΡΡΡΡΡ ΡΠ΅ΡΠ΅Π· ΠΏΠ΅ΡΠ΅ΡΠ΅ΡΠ΅Π½ΠΈΡ Π»ΡΡΠ΅ΠΉ Π² ΠΏΡΠΎΡΡΡΠ°Π½ΡΡΠ²Π΅ ΠΏΠ°ΡΠ°ΠΌΠ΅ΡΡΠΎΠ². ΠΡΠΎ Π΄Π°ΡΡ ΡΠΏΡΠ°Π²Π»ΡΠ΅ΠΌΡΠΉ ΠΌΠ΅ΡΠ΄ΠΆ Ρ ΡΠΎΡ ΡΠ°Π½Π΅Π½ΠΈΠ΅ΠΌ ΡΡΡΡΠΊΡΡΡΡ ΠΈ Π±Π΅Π· Π²ΡΡΠΎΠΆΠ΄Π΅Π½ΠΈΡ Π² Π½Π°ΠΈΠ²Π½ΠΎΠ΅ ΡΡΡΠ΅Π΄Π½Π΅Π½ΠΈΠ΅.
ΠΡΠ΅Π΄ΡΠ΄ΡΡΠΈΠΉ ΡΠ΅Π»ΠΈΠ·: Vikra MixedPrc (MixP_4.9b_S)
ΠΡΠ°ΡΠΊΠΎΠ΅ ΠΎΠΏΠΈΡΠ°Π½ΠΈΠ΅
12.25B Mistral-based language model
Hybrid mixed-precision merged GGUF quantization
ΠΠΊΡΠΏΠ΅ΡΠΈΠΌΠ΅Π½ΡΠ°Π»ΡΠ½ΡΠΉ ΡΠ΅ΠΆΠΈΠΌ Π°Π½ΠΈΠ·ΠΎΡΡΠΎΠΏΠ½ΠΎΠ³ΠΎ ΠΊΠ²Π°Π½ΡΠΎΠ²Π°Π½ΠΈΡ
ΠΠΎΠ»Π½Π°Ρ Π²Π΅ΡΡΠΈΡ ΠΌΠ΅ΡΠ΄ΠΆΠ° (Π±Π΅Π· ΠΊΠ²Π°Π½ΡΠΎΠ²Π°Π½ΠΈΡ): https://huggingface.co/srs6901/Vikras-MixP/tree/main/Vikra-FullP
GGUF-ΠΊΠ²Π°Π½Ρ: https://huggingface.co/srs6901/Vikras-MixP/blob/main/Vikra-MixP_4.9b_S.gguf
MixP_4.9b_S: Π΄Π΅ΡΠ°Π»ΠΈ
ΠΡΡ ΠΈΡΠ΅ΠΊΡΡΡΠ° (Π΄Π»Ρ MixP ΡΠ΅Π»ΠΈΠ·Π°)
| ΠΠ°ΡΠ°ΠΌΠ΅ΡΡ | ΠΠ½Π°ΡΠ΅Π½ΠΈΠ΅ |
|---|---|
| Architecture | Mistral-based |
| Params | ~12.25B |
| Layers | 40 |
| Hidden size | 5120 |
| FFN size | 14336 |
| Heads | 32 (8 KV heads, GQA) |
| Context | 1,024,000 |
| Vocab | 131,072 (Tekken BPE) |
| RoPE theta | 1,000,000 |
MixP_4.9b_S β ΡΡ Π΅ΠΌΠ° ΠΊΠ²Π°Π½ΡΠΎΠ²Π°Π½ΠΈΡ
ΠΠΈΠ±ΡΠΈΠ΄Π½Π°Ρ mixed precision ΡΡ Π΅ΠΌΠ° Ρ ΠΏΠΎΠΊΠΎΠΌΠΏΠΎΠ½Π΅Π½ΡΠ½ΡΠΌ ΡΠ°ΡΠΏΡΠ΅Π΄Π΅Π»Π΅Π½ΠΈΠ΅ΠΌ ΡΠΈΠΏΠΎΠ².
| Tensor group | Quant type | BPW |
|---|---|---|
| token_embd, output | BF16 | 16 |
| attn_norm, ffn_norm, output_norm | F32 | 32 |
| attn_q | Q4_K | 4.5 |
| attn_k | Q5_K | 5.5 |
| attn_v | Q3_K | 3.44 |
| attn_output | Q4_K | 4.5 |
| ffn_gate | Q3_K | 3.44 |
| ffn_up | Q5_K | 5.5 |
| ffn_down | Q5_K / Q6_K | 5.5β6.56 |
ΠΡΠΎΠ³ΠΎ:
- Quantized layers only: ~4.89 BPW
- Full model average: ~6.11 BPW
- File size: ~8.71 GB
ΠΠ»ΡΡΠ΅Π²Π°Ρ ΠΈΠ΄Π΅Ρ MixP
MixP β ΡΡΠΎ Π½Π΅ Β«ΡΠΆΠ°ΡΡ Π²ΡΡ ΠΎΠ΄ΠΈΠ½Π°ΠΊΠΎΠ²ΠΎΒ».
ΠΡΠΎ Π°Π½ΠΈΠ·ΠΎΡΡΠΎΠΏΠ½ΠΎΠ΅ ΠΊΠ²Π°Π½ΡΠΎΠ²Π°Π½ΠΈΠ΅ ΠΈΠ½ΡΠΎΡΠΌΠ°ΡΠΈΠΎΠ½Π½ΡΡ ΠΊΠ°Π½Π°Π»ΠΎΠ²:
β’ Q/K ΡΠΎΡ ΡΠ°Π½ΡΡΡΡΡ Π² Π±ΠΎΠ»Π΅Π΅ Π²ΡΡΠΎΠΊΠΎΠΉ ΡΠΎΡΠ½ΠΎΡΡΠΈ β’ V ΠΈ gate Π½Π°ΠΌΠ΅ΡΠ΅Π½Π½ΠΎ ΠΊΠ²Π°Π½ΡΠΎΠ²Π°Π½Ρ Π΄ΠΎ Q3_K β’ ΠΠΎΡΠΌΡ ΠΈ Π²ΡΡ ΠΎΠ΄Π½ΠΎΠΉ ΡΠ»ΠΎΠΉ ΠΎΡΡΠ°ΡΡΡΡ Π² Π²ΡΡΠΎΠΊΠΎΠΉ ΡΠΎΡΠ½ΠΎΡΡΠΈ
Π’Π°ΠΊΠΎΠ΅ ΡΠ°ΡΠΏΡΠ΅Π΄Π΅Π»Π΅Π½ΠΈΠ΅ ΠΈΠ·ΠΌΠ΅Π½ΡΠ΅Ρ ΡΠΈΡΠ»Π΅Π½Π½ΡΡ Π΄ΠΈΠ½Π°ΠΌΠΈΠΊΡ ΠΌΠΎΠ΄Π΅Π»ΠΈ:
β’ ΡΡΠΈΠ»ΠΈΠ²Π°Π΅ΡΡΡ ΡΡΡΡΠΊΡΡΡΠ½Π°Ρ sparsification β’ ΠΌΠ΅Π½ΡΠ΅ΡΡΡ ΡΠ°ΡΠΏΡΠ΅Π΄Π΅Π»Π΅Π½ΠΈΠ΅ Π½ΠΎΡΠΌ ΡΠΊΡΡΡΡΡ ΠΏΡΠ΅Π΄ΡΡΠ°Π²Π»Π΅Π½ΠΈΠΉ β’ ΠΌΠ΅Π½ΡΠ΅ΡΡΡ ΡΠ½ΡΡΠΎΠΏΠΈΡ Π»ΠΎΠ³ΠΈΡΠΎΠ² β’ ΠΏΠΎΡΠ²Π»ΡΠ΅ΡΡΡ ΡΠ΅ΠΆΠΈΠΌΠ½Π°Ρ ΡΡΠ²ΡΡΠ²ΠΈΡΠ΅Π»ΡΠ½ΠΎΡΡΡ
ΠΡΠΎ Π½Π΅ Π½ΠΎΠ²Π°Ρ Π°ΡΡ ΠΈΡΠ΅ΠΊΡΡΡΠ°. ΠΡΠΎ ΠΈΠ·ΠΌΠ΅Π½Π΅Π½ΠΈΠ΅ ΡΠΈΡΠ»Π΅Π½Π½ΠΎΠΉ Π³Π΅ΠΎΠΌΠ΅ΡΡΠΈΠΈ ΡΡΡΠ΅ΡΡΠ²ΡΡΡΠ΅ΠΉ.
ΠΠ°Π±Π»ΡΠ΄Π°Π΅ΠΌΡΠ΅ ΡΡΡΠ΅ΠΊΡΡ
- ΡΠΎΡ ΡΠ°Π½Π΅Π½ΠΈΠ΅ top-1 ΠΏΡΠ΅Π΄ΡΠΊΠ°Π·Π°Π½ΠΈΠΉ Π½Π° ΠΏΡΠΎΡΡΡΡ Π·Π°Π΄Π°ΡΠ°Ρ
- ΡΠΎΡΡ entropy Π±Π΅Π· ΡΠ°Π·ΡΡΡΠ΅Π½ΠΈΡ ΠΌΠ°ΠΊΡΠΈΠΌΠ°Π»ΡΠ½ΠΎΠΉ Π²Π΅ΡΠΎΡΡΠ½ΠΎΡΡΠΈ
- ΡΠ°ΡΡΠΈΡΠ΅Π½ΠΈΠ΅ hidden norm Π½Π° ΡΠ»ΠΎΠΆΠ½ΡΡ Π·Π°Π΄Π°ΡΠ°Ρ
- Π±ΠΈΡΡΡΠΊΠ°ΡΠΈΡ ΡΠ΅ΠΆΠΈΠΌΠΎΠ²: ΠΏΡΠΎΡΡΡΠ΅ Π·Π°Π΄Π°ΡΠΈ β ΠΈΠ½Π²Π°ΡΠΈΠ°Π½ΡΠ½Ρ, ΡΠ»ΠΎΠΆΠ½ΡΠ΅ β ΡΡΠ²ΡΡΠ²ΠΈΡΠ΅Π»ΡΠ½Ρ
ΠΡΠΈ ΡΡΡΠ΅ΠΊΡΡ ΠΎΠΏΠΈΡΡΠ²Π°ΡΡΡΡ ΠΊΠ°ΠΊ Π³Π΅ΠΎΠΌΠ΅ΡΡΠΈΡΠ΅ΡΠΊΠΈΠΉ ΡΠ΄Π²ΠΈΠ³ ΠΏΡΠ΅Π΄ΡΡΠ°Π²Π»Π΅Π½ΠΈΠΉ, Π° Π½Π΅ ΠΊΠ°ΠΊ ΡΠ½ΠΈΠ²Π΅ΡΡΠ°Π»ΡΠ½ΠΎΠ΅ ΡΠ»ΡΡΡΠ΅Π½ΠΈΠ΅ ΠΊΠ°ΡΠ΅ΡΡΠ²Π°.
math_subattention (ΡΠ°Π±ΠΎΡΠ°Ρ Π³ΠΈΠΏΠΎΡΠ΅Π·Π°)
Π ΡΠΊΡΠΏΠ΅ΡΠΈΠΌΠ΅Π½ΡΠ°Ρ Π½Π°Π±Π»ΡΠ΄Π°Π΅ΡΡΡ ΡΡΡΠ΅ΠΊΡ, ΡΡΠ»ΠΎΠ²Π½ΠΎ ΠΎΠ±ΠΎΠ·Π½Π°ΡΠ΅Π½Π½ΡΠΉ ΠΊΠ°ΠΊ:
βmath_subattentionβ
ΠΠΎΠ΄ ΡΡΠΈΠΌ ΠΏΠΎΠ΄ΡΠ°Π·ΡΠΌΠ΅Π²Π°Π΅ΡΡΡ:
β’ ΡΠΌΠ΅Π½ΡΡΠ΅Π½ΠΈΠ΅ Π²ΠΊΠ»Π°Π΄Π° ΠΌΠ΅Π»ΠΊΠΈΡ ΠΊΠΎΠΌΠΏΠΎΠ½Π΅Π½Ρ V β’ ΡΡΠΈΠ»Π΅Π½ΠΈΠ΅ Π΄ΠΎΠΌΠΈΠ½ΠΈΡΡΡΡΠΈΡ Π½Π°ΠΏΡΠ°Π²Π»Π΅Π½ΠΈΠΉ residual stream β’ ΠΏΠΎΠ²ΡΡΠ΅Π½Π½Π°Ρ ΠΈΠ½Π΅ΡΡΠΈΡ ΠΏΡΠ΅Π΄ΡΠ΄ΡΡΠ΅Π³ΠΎ ΡΠΎΠΊΠ΅Π½Π° β’ ΡΠ½ΠΈΠΆΠ΅Π½ΠΈΠ΅ ΡΠ°ΡΡΠΎΡΡ ΠΌΠ΅Π»ΠΊΠΈΡ ΠΏΠ΅ΡΠ΅ΠΊΠ»ΡΡΠ΅Π½ΠΈΠΉ Π»ΠΎΠ³ΠΈΡΠΎΠ²
ΠΡΠΎ Π½Π΅ claim ΠΎ Π½ΠΎΠ²ΠΎΠΉ Π°ΡΡ ΠΈΡΠ΅ΠΊΡΡΡΠ΅. ΠΡΠΎ ΡΠ°Π±ΠΎΡΠ°Ρ Π³ΠΈΠΏΠΎΡΠ΅Π·Π° ΠΎ Π΄ΠΈΠ½Π°ΠΌΠΈΠΊΠ΅, Π²ΠΎΠ·Π½ΠΈΠΊΠ°ΡΡΠ΅ΠΉ ΠΏΡΠΈ Q3_K symmetric quantization.
Π’Π΅ΡΠΌΠΈΠ½ ΠΈΡΠΏΠΎΠ»ΡΠ·ΡΠ΅ΡΡΡ ΠΎΠΏΠΈΡΠ°ΡΠ΅Π»ΡΠ½ΠΎ.
ΠΠ΅ΡΠΏΠ»Π΅ΠΊΡΠΈΡ
ΠΠ΅ΡΡΠΈΠΊΠ° ΠΈΠ·ΠΌΠ΅ΡΠ΅Π½Π° Π½Π° wikitext-2-raw-test (full):
| Model | Precision | PPL |
|---|---|---|
| Vikra MixP_4.9b_S | 6.11 BPW | 5.50 Β± 0.03 |
| Baseline BF16 | Full | 6.02 Β± 0.03 |
ΠΠ»Π°Π½Ρ ΡΠ°Π·Π²ΠΈΡΠΈΡ
ΠΠ»Π°Π½ΠΈΡΡΡΡΡΡ ΠΏΠΎΠ΄ΡΠ΅ΠΌΠ΅ΠΉΡΡΠ²Π°:
- MixP β Mixed Precision
- FullP β Full Precision Π²Π΅ΡΡΠΈΠΈ
- HCT β multi-merge ΡΠΊΡΠΏΠ΅ΡΠΈΠΌΠ΅Π½ΡΡ
- S / M / L β Π²Π°ΡΠΈΠ°Π½ΡΡ ΡΠ°ΡΠΏΡΠ΅Π΄Π΅Π»Π΅Π½ΠΈΡ Π±ΠΈΡΠ½ΠΎΡΡΠΈ
ΠΡΠ΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ ΡΠ΅ΠΌΠ΅ΠΉΡΡΠ²Π° Π½Π°Π·ΡΠ²Π°ΡΡΡΡ Vikra. Π Π΅ΠΏΠΎΠ·ΠΈΡΠΎΡΠΈΠΉ β Vikras.
ΠΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΠ΅
llama-cli -m Vikra-MixP_4.9b_S.gguf -ngl 99 -c 4096
llama-server -m Vikra-MixP_4.9b_S.gguf -ngl 99 -c 4096
ΠΠ°ΠΊΠ»ΡΡΠ΅Π½ΠΈΠ΅
Vikras β ΠΈΡΡΠ»Π΅Π΄ΠΎΠ²Π°ΡΠ΅Π»ΡΡΠΊΠΈΠΉ ΠΏΡΠΎΠ΅ΠΊΡ.
ΠΠ½ ΠΈΡΡΠ»Π΅Π΄ΡΠ΅Ρ, ΠΊΠ°ΠΊ ΠΌΠ΅Π½ΡΠ΅ΡΡΡ ΠΏΠΎΠ²Π΅Π΄Π΅Π½ΠΈΠ΅ ΡΡΠ°Π½ΡΡΠΎΡΠΌΠ΅ΡΠ°, Π΅ΡΠ»ΠΈ Π΅Π³ΠΎ:
- ΡΠΆΠΈΠΌΠ°ΡΡ
- ΡΠΌΠ΅ΡΠΈΠ²Π°ΡΡ
- ΠΈΠ·ΠΌΠ΅Π½ΡΡΡ ΡΠΈΡΠ»Π΅Π½Π½ΡΡ Π³Π΅ΠΎΠΌΠ΅ΡΡΠΈΡ
ΠΡΠ»ΠΈ Π²Π°ΠΌ ΠΈΠ½ΡΠ΅ΡΠ΅ΡΠ½Ρ hidden space dynamics / regime sensitivity / anisotropic quantization β Π΄ΠΎΠ±ΡΠΎ ΠΏΠΎΠΆΠ°Π»ΠΎΠ²Π°ΡΡ.
Vikras β Experimental Family of Language Models (EN)
Table of Contents
- Project overview
- Current Release: HCT/YeAM
- HCT (architecture) / YeAM (implementation invariant)
- Previous Release: Vikra MixedPrc (MixP_4.9b_S)
- MixP_4.9b_S: details
- Roadmap
- Usage
- Closing
Project overview
Vikra is an experimental family of language models exploring how:
- representation geometry
- quantization
- hybrid merges
affect transformer numerical dynamics.
The Vikras project is not tied to a single base model or architecture. It is a family of models unified by a numerical invariance philosophy of experimentation.
- Vikra_% β a specific model
- Vikras β the experimental family
- S / M / L β aggressiveness and bit allocation variants
- MixP / FullP / HCT β quantization / merge invariants
Current Release: HCT/YeAM
Releases
- Vikra-HCT-YeAM-PhiMma-1B
- Vikra-HCT-YeAM-LLaGemma-1B
- Vikra-HCT-YeAM-3_3.2_QweLLa-1.7B
- Vikra-HCT-YeAM-Vikhr-NemoGemma-12B_plus_1B
HCT (architecture) / YeAM (implementation invariant)
HCT is an architectural invariant. In English: Heterogeneous Compatibility Transfer β a practical way to assemble compatible checkpoints and derived releases while moving across bases / model families.
YeAM (Yet Another Merge) is an implementation invariant of HCT and a standalone HFβHF merge scheme: it is not βjust another SLERP/DARE/TILESβ and not a cosmetic variant of averaging.
YeAM produces a standard HF output (safetensors + index) and supports:
- direct weight-to-weight merging
- targeted knowledge injection into a chosen model (knowledge distillation mode), aligned across multiple sources
- an additional Attention-layer merge as a second technique on top of YeAM
- merging smaller models into larger ones (scale-up merge) while keeping a compatible HF format
YeAM operates in a real 4D formulation: updates are encoded geometrically and aligned via ray intersections in parameter space. This produces controlled merges that preserve structure instead of collapsing into naive averaging.
Previous Release: Vikra MixedPrc (MixP_4.9b_S)
Short Description
12.25B Mistral-based language model
Hybrid mixed-precision merged GGUF quantization
Experimental anisotropic quantization regime
Full merge version (non-quantized): https://huggingface.co/srs6901/Vikras-MixP/tree/main/Vikra-FullP
GGUF quant: https://huggingface.co/srs6901/Vikras-MixP/blob/main/Vikra-MixP_4.9b_S.gguf
MixP_4.9b_S: details
Architecture (for the MixP release)
| Parameter | Value |
|---|---|
| Architecture | Mistral-based |
| Params | ~12.25B |
| Layers | 40 |
| Hidden size | 5120 |
| FFN size | 14336 |
| Heads | 32 (8 KV heads, GQA) |
| Context | 1,024,000 |
| Vocab | 131,072 (Tekken BPE) |
| RoPE theta | 1,000,000 |
MixP_4.9b_S β Quantization Scheme
A hybrid mixed-precision scheme with per-tensor type allocation.
| Tensor group | Quant type | BPW |
|---|---|---|
| token_embd, output | BF16 | 16 |
| attn_norm, ffn_norm, output_norm | F32 | 32 |
| attn_q | Q4_K | 4.5 |
| attn_k | Q5_K | 5.5 |
| attn_v | Q3_K | 3.44 |
| attn_output | Q4_K | 4.5 |
| ffn_gate | Q3_K | 3.44 |
| ffn_up | Q5_K | 5.5 |
| ffn_down | Q5_K / Q6_K | 5.5β6.56 |
Totals:
- Quantized layers only: ~4.89 BPW
- Full model average: ~6.11 BPW
- File size: ~8.71 GB
Core idea of MixP
MixP is not βcompress everything equallyβ.
It is anisotropic quantization of information channels:
- Q/K remain in higher precision
- V and gate are intentionally quantized down to Q3_K
- norms and the output layer remain in higher precision
This redistribution changes the numerical dynamics of the model:
- increased structural sparsification
- shifts in hidden norm distribution
- changes in logit entropy
- regime sensitivity
This is not a new architecture. It is a modification of the numerical geometry of an existing one.
Observed effects
- preservation of top-1 predictions on simple tasks
- increased entropy without collapse of maximum probability
- expansion of hidden norms on complex tasks
- mode bifurcation: simple tasks β invariant, complex tasks sensitive
These effects are interpreted as a geometric shift of representations rather than a universal quality improvement.
math_subattention (working hypothesis)
In experiments, an effect informally referred to as:
βmath_subattentionβ
This describes:
- reduced contribution of small V components
- dominance of stronger residual directions
- increased inertia from previous token state
- reduced frequency of small logit switching
This is not an architectural claim. It is a working hypothesis of dynamics emerging from Q3_K symmetric quantization.
The term is used descriptively.
Perplexity
Measured on wikitext-2-raw-test (full):
| Model | Precision | PPL |
|---|---|---|
| Vikra MixP_4.9b_S | 6.11 BPW | 5.50 Β± 0.03 |
| Baseline BF16 | Full | 6.02 Β± 0.03 |
Roadmap
Planned subfamilies:
- MixP β Mixed Precision
- FullP β Full Precision variants
- HCT β multi-merge experiments
- S / M / L β different bit allocation regimes
All models in the family are called Vikra. The repository is Vikras.
Usage
llama-cli -m Vikra-MixP_4.9b_S.gguf -ngl 99 -c 4096
llama-server -m Vikra-MixP_4.9b_S.gguf -ngl 99 -c 4096
Closing
Vikras is a research project.
It explores how transformer behavior changes when we:
- compress
- merge
- alter numerical geometry
If you are interested in hidden space dynamics / regime sensitivity / anisotropic quantization β welcome.
- Downloads last month
- 72