Vikras β€” Experimental Family of Language Models

EN below

Π‘ΠΎΠ΄Π΅Ρ€ΠΆΠ°Π½ΠΈΠ΅


ΠšΠΎΡ€ΠΎΡ‚ΠΊΠΎ ΠΎ ΠΏΡ€ΠΎΠ΅ΠΊΡ‚Π΅

Vikra β€” ΡΠΊΡΠΏΠ΅Ρ€ΠΈΠΌΠ΅Π½Ρ‚Π°Π»ΡŒΠ½ΠΎΠ΅ сСмСйство языковых ΠΌΠΎΠ΄Π΅Π»Π΅ΠΉ, ΠΈΡΡΠ»Π΅Π΄ΡƒΡŽΡ‰Π΅Π΅ влияниС:

  • Π³Π΅ΠΎΠΌΠ΅Ρ‚Ρ€ΠΈΠΈ прСдставлСний
  • квантования
  • Π³ΠΈΠ±Ρ€ΠΈΠ΄Π½Ρ‹Ρ… ΠΌΠ΅Ρ€Π΄ΠΆΠ΅ΠΉ

Π½Π° Ρ‡ΠΈΡΠ»Π΅Π½Π½ΡƒΡŽ Π΄ΠΈΠ½Π°ΠΌΠΈΠΊΡƒ трансформСров.

ΠŸΡ€ΠΎΠ΅ΠΊΡ‚ Vikras Π½Π΅ ограничиваСтся ΠΎΠ΄Π½ΠΎΠΉ Π±Π°Π·ΠΎΠΉ ΠΈΠ»ΠΈ ΠΎΠ΄Π½ΠΎΠΉ Π°Ρ€Ρ…ΠΈΡ‚Π΅ΠΊΡ‚ΡƒΡ€ΠΎΠΉ: это сСмСйство ΠΌΠΎΠ΄Π΅Π»Π΅ΠΉ, ΠΎΠ±ΡŠΠ΅Π΄ΠΈΠ½Ρ‘Π½Π½Ρ‹Ρ… ΠΈΠ΄Π΅Π΅ΠΉ числСнной инвариантности экспСримСнта.

  • Vikra_% β€” имя ΠΊΠΎΠ½ΠΊΡ€Π΅Ρ‚Π½ΠΎΠΉ ΠΌΠΎΠ΄Π΅Π»ΠΈ
  • Vikras β€” сСмСйство экспСримСнтов
  • S / M / L β€” ΡΡ‚Π΅ΠΏΠ΅Π½ΡŒ агрСссивности ΠΈ распрСдСлСния битности
  • MixP / FullP / HCT β€” схСмы ΠΈ ΠΈΠ½Π²Π°Ρ€ΠΈΠ°Π½Ρ‚Ρ‹ квантования/ΠΌΠ΅Ρ€Π΄ΠΆΠ΅ΠΉ

Π’Π΅ΠΊΡƒΡ‰ΠΈΠΉ Ρ€Π΅Π»ΠΈΠ·: HCT/YeAM

Π Π΅Π»ΠΈΠ·Ρ‹


HCT (Π°Ρ€Ρ…ΠΈΡ‚Π΅ΠΊΡ‚ΡƒΡ€Π°) / YeAM (ΠΈΠ½Π²Π°Ρ€ΠΈΠ°Π½Ρ‚ Ρ€Π΅Π°Π»ΠΈΠ·Π°Ρ†ΠΈΠΈ)

HCT β€” Π°Ρ€Ρ…ΠΈΡ‚Π΅ΠΊΡ‚ΡƒΡ€Π½Ρ‹ΠΉ ΠΈΠ½Π²Π°Ρ€ΠΈΠ°Π½Ρ‚: практичСский способ ΡΠΎΠ±ΠΈΡ€Π°Ρ‚ΡŒ совмСстимыС ΠΌΠΎΠ΄Π΅Π»ΠΈ ΠΈ ΠΏΡ€ΠΎΠΈΠ·Π²ΠΎΠ΄Π½Ρ‹Π΅ Ρ€Π΅Π»ΠΈΠ·Ρ‹ ΠΏΡ€ΠΈ пСрСносС ΠΌΠ΅ΠΆΠ΄Ρƒ Π±Π°Π·Π°ΠΌΠΈ/сСмСйствами.

YeAM (Yet Another Merge) β€” ΠΈΠ½Π²Π°Ρ€ΠΈΠ°Π½Ρ‚ Ρ€Π΅Π°Π»ΠΈΠ·Π°Ρ†ΠΈΠΈ HCT ΠΈ ΡΠ°ΠΌΠΎΡΡ‚ΠΎΡΡ‚Π΅Π»ΡŒΠ½Π°Ρ схСма ΠΌΠ΅Ρ€Π΄ΠΆΠ° HFβ†’HF: это Π½Π΅ Β«Π΅Ρ‰Ρ‘ ΠΎΠ΄ΠΈΠ½ SLERP/DARE/TILESΒ» ΠΈ Π½Π΅ космСтичСская вариация усрСднСния.

YeAM Π²Ρ‹Π΄Π°Ρ‘Ρ‚ стандартный HF-Ρ€Π΅Π·ΡƒΠ»ΡŒΡ‚Π°Ρ‚ (safetensors + index) ΠΈ ΠΏΠΎΠ΄Π΄Π΅Ρ€ΠΆΠΈΠ²Π°Π΅Ρ‚:

  • прямой weight-to-weight ΠΌΠ΅Ρ€Π΄ΠΆ
  • Π½Π°ΠΏΡ€Π°Π²Π»Π΅Π½Π½ΠΎΠ΅ Π΄ΠΎΠ±Π°Π²Π»Π΅Π½ΠΈΠ΅ Π·Π½Π°Π½ΠΈΠΉ Π² Π²Ρ‹Π±Ρ€Π°Π½Π½ΡƒΡŽ модСль (knowledge distillation / knowledge injection), согласованноС ΠΏΠΎ нСскольким источникам
  • Π΄ΠΎΠΏΠΎΠ»Π½ΠΈΡ‚Π΅Π»ΡŒΠ½Ρ‹ΠΉ ΠΌΠ΅Ρ€Π΄ΠΆ Attention-слоёв ΠΊΠ°ΠΊ ΠΎΡ‚Π΄Π΅Π»ΡŒΠ½ΡƒΡŽ Ρ‚Π΅Ρ…Π½ΠΈΠΊΡƒ ΠΏΠΎΠ²Π΅Ρ€Ρ… YeAM
  • ΠΌΠ΅Ρ€Π΄ΠΆ ΠΌΠ΅Π½ΡŒΡˆΠΈΡ… ΠΌΠΎΠ΄Π΅Π»Π΅ΠΉ Π² Π±ΠΎΠ»Π΅Π΅ ΠΊΡ€ΡƒΠΏΠ½Ρ‹Π΅ (scale-up merge) ΠΏΡ€ΠΈ сохранСнии совмСстимого HF-Ρ„ΠΎΡ€ΠΌΠ°Ρ‚Π°

ΠœΠ°Ρ‚Π΅ΠΌΠ°Ρ‚ΠΈΡ‡Π΅ΡΠΊΠΈ YeAM Ρ€Π°Π±ΠΎΡ‚Π°Π΅Ρ‚ Π² Ρ€Π΅Π°Π»ΡŒΠ½ΠΎΠΉ 4D-постановкС: обновлСния ΠΊΠΎΠ΄ΠΈΡ€ΡƒΡŽΡ‚ΡΡ гСомСтричСски ΠΈ ΡΠΎΠ³Π»Π°ΡΡƒΡŽΡ‚ΡΡ Ρ‡Π΅Ρ€Π΅Π· пСрСсСчСния Π»ΡƒΡ‡Π΅ΠΉ Π² пространствС ΠΏΠ°Ρ€Π°ΠΌΠ΅Ρ‚Ρ€ΠΎΠ². Π­Ρ‚ΠΎ Π΄Π°Ρ‘Ρ‚ управляСмый ΠΌΠ΅Ρ€Π΄ΠΆ с сохранСниСм структуры ΠΈ Π±Π΅Π· выроТдСния Π² Π½Π°ΠΈΠ²Π½ΠΎΠ΅ усрСднСниС.


ΠŸΡ€Π΅Π΄Ρ‹Π΄ΡƒΡ‰ΠΈΠΉ Ρ€Π΅Π»ΠΈΠ·: Vikra MixedPrc (MixP_4.9b_S)

ΠšΡ€Π°Ρ‚ΠΊΠΎΠ΅ описаниС

12.25B Mistral-based language model
Hybrid mixed-precision merged GGUF quantization
Π­ΠΊΡΠΏΠ΅Ρ€ΠΈΠΌΠ΅Π½Ρ‚Π°Π»ΡŒΠ½Ρ‹ΠΉ Ρ€Π΅ΠΆΠΈΠΌ Π°Π½ΠΈΠ·ΠΎΡ‚Ρ€ΠΎΠΏΠ½ΠΎΠ³ΠΎ квантования

Полная вСрсия ΠΌΠ΅Ρ€Π΄ΠΆΠ° (Π±Π΅Π· квантования): https://huggingface.co/srs6901/Vikras-MixP/tree/main/Vikra-FullP

GGUF-ΠΊΠ²Π°Π½Ρ‚: https://huggingface.co/srs6901/Vikras-MixP/blob/main/Vikra-MixP_4.9b_S.gguf


MixP_4.9b_S: Π΄Π΅Ρ‚Π°Π»ΠΈ

АрхитСктура (для MixP Ρ€Π΅Π»ΠΈΠ·Π°)

ΠŸΠ°Ρ€Π°ΠΌΠ΅Ρ‚Ρ€ Π—Π½Π°Ρ‡Π΅Π½ΠΈΠ΅
Architecture Mistral-based
Params ~12.25B
Layers 40
Hidden size 5120
FFN size 14336
Heads 32 (8 KV heads, GQA)
Context 1,024,000
Vocab 131,072 (Tekken BPE)
RoPE theta 1,000,000

MixP_4.9b_S β€” схСма квантования

Гибридная mixed precision схСма с ΠΏΠΎΠΊΠΎΠΌΠΏΠΎΠ½Π΅Π½Ρ‚Π½Ρ‹ΠΌ распрСдСлСниСм Ρ‚ΠΈΠΏΠΎΠ².

Tensor group Quant type BPW
token_embd, output BF16 16
attn_norm, ffn_norm, output_norm F32 32
attn_q Q4_K 4.5
attn_k Q5_K 5.5
attn_v Q3_K 3.44
attn_output Q4_K 4.5
ffn_gate Q3_K 3.44
ffn_up Q5_K 5.5
ffn_down Q5_K / Q6_K 5.5–6.56

Π˜Ρ‚ΠΎΠ³ΠΎ:

  • Quantized layers only: ~4.89 BPW
  • Full model average: ~6.11 BPW
  • File size: ~8.71 GB

ΠšΠ»ΡŽΡ‡Π΅Π²Π°Ρ идСя MixP

MixP β€” это Π½Π΅ Β«ΡΠΆΠ°Ρ‚ΡŒ всё ΠΎΠ΄ΠΈΠ½Π°ΠΊΠΎΠ²ΠΎΒ».

Π­Ρ‚ΠΎ Π°Π½ΠΈΠ·ΠΎΡ‚Ρ€ΠΎΠΏΠ½ΠΎΠ΅ ΠΊΠ²Π°Π½Ρ‚ΠΎΠ²Π°Π½ΠΈΠ΅ ΠΈΠ½Ρ„ΠΎΡ€ΠΌΠ°Ρ†ΠΈΠΎΠ½Π½Ρ‹Ρ… ΠΊΠ°Π½Π°Π»ΠΎΠ²:

β€’ Q/K ΡΠΎΡ…Ρ€Π°Π½ΡΡŽΡ‚ΡΡ Π² Π±ΠΎΠ»Π΅Π΅ высокой точности β€’ V ΠΈ gate Π½Π°ΠΌΠ΅Ρ€Π΅Π½Π½ΠΎ ΠΊΠ²Π°Π½Ρ‚ΠΎΠ²Π°Π½Ρ‹ Π΄ΠΎ Q3_K β€’ Нормы ΠΈ Π²Ρ‹Ρ…ΠΎΠ΄Π½ΠΎΠΉ слой ΠΎΡΡ‚Π°ΡŽΡ‚ΡΡ Π² высокой точности

Π’Π°ΠΊΠΎΠ΅ распрСдСлСниС измСняСт Ρ‡ΠΈΡΠ»Π΅Π½Π½ΡƒΡŽ Π΄ΠΈΠ½Π°ΠΌΠΈΠΊΡƒ ΠΌΠΎΠ΄Π΅Π»ΠΈ:

β€’ усиливаСтся структурная sparsification β€’ мСняСтся распрСдСлСниС Π½ΠΎΡ€ΠΌ скрытых прСдставлСний β€’ мСняСтся энтропия Π»ΠΎΠ³ΠΈΡ‚ΠΎΠ² β€’ появляСтся рСТимная Ρ‡ΡƒΠ²ΡΡ‚Π²ΠΈΡ‚Π΅Π»ΡŒΠ½ΠΎΡΡ‚ΡŒ

Π­Ρ‚ΠΎ Π½Π΅ новая Π°Ρ€Ρ…ΠΈΡ‚Π΅ΠΊΡ‚ΡƒΡ€Π°. Π­Ρ‚ΠΎ ΠΈΠ·ΠΌΠ΅Π½Π΅Π½ΠΈΠ΅ числСнной Π³Π΅ΠΎΠΌΠ΅Ρ‚Ρ€ΠΈΠΈ ΡΡƒΡ‰Π΅ΡΡ‚Π²ΡƒΡŽΡ‰Π΅ΠΉ.

ΠΠ°Π±Π»ΡŽΠ΄Π°Π΅ΠΌΡ‹Π΅ эффСкты

  • сохранСниС top-1 прСдсказаний Π½Π° простых Π·Π°Π΄Π°Ρ‡Π°Ρ…
  • рост entropy Π±Π΅Π· Ρ€Π°Π·Ρ€ΡƒΡˆΠ΅Π½ΠΈΡ максимальной вСроятности
  • Ρ€Π°ΡΡˆΠΈΡ€Π΅Π½ΠΈΠ΅ hidden norm Π½Π° слоТных Π·Π°Π΄Π°Ρ‡Π°Ρ…
  • бифуркация Ρ€Π΅ΠΆΠΈΠΌΠΎΠ²: простыС Π·Π°Π΄Π°Ρ‡ΠΈ β‰ˆ ΠΈΠ½Π²Π°Ρ€ΠΈΠ°Π½Ρ‚Π½Ρ‹, слоТныС β€” Ρ‡ΡƒΠ²ΡΡ‚Π²ΠΈΡ‚Π΅Π»ΡŒΠ½Ρ‹

Π­Ρ‚ΠΈ эффСкты ΠΎΠΏΠΈΡΡ‹Π²Π°ΡŽΡ‚ΡΡ ΠΊΠ°ΠΊ гСомСтричСский сдвиг прСдставлСний, Π° Π½Π΅ ΠΊΠ°ΠΊ ΡƒΠ½ΠΈΠ²Π΅Ρ€ΡΠ°Π»ΡŒΠ½ΠΎΠ΅ ΡƒΠ»ΡƒΡ‡ΡˆΠ΅Π½ΠΈΠ΅ качСства.

math_subattention (рабочая Π³ΠΈΠΏΠΎΡ‚Π΅Π·Π°)

Π’ экспСримСнтах Π½Π°Π±Π»ΡŽΠ΄Π°Π΅Ρ‚ΡΡ эффСкт, условно ΠΎΠ±ΠΎΠ·Π½Π°Ρ‡Π΅Π½Π½Ρ‹ΠΉ ΠΊΠ°ΠΊ:

β€œmath_subattention”

Под этим подразумСваСтся:

β€’ ΡƒΠΌΠ΅Π½ΡŒΡˆΠ΅Π½ΠΈΠ΅ Π²ΠΊΠ»Π°Π΄Π° ΠΌΠ΅Π»ΠΊΠΈΡ… ΠΊΠΎΠΌΠΏΠΎΠ½Π΅Π½Ρ‚ V β€’ усилСниС Π΄ΠΎΠΌΠΈΠ½ΠΈΡ€ΡƒΡŽΡ‰ΠΈΡ… Π½Π°ΠΏΡ€Π°Π²Π»Π΅Π½ΠΈΠΉ residual stream β€’ ΠΏΠΎΠ²Ρ‹ΡˆΠ΅Π½Π½Π°Ρ инСрция ΠΏΡ€Π΅Π΄Ρ‹Π΄ΡƒΡ‰Π΅Π³ΠΎ Ρ‚ΠΎΠΊΠ΅Π½Π° β€’ сниТСниС частоты ΠΌΠ΅Π»ΠΊΠΈΡ… ΠΏΠ΅Ρ€Π΅ΠΊΠ»ΡŽΡ‡Π΅Π½ΠΈΠΉ Π»ΠΎΠ³ΠΈΡ‚ΠΎΠ²

Π­Ρ‚ΠΎ Π½Π΅ claim ΠΎ Π½ΠΎΠ²ΠΎΠΉ Π°Ρ€Ρ…ΠΈΡ‚Π΅ΠΊΡ‚ΡƒΡ€Π΅. Π­Ρ‚ΠΎ рабочая Π³ΠΈΠΏΠΎΡ‚Π΅Π·Π° ΠΎ Π΄ΠΈΠ½Π°ΠΌΠΈΠΊΠ΅, Π²ΠΎΠ·Π½ΠΈΠΊΠ°ΡŽΡ‰Π΅ΠΉ ΠΏΡ€ΠΈ Q3_K symmetric quantization.

Π’Π΅Ρ€ΠΌΠΈΠ½ ΠΈΡΠΏΠΎΠ»ΡŒΠ·ΡƒΠ΅Ρ‚ΡΡ ΠΎΠΏΠΈΡΠ°Ρ‚Π΅Π»ΡŒΠ½ΠΎ.

ΠŸΠ΅Ρ€ΠΏΠ»Π΅ΠΊΡΠΈΡ

ΠœΠ΅Ρ‚Ρ€ΠΈΠΊΠ° ΠΈΠ·ΠΌΠ΅Ρ€Π΅Π½Π° Π½Π° wikitext-2-raw-test (full):

Model Precision PPL
Vikra MixP_4.9b_S 6.11 BPW 5.50 Β± 0.03
Baseline BF16 Full 6.02 Β± 0.03

ΠŸΠ»Π°Π½Ρ‹ развития

ΠŸΠ»Π°Π½ΠΈΡ€ΡƒΡŽΡ‚ΡΡ подсСмСйства:

  • MixP β€” Mixed Precision
  • FullP β€” Full Precision вСрсии
  • HCT β€” multi-merge экспСримСнты
  • S / M / L β€” Π²Π°Ρ€ΠΈΠ°Π½Ρ‚Ρ‹ распрСдСлСния битности

ВсС ΠΌΠΎΠ΄Π΅Π»ΠΈ сСмСйства Π½Π°Π·Ρ‹Π²Π°ΡŽΡ‚ΡΡ Vikra. Π Π΅ΠΏΠΎΠ·ΠΈΡ‚ΠΎΡ€ΠΈΠΉ β€” Vikras.


ИспользованиС

llama-cli -m Vikra-MixP_4.9b_S.gguf -ngl 99 -c 4096
llama-server -m Vikra-MixP_4.9b_S.gguf -ngl 99 -c 4096

Π—Π°ΠΊΠ»ΡŽΡ‡Π΅Π½ΠΈΠ΅

Vikras β€” ΠΈΡΡΠ»Π΅Π΄ΠΎΠ²Π°Ρ‚Π΅Π»ΡŒΡΠΊΠΈΠΉ ΠΏΡ€ΠΎΠ΅ΠΊΡ‚.

Он исслСдуСт, ΠΊΠ°ΠΊ мСняСтся ΠΏΠΎΠ²Π΅Π΄Π΅Π½ΠΈΠ΅ трансформСра, Ссли Π΅Π³ΠΎ:

  • ΡΠΆΠΈΠΌΠ°Ρ‚ΡŒ
  • ΡΠΌΠ΅ΡˆΠΈΠ²Π°Ρ‚ΡŒ
  • ΠΈΠ·ΠΌΠ΅Π½ΡΡ‚ΡŒ Ρ‡ΠΈΡΠ»Π΅Π½Π½ΡƒΡŽ Π³Π΅ΠΎΠΌΠ΅Ρ‚Ρ€ΠΈΡŽ

Если Π²Π°ΠΌ интСрСсны hidden space dynamics / regime sensitivity / anisotropic quantization β€” Π΄ΠΎΠ±Ρ€ΠΎ ΠΏΠΎΠΆΠ°Π»ΠΎΠ²Π°Ρ‚ΡŒ.


Vikras β€” Experimental Family of Language Models (EN)

Table of Contents


Project overview

Vikra is an experimental family of language models exploring how:

  • representation geometry
  • quantization
  • hybrid merges

affect transformer numerical dynamics.

The Vikras project is not tied to a single base model or architecture. It is a family of models unified by a numerical invariance philosophy of experimentation.

  • Vikra_% β€” a specific model
  • Vikras β€” the experimental family
  • S / M / L β€” aggressiveness and bit allocation variants
  • MixP / FullP / HCT β€” quantization / merge invariants

Current Release: HCT/YeAM

Releases


HCT (architecture) / YeAM (implementation invariant)

HCT is an architectural invariant. In English: Heterogeneous Compatibility Transfer β€” a practical way to assemble compatible checkpoints and derived releases while moving across bases / model families.

YeAM (Yet Another Merge) is an implementation invariant of HCT and a standalone HFβ†’HF merge scheme: it is not β€œjust another SLERP/DARE/TILES” and not a cosmetic variant of averaging.

YeAM produces a standard HF output (safetensors + index) and supports:

  • direct weight-to-weight merging
  • targeted knowledge injection into a chosen model (knowledge distillation mode), aligned across multiple sources
  • an additional Attention-layer merge as a second technique on top of YeAM
  • merging smaller models into larger ones (scale-up merge) while keeping a compatible HF format

YeAM operates in a real 4D formulation: updates are encoded geometrically and aligned via ray intersections in parameter space. This produces controlled merges that preserve structure instead of collapsing into naive averaging.


Previous Release: Vikra MixedPrc (MixP_4.9b_S)

Short Description

12.25B Mistral-based language model
Hybrid mixed-precision merged GGUF quantization
Experimental anisotropic quantization regime

Full merge version (non-quantized): https://huggingface.co/srs6901/Vikras-MixP/tree/main/Vikra-FullP

GGUF quant: https://huggingface.co/srs6901/Vikras-MixP/blob/main/Vikra-MixP_4.9b_S.gguf


MixP_4.9b_S: details

Architecture (for the MixP release)

Parameter Value
Architecture Mistral-based
Params ~12.25B
Layers 40
Hidden size 5120
FFN size 14336
Heads 32 (8 KV heads, GQA)
Context 1,024,000
Vocab 131,072 (Tekken BPE)
RoPE theta 1,000,000

MixP_4.9b_S β€” Quantization Scheme

A hybrid mixed-precision scheme with per-tensor type allocation.

Tensor group Quant type BPW
token_embd, output BF16 16
attn_norm, ffn_norm, output_norm F32 32
attn_q Q4_K 4.5
attn_k Q5_K 5.5
attn_v Q3_K 3.44
attn_output Q4_K 4.5
ffn_gate Q3_K 3.44
ffn_up Q5_K 5.5
ffn_down Q5_K / Q6_K 5.5–6.56

Totals:

  • Quantized layers only: ~4.89 BPW
  • Full model average: ~6.11 BPW
  • File size: ~8.71 GB

Core idea of MixP

MixP is not β€œcompress everything equally”.

It is anisotropic quantization of information channels:

  • Q/K remain in higher precision
  • V and gate are intentionally quantized down to Q3_K
  • norms and the output layer remain in higher precision

This redistribution changes the numerical dynamics of the model:

  • increased structural sparsification
  • shifts in hidden norm distribution
  • changes in logit entropy
  • regime sensitivity

This is not a new architecture. It is a modification of the numerical geometry of an existing one.

Observed effects

  • preservation of top-1 predictions on simple tasks
  • increased entropy without collapse of maximum probability
  • expansion of hidden norms on complex tasks
  • mode bifurcation: simple tasks β‰ˆ invariant, complex tasks sensitive

These effects are interpreted as a geometric shift of representations rather than a universal quality improvement.

math_subattention (working hypothesis)

In experiments, an effect informally referred to as:

β€œmath_subattention”

This describes:

  • reduced contribution of small V components
  • dominance of stronger residual directions
  • increased inertia from previous token state
  • reduced frequency of small logit switching

This is not an architectural claim. It is a working hypothesis of dynamics emerging from Q3_K symmetric quantization.

The term is used descriptively.

Perplexity

Measured on wikitext-2-raw-test (full):

Model Precision PPL
Vikra MixP_4.9b_S 6.11 BPW 5.50 Β± 0.03
Baseline BF16 Full 6.02 Β± 0.03

Roadmap

Planned subfamilies:

  • MixP β€” Mixed Precision
  • FullP β€” Full Precision variants
  • HCT β€” multi-merge experiments
  • S / M / L β€” different bit allocation regimes

All models in the family are called Vikra. The repository is Vikras.


Usage

llama-cli -m Vikra-MixP_4.9b_S.gguf -ngl 99 -c 4096
llama-server -m Vikra-MixP_4.9b_S.gguf -ngl 99 -c 4096

Closing

Vikras is a research project.

It explores how transformer behavior changes when we:

  • compress
  • merge
  • alter numerical geometry

If you are interested in hidden space dynamics / regime sensitivity / anisotropic quantization β€” welcome.

Downloads last month
72
GGUF
Model size
2B params
Architecture
qwen3
Hardware compatibility
Log In to add your hardware

4-bit

6-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ 1 Ask for provider support

Collection including srs6901/Vikras-MixP