Spaces:

vertex-snn
/

README

Sleeping

App Files Files Community

README / README.md

aminevertex

Update README.md

992a107 verified 3 months ago

preview code

Raw

History Blame Contribute Delete

1.81 kB

A newer version of the Gradio SDK is available: 6.24.0

Upgrade

metadata

title: VERTEX-1B-SNN Demo
emoji: ⚡
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 6.14.0
app_file: app.py
pinned: false
license: apache-2.0

⚡ VERTEX-1B-SNN

1B Parameter Spiking Neural Network distilled from Llama 3 8B, Qwen 3.6 27B, CodeLlama 7B, and Llama 2 7B.

Built by Mohamed Amine, CEO of VERTEX AI, Morocco.

Model Architecture

Component	Specification
Parameters	1B
Architecture	Spiking Transformer (SNN)
Hidden Dim	512
Layers	8
Attention Heads	8
Spike Time Steps	12
Vocabulary	128,256 (Llama 3)
Precision	BF16

Distillation

Hard Loss: Ground-truth next-token prediction
Soft Loss: KL divergence from teacher ensemble
Consistency: Entropy regularization
Router Loss: Task-conditioned specialization (general/code/reasoning)

Teachers

Model	Weight	Specialization
Llama 3 8B	30%	General + Reasoning
Qwen 3.6 27B	25%	Code + Agentic
Llama 2 7B	25%	General
CodeLlama 7B	20%	Pure Code

Hardware

Optimized for NVIDIA RTX 5070 Ti (16GB VRAM) with sequential teacher loading, 4-bit quantization, and LoRA adapters.

Citation

@misc{vertex-1b-snn-2026,
  title={VERTEX-1B-SNN: Efficient Spiking Neural Network Distillation for Consumer GPUs},
  author={Mohamed Amine},
  year={2026},
  organization={VERTEX AI},
  url={https://huggingface.co/vertex-snn}
}