AI & ML interests
Spiking Neural Networks (SNN) for efficient LLM distillation. We build 1B-parameter student models distilled from Llama 3, Qwen 3.6, and CodeLlama using biologically-inspired spike-based activation. Focus: ultra-low latency inference on consumer GPUs (RTX 5070 Ti), neuromorphic computing, and multi-teacher ensemble distillation. Open-source models for code generation and general reasoning.
Organization Card
⚡ VERTEX-1B-SNN
1B Parameter Spiking Neural Network distilled from Llama 3 8B, Qwen 3.6 27B, CodeLlama 7B, and Llama 2 7B.
Built by Mohamed Amine, CEO of VERTEX AI, Morocco.
Model Architecture
| Component | Specification |
|---|---|
| Parameters | 1B |
| Architecture | Spiking Transformer (SNN) |
| Hidden Dim | 512 |
| Layers | 8 |
| Attention Heads | 8 |
| Spike Time Steps | 12 |
| Vocabulary | 128,256 (Llama 3) |
| Precision | BF16 |
Distillation
- Hard Loss: Ground-truth next-token prediction
- Soft Loss: KL divergence from teacher ensemble
- Consistency: Entropy regularization
- Router Loss: Task-conditioned specialization (general/code/reasoning)
Teachers
| Model | Weight | Specialization |
|---|---|---|
| Llama 3 8B | 30% | General + Reasoning |
| Qwen 3.6 27B | 25% | Code + Agentic |
| Llama 2 7B | 25% | General |
| CodeLlama 7B | 20% | Pure Code |
Hardware
Optimized for NVIDIA RTX 5070 Ti (16GB VRAM) with sequential teacher loading, 4-bit quantization, and LoRA adapters.
Citation
@misc{vertex-1b-snn-2026,
title={VERTEX-1B-SNN: Efficient Spiking Neural Network Distillation for Consumer GPUs},
author={Mohamed Amine},
year={2026},
organization={VERTEX AI},
url={https://huggingface.co/vertex-snn}
}
Links
models 0
None public yet
datasets 0
None public yet