Add pipeline tag, library name and GitHub link (#1)
Browse files- Add pipeline tag, library name and GitHub link (ae11af463d2597f4095becb55c6a57b937ea0972)
Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -1,14 +1,19 @@
|
|
| 1 |
---
|
| 2 |
-
license: mit
|
| 3 |
language:
|
| 4 |
- en
|
|
|
|
| 5 |
metrics:
|
| 6 |
- accuracy
|
|
|
|
|
|
|
| 7 |
---
|
|
|
|
| 8 |
# Polar Sparsity: High Throughput Batched LLM Inferencing with Scalable Contextual Sparsity
|
| 9 |
|
| 10 |
Polar Sparsity is a framework for efficient sparse inferencing in large language models (LLMs), leveraging custom Triton kernels and learned routers for selective activation of MLP neurons and attention heads. This repository provides tools for data collection, router training, benchmarking, and end-to-end sparse generation.
|
| 11 |
|
|
|
|
|
|
|
| 12 |
---
|
| 13 |
|
| 14 |
## ⚠️ Requirements
|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
language:
|
| 3 |
- en
|
| 4 |
+
license: mit
|
| 5 |
metrics:
|
| 6 |
- accuracy
|
| 7 |
+
pipeline_tag: text-generation
|
| 8 |
+
library_name: transformers
|
| 9 |
---
|
| 10 |
+
|
| 11 |
# Polar Sparsity: High Throughput Batched LLM Inferencing with Scalable Contextual Sparsity
|
| 12 |
|
| 13 |
Polar Sparsity is a framework for efficient sparse inferencing in large language models (LLMs), leveraging custom Triton kernels and learned routers for selective activation of MLP neurons and attention heads. This repository provides tools for data collection, router training, benchmarking, and end-to-end sparse generation.
|
| 14 |
|
| 15 |
+
Code: https://github.com/susavlsh10/Polar-Sparsity
|
| 16 |
+
|
| 17 |
---
|
| 18 |
|
| 19 |
## ⚠️ Requirements
|