Add pipeline tag, license and paper link

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +7 -5
README.md CHANGED
@@ -1,13 +1,15 @@
1
  ---
2
- tags:
3
- - sketchtune
4
- - sketch to adapt
5
  library_name: transformers
 
 
 
 
 
6
  ---
7
 
8
  # Base Models for Fine-tuning in *(ICML 2025) Sketch to Adapt: Fine-Tunable Sketches for Efficient LLM Adaptation*
9
 
10
- This repository hosts the compressed base models used in the fine-tuning experiments from our ICML 2025 paper: **Sketch to Adapt: Fine-Tunable Sketches for Efficient LLM Adaptation**. The available models and formats are as follows.
11
 
12
  | Model | Bits | GPR (Groups Per Row) |
13
  |---------------|--------|--------------------|
@@ -34,7 +36,7 @@ SketchTune is a novel method for adapting large language models (LLMs) that focu
34
  * Even with base models that are **2.6–3.5× smaller**, SketchTune **outperforms LoRA, DoRA, and S2FT** on commonsense and math reasoning benchmarks.
35
  * On the GSM8K math dataset, SketchTune achieves a **14.48% higher accuracy than LoftQ**, while training **7.3× fewer parameters**.
36
 
37
- For a deep dive into how sketching works, including math details and extensive test results, check out our full paper: [https://arxiv.org/abs/2410.06364](https://arxiv.org/abs/2410.06364).
38
 
39
  ### Citation
40
 
 
1
  ---
 
 
 
2
  library_name: transformers
3
+ tags:
4
+ - sketchtune
5
+ - sketch to adapt
6
+ pipeline_tag: text-generation
7
+ license: apache-2.0
8
  ---
9
 
10
  # Base Models for Fine-tuning in *(ICML 2025) Sketch to Adapt: Fine-Tunable Sketches for Efficient LLM Adaptation*
11
 
12
+ This repository hosts the compressed base models used in the fine-tuning experiments from our ICML 2025 paper: **[Sketch to Adapt: Fine-Tunable Sketches for Efficient LLM Adaptation](https://huggingface.co/papers/2410.06364)**. The available models and formats are as follows.
13
 
14
  | Model | Bits | GPR (Groups Per Row) |
15
  |---------------|--------|--------------------|
 
36
  * Even with base models that are **2.6–3.5× smaller**, SketchTune **outperforms LoRA, DoRA, and S2FT** on commonsense and math reasoning benchmarks.
37
  * On the GSM8K math dataset, SketchTune achieves a **14.48% higher accuracy than LoftQ**, while training **7.3× fewer parameters**.
38
 
39
+ For a deep dive into how sketching works, including math details and extensive test results, check out our full paper: [https://huggingface.co/papers/2410.06364](https://huggingface.co/papers/2410.06364).
40
 
41
  ### Citation
42