CompactAI commited on
Commit
c847234
Β·
verified Β·
1 Parent(s): 2aa0313

Upload folder using huggingface_hub

Browse files
Files changed (4) hide show
  1. README.md +14 -33
  2. comparison_graph.png +0 -0
  3. model.safetensors +1 -1
  4. tokenizer.json +1 -1
README.md CHANGED
@@ -5,7 +5,6 @@ tags:
5
  - html
6
  - optimized
7
  - wanda
8
- - activation-pruning
9
  base_model: Qwen/Qwen3-0.6B
10
  pipeline_tag: text-generation
11
  ---
@@ -14,35 +13,28 @@ pipeline_tag: text-generation
14
 
15
  > 🎯 **HTML-optimized** | πŸ“¦ **Safe** pruning | ⚑ **1% weights pruned**
16
 
17
- This model is a **conservatively pruned** version of [Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B), specialized for **HTML** tasks using activation-aware weight pruning (Wanda-style).
18
 
19
- ## ✨ Key Features
20
-
21
- - **Specialization**: Optimized for Html tasks
22
- - **Pruning Method**: Wanda-style (|W| Γ— |activation|) importance scoring
23
- - **Size Reduction**: 1% weights pruned
24
- - **Use Case**: High accuracy retention, ideal for production use
25
-
26
- ## πŸ“Š Performance Comparison
27
 
28
  | Category | Original | Pruned | Change |
29
  |----------|----------|--------|--------|
30
- | Python | 30.0% | 40.0% | ↑ 10.0% |
31
  | **Html** | 0.0% | 0.0% ⭐ | β†’ |
32
- | Trivia | 90.0% | 90.0% | β†’ |
33
- | Math | 96.7% | 93.3% | ↓ 3.3% |
34
- | Reasoning | 36.7% | 36.7% | β†’ |
35
- | Medical | 83.3% | 86.7% | ↑ 3.3% |
36
- | Linux | 93.3% | 93.3% | β†’ |
37
- | Writing | 53.3% | 60.0% | ↑ 6.7% |
38
 
39
- **Average**: 60.4% β†’ 62.5% (+2.1%)
40
 
41
 
42
 
43
  ![Comparison Graph](comparison_graph.png)
44
 
45
- ## πŸš€ Quick Start
46
 
47
  ```python
48
  from transformers import AutoModelForCausalLM, AutoTokenizer
@@ -50,31 +42,20 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
50
  model = AutoModelForCausalLM.from_pretrained("CompactAI/Qwen3-0.6B-html-safe")
51
  tokenizer = AutoTokenizer.from_pretrained("CompactAI/Qwen3-0.6B-html-safe")
52
 
53
- # Example usage
54
  inputs = tokenizer("Your prompt here", return_tensors="pt")
55
  outputs = model.generate(**inputs, max_new_tokens=100)
56
  print(tokenizer.decode(outputs[0], skip_special_tokens=True))
57
  ```
58
 
59
- ## πŸ“‹ Technical Details
60
 
61
  | Property | Value |
62
  |----------|-------|
63
  | Base Model | [Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) |
64
  | Specialization | Html |
65
  | Prune Mode | Safe |
66
- | Pruning Method | Activation-based weight pruning (Wanda) |
67
  | Weight Reduction | 1% weights pruned |
68
 
69
- ## πŸ”— Related Models
70
-
71
- This model is part of the **Qwen3-0.6B** pruned model collection. Variants:
72
- - **Safe** - Conservative pruning (~10-20%), high accuracy retention
73
- - **Aggressive** - Maximum compression (~40-50%), best for edge deployment
74
 
75
- ## πŸ“œ License
76
-
77
- This model inherits the license from the base model [Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B).
78
-
79
- ---
80
- *Generated by ZANNPS [Zeto Automatic Neural Network Pruning System]*
 
5
  - html
6
  - optimized
7
  - wanda
 
8
  base_model: Qwen/Qwen3-0.6B
9
  pipeline_tag: text-generation
10
  ---
 
13
 
14
  > 🎯 **HTML-optimized** | πŸ“¦ **Safe** pruning | ⚑ **1% weights pruned**
15
 
16
+ This model is a **conservatively pruned** version of [Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B).
17
 
18
+ ## Performance Comparison
 
 
 
 
 
 
 
19
 
20
  | Category | Original | Pruned | Change |
21
  |----------|----------|--------|--------|
22
+ | Python | 0.0% | 0.0% | β†’ |
23
  | **Html** | 0.0% | 0.0% ⭐ | β†’ |
24
+ | Trivia | 28.6% | 28.6% | β†’ |
25
+ | Math | 33.3% | 33.3% | β†’ |
26
+ | Reasoning | 20.0% | 20.0% | β†’ |
27
+ | Medical | 66.7% | 66.7% | β†’ |
28
+ | Linux | 40.0% | 40.0% | β†’ |
29
+ | Writing | 16.7% | 16.7% | β†’ |
30
 
31
+ **Average**: 25.7% β†’ 25.7% (+0.0%)
32
 
33
 
34
 
35
  ![Comparison Graph](comparison_graph.png)
36
 
37
+ ## Quick Start
38
 
39
  ```python
40
  from transformers import AutoModelForCausalLM, AutoTokenizer
 
42
  model = AutoModelForCausalLM.from_pretrained("CompactAI/Qwen3-0.6B-html-safe")
43
  tokenizer = AutoTokenizer.from_pretrained("CompactAI/Qwen3-0.6B-html-safe")
44
 
 
45
  inputs = tokenizer("Your prompt here", return_tensors="pt")
46
  outputs = model.generate(**inputs, max_new_tokens=100)
47
  print(tokenizer.decode(outputs[0], skip_special_tokens=True))
48
  ```
49
 
50
+ ## Technical Details
51
 
52
  | Property | Value |
53
  |----------|-------|
54
  | Base Model | [Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) |
55
  | Specialization | Html |
56
  | Prune Mode | Safe |
 
57
  | Weight Reduction | 1% weights pruned |
58
 
59
+ ## License
 
 
 
 
60
 
61
+ This model inherits the license from the base model.
 
 
 
 
 
comparison_graph.png CHANGED
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fac42828b0df838ff64948a77496fdd08c2b7a6e097ec6cd514db348c76d0fcd
3
  size 1503300016
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef9fcca52692b642e0f789180f6132e343b22bf19f45041240b41945ac250d1e
3
  size 1503300016
tokenizer.json CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fea4f89c198c65a418ebfd87d7480db83fe21f31c7f56cd2ecea1110b1dff53e
3
  size 11422917
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bb5aa816bcb7fb495b5269f933d2710c6170d4dd410f5010a21bbfdd8c41f963
3
  size 11422917