wpferrell commited on
Commit
72ecce9
Β·
verified Β·
1 Parent(s): 1327552

docs: full documentation update v3.13.1

Browse files
Files changed (1) hide show
  1. README.md +44 -35
README.md CHANGED
@@ -8,9 +8,9 @@ tags:
8
 
9
  [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.20279248.svg)](https://doi.org/10.5281/zenodo.20279248)
10
 
11
- # Phi-3.5 Mini Instruct β€” BigSmall Compressed
12
 
13
- **65.6% smaller than [microsoft/Phi-3.5-mini-instruct](https://huggingface.co/microsoft/Phi-3.5-mini-instruct). Bit-identical weights. Drop-in replacement.**
14
 
15
  ## Use it in 2 lines
16
 
@@ -19,27 +19,29 @@ pip install bigsmall
19
  ```
20
 
21
  ```python
22
- from transformers import AutoModelForCausalLM, AutoTokenizer
23
-
24
  model = AutoModelForCausalLM.from_pretrained("wpferrell/phi-3.5-mini-instruct-bigsmall")
25
- tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3.5-mini-instruct")
26
  ```
27
 
 
 
28
  ## Size comparison
29
 
30
- | | Size |
31
  |---|---|
32
- | Original ([microsoft/Phi-3.5-mini-instruct](https://huggingface.co/microsoft/Phi-3.5-mini-instruct)) | 7.2 GB |
33
  | This compressed version | 4.67 GB |
34
- | Compression ratio | 65.6% of original |
35
 
36
  ## What "lossless" means
37
 
38
- Every weight is mathematically identical to the original. Not quantized. Not pruned. Identical model behaviour. Compressed using entropy coding β€” like ZIP but designed for neural weights.
39
 
40
- Every tensor's md5 is stored in the file header; `bigsmall verify` decompresses and verifies bit-for-bit.
 
 
41
 
42
- ## Low VRAM streaming
43
 
44
  ```python
45
  from bigsmall import BigSmallStreamingModel
@@ -47,43 +49,50 @@ from bigsmall import BigSmallStreamingModel
47
  model = BigSmallStreamingModel.from_pretrained(
48
  "wpferrell/phi-3.5-mini-instruct-bigsmall",
49
  device="cuda",
50
- lru_max_vram_gb=2.0, # cache 2 GB of decoded layers (v3.13.0+)
51
  )
52
- out = model.generate(input_ids, max_new_tokens=100)
53
  ```
54
 
55
- Streams layers on demand β€” uses ~12Γ— less VRAM than a full load.
56
 
57
- ## New in BigSmall v3.13.0
58
 
59
- - **Delta compression** β€” if you have the base model already, store a fine-tune as a tiny patch on top:
60
- ```bash
61
- bigsmall compress my_finetune.safetensors --delta-from base.safetensors -o patch.bs
62
- ```
63
- - **`bigsmall scan`** β€” analyse a model before compressing.
64
- - **`bigsmall apply`** β€” reconstruct a fine-tune from base + patch.
65
- - **`bigsmall verify --sample 0.001`** β€” fast probabilistic integrity check.
66
- - **`bigsmall compress --ecc`** β€” optional Reed-Solomon error recovery.
67
- - **`bigsmall compress --resume`** β€” resumable compression.
68
 
69
- [Full release notes β†’](https://github.com/wpferrell/Bigsmall/releases/tag/v3.13.0)
70
 
71
- ## All pre-compressed models
72
 
73
- See [wpferrell on HuggingFace](https://huggingface.co/wpferrell) for all available models.
74
 
75
- ## License
 
 
 
76
 
77
- Model weights: same license as the original model ([microsoft/Phi-3.5-mini-instruct](https://huggingface.co/microsoft/Phi-3.5-mini-instruct)).
78
- BigSmall format: [Elastic License 2.0](https://www.elastic.co/licensing/elastic-license) β€” free for personal, research, and internal commercial use.
79
- Commercial/SaaS licensing: wpferrell@gmail.com
80
 
81
- ## Requires
82
 
83
- `bigsmall >= 3.13.0` to access the latest features. Older versions (>= 3.0.0) can still decode this model.
 
 
84
 
85
  ## Citation
86
 
87
- If you use BigSmall in research:
 
 
 
 
 
 
 
 
 
 
88
 
89
- > Ferrell, W. (2026). *BigSmall: Lossless Compression for Neural Network Weights.* Zenodo. https://doi.org/10.5281/zenodo.20279248
 
8
 
9
  [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.20279248.svg)](https://doi.org/10.5281/zenodo.20279248)
10
 
11
+ # Phi-3.5 Mini Instruct β€” Lossless Compressed
12
 
13
+ > **7.12 GB β†’ 4.67 GB (34% smaller). Bit-identical weights. Drop-in replacement.**
14
 
15
  ## Use it in 2 lines
16
 
 
19
  ```
20
 
21
  ```python
22
+ from transformers import AutoModelForCausalLM
 
23
  model = AutoModelForCausalLM.from_pretrained("wpferrell/phi-3.5-mini-instruct-bigsmall")
 
24
  ```
25
 
26
+ It works exactly like loading the original model. No code changes needed.
27
+
28
  ## Size comparison
29
 
30
+ | | Size |
31
  |---|---|
32
+ | Original ([microsoft/Phi-3.5-mini-instruct](https://huggingface.co/microsoft/Phi-3.5-mini-instruct)) | 7.12 GB |
33
  | This compressed version | 4.67 GB |
34
+ | Saved | 2.45 GB (34%) |
35
 
36
  ## What "lossless" means
37
 
38
+ Every weight is mathematically identical to the original model.
39
 
40
+ - **Not quantized.** Quantization rounds weights and changes model behaviour.
41
+ - **Not pruned.** Pruning removes parts of the model.
42
+ - **Bit-for-bit identical.** md5 is verified on every tensor at decompression.
43
 
44
+ ## Low-VRAM streaming
45
 
46
  ```python
47
  from bigsmall import BigSmallStreamingModel
 
49
  model = BigSmallStreamingModel.from_pretrained(
50
  "wpferrell/phi-3.5-mini-instruct-bigsmall",
51
  device="cuda",
52
+ lru_max_vram_gb=2.0,
53
  )
 
54
  ```
55
 
56
+ Uses up to ~12Γ— less VRAM than standard loading by streaming layers on demand.
57
 
58
+ ## Decompress to safetensors
59
 
60
+ ```bash
61
+ pip install bigsmall
62
+ bigsmall decompress wpferrell/phi-3.5-mini-instruct-bigsmall -o phi-3.5-mini-instruct-bigsmall/
63
+ ```
 
 
 
 
 
64
 
65
+ ## Original model
66
 
67
+ This is a lossless-compressed copy of [microsoft/Phi-3.5-mini-instruct](https://huggingface.co/microsoft/Phi-3.5-mini-instruct). All credit to the original authors. The weights are unchanged.
68
 
69
+ ## Want to compress your own model?
70
 
71
+ ```bash
72
+ pip install bigsmall
73
+ bigsmall compress my-model/ -o my-model.bs
74
+ ```
75
 
76
+ See [github.com/wpferrell/Bigsmall](https://github.com/wpferrell/Bigsmall) for the full docs.
 
 
77
 
78
+ ## License
79
 
80
+ - **Model weights:** mit β€” same as [microsoft/Phi-3.5-mini-instruct](https://huggingface.co/microsoft/Phi-3.5-mini-instruct).
81
+ - **BigSmall format:** [Elastic License 2.0](https://github.com/wpferrell/Bigsmall/blob/main/LICENSE) β€” free for personal, research, and commercial use.
82
+ - **Commercial SaaS licensing:** wpferrell@gmail.com
83
 
84
  ## Citation
85
 
86
+ ```bibtex
87
+ @misc{bigsmall2026,
88
+ title={BigSmall: Lossless Neural Network Weight Compression},
89
+ author={Ferrell, Will},
90
+ year={2026},
91
+ doi={10.5281/zenodo.20279248},
92
+ url={https://doi.org/10.5281/zenodo.20279248}
93
+ }
94
+ ```
95
+
96
+ ## Requires
97
 
98
+ `bigsmall >= 3.13.0` for the latest features. Earlier versions (>= 3.0.0) can still decode this model.