Abhaykoul commited on
Commit
bc49d36
·
verified ·
1 Parent(s): 9121d4b

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +37 -0
README.md ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ language:
4
+ - en
5
+ license: mit
6
+ tags:
7
+ - HelpingAI
8
+ - GGUF
9
+ - lite
10
+ - code
11
+ ---
12
+
13
+ #### Description
14
+
15
+ Optimize your engagement with [This project](https://huggingface.co/OEvortex/HelpingAI-Lite) by seamlessly integrating GGUF Format model files.
16
+
17
+ ### GGUF Technical Specifications
18
+
19
+ Delve into the intricacies of GGUF, a meticulously crafted format that builds upon the robust foundation of the GGJT model. Tailored for heightened extensibility and user-centric functionality, GGUF introduces a suite of indispensable features:
20
+
21
+ **Single-file Deployment:** Streamline distribution and loading effortlessly. GGUF models have been meticulously architected for seamless deployment, necessitating no external files for supplementary information.
22
+
23
+ **Extensibility:** Safeguard the future of your models. GGUF seamlessly accommodates the integration of new features into GGML-based executors, ensuring compatibility with existing models.
24
+
25
+ **mmap Compatibility:** Prioritize efficiency. GGUF models are purposefully engineered to support mmap, facilitating rapid loading and saving, thus optimizing your workflow.
26
+
27
+ **User-Friendly:** Simplify your coding endeavors. Load and save models effortlessly, irrespective of the programming language used, obviating the dependency on external libraries.
28
+
29
+ **Full Information:** A comprehensive repository in a single file. GGUF models encapsulate all requisite information for loading, eliminating the need for users to furnish additional data.
30
+
31
+ The differentiator between GGJT and GGUF lies in the deliberate adoption of a key-value structure for hyperparameters (now termed metadata). Bid farewell to untyped lists, and embrace a structured approach that seamlessly accommodates new metadata without compromising compatibility with existing models. Augment your model with supplementary information for enhanced inference and model identification.
32
+
33
+ ### Quantization
34
+
35
+ Embark on a meticulous exploration of quantization with an array of well-defined methods at your disposal:
36
+
37
+ QUANTIZATION = q2_k, q3_k_l, q3_k_m, q3_k_s, q4_0, q4_1, q4_k_m, q4_k_s, q5_0, q5_1, q5_k_m, q5_k_s, q6_k, q8_0