Instructions to use QuantFactory/HelpingAI-9B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use QuantFactory/HelpingAI-9B-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf QuantFactory/HelpingAI-9B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf QuantFactory/HelpingAI-9B-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf QuantFactory/HelpingAI-9B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf QuantFactory/HelpingAI-9B-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf QuantFactory/HelpingAI-9B-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf QuantFactory/HelpingAI-9B-GGUF:Q4_K_M
Use Docker
docker model run hf.co/QuantFactory/HelpingAI-9B-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use QuantFactory/HelpingAI-9B-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "QuantFactory/HelpingAI-9B-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "QuantFactory/HelpingAI-9B-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/QuantFactory/HelpingAI-9B-GGUF:Q4_K_M
- Ollama
How to use QuantFactory/HelpingAI-9B-GGUF with Ollama:
ollama run hf.co/QuantFactory/HelpingAI-9B-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use QuantFactory/HelpingAI-9B-GGUF with Docker Model Runner:
docker model run hf.co/QuantFactory/HelpingAI-9B-GGUF:Q4_K_M
- Lemonade
How to use QuantFactory/HelpingAI-9B-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull QuantFactory/HelpingAI-9B-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.HelpingAI-9B-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Upload LICENSE.md
Browse files- LICENSE.md +50 -0
LICENSE.md
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
************************************************
|
| 2 |
+
**** HelpingAI License ****
|
| 3 |
+
************************************************
|
| 4 |
+
|
| 5 |
+
Version 2.0
|
| 6 |
+
|
| 7 |
+
Developed by Abhay Koul
|
| 8 |
+
|
| 9 |
+
### Preamble
|
| 10 |
+
|
| 11 |
+
The HelpingAI License governs the use of HelpingAI's digital assets, including but not limited to software, scripts, datasets, documents, images, audio recordings, videos. The HelpingAI License aims to provide clear, comprehensive terms for accessing, modifying, and sharing resources, while promoting ethical development practices.
|
| 12 |
+
|
| 13 |
+
### Grant of Rights
|
| 14 |
+
|
| 15 |
+
Under the HelpingAI License, HelpingAI grants you the rights to copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Content, provided you comply with the terms and conditions outlined in this document.
|
| 16 |
+
|
| 17 |
+
### Terms and Conditions
|
| 18 |
+
|
| 19 |
+
To exercise the rights granted in the previous section, you must adhere to the following terms and conditions:
|
| 20 |
+
|
| 21 |
+
2.1. **Redistribution of Source Code.**
|
| 22 |
+
If you redistribute the Source Code, you must include the complete HelpingAI License with your distribution. You must also add clear notifications in all modified files stating:
|
| 23 |
+
|
| 24 |
+
> "This Work is released under the HelpingAI License v2.0."
|
| 25 |
+
|
| 26 |
+
2.2. **Distribution in Binary Form.**
|
| 27 |
+
If you distribute Binaries derived from the Source Code, you must include the following statement in your distribution:
|
| 28 |
+
|
| 29 |
+
> "This Work is based on the HelpingAI Licensed Work, under the HelpingAI License v2.0."
|
| 30 |
+
|
| 31 |
+
2.3. **Notification of Changes.**
|
| 32 |
+
You must clearly indicate any modifications you make to the Source Code or Documentation, including detailed comments about the nature and extent of the changes. Include the date and originator of the modifications.
|
| 33 |
+
|
| 34 |
+
2.4. **Branding Attribution.**
|
| 35 |
+
You must not remove or alter any HelpingAI branding, logos, or notices included in the Content without explicit prior consent from HelpingAI.
|
| 36 |
+
|
| 37 |
+
2.5. **Disclaimer of Warranty.**
|
| 38 |
+
The Content is provided "AS IS," without any implied warranties, including but not limited to warranties of merchantability, fitness for a particular purpose, and non-infringement.
|
| 39 |
+
|
| 40 |
+
2.6. **Limitation of Liability.**
|
| 41 |
+
To the maximum extent permitted by law, neither HelpingAI nor any contributor shall be liable for any loss, personal injury, property damage, or any indirect, special, incidental, or consequential damages arising from or related to the use of the Content.
|
| 42 |
+
|
| 43 |
+
2.7. **Governing Law.**
|
| 44 |
+
This HelpingAI License shall be governed and construed in accordance with the laws of the jurisdiction where HelpingAI primarily operates.
|
| 45 |
+
|
| 46 |
+
### Definitions
|
| 47 |
+
|
| 48 |
+
3.1. **"Source Code"** refers to the preferred form for making modifications to the Content, typically represented by human-readable programming languages, scripts, or documentation formats.
|
| 49 |
+
|
| 50 |
+
3.2. **"Binaries"** refers to compiled forms of the Source Code, such as executables, libraries, or similar artifacts produced from the Source Code.
|