Text Generation
Transformers
PyTorch
Safetensors
English
llama
Eval Results (legacy)
text-generation-inference
Instructions to use pankajmathur/orca_mini_3b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use pankajmathur/orca_mini_3b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="pankajmathur/orca_mini_3b")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("pankajmathur/orca_mini_3b") model = AutoModelForCausalLM.from_pretrained("pankajmathur/orca_mini_3b") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use pankajmathur/orca_mini_3b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "pankajmathur/orca_mini_3b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "pankajmathur/orca_mini_3b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/pankajmathur/orca_mini_3b
- SGLang
How to use pankajmathur/orca_mini_3b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "pankajmathur/orca_mini_3b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "pankajmathur/orca_mini_3b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "pankajmathur/orca_mini_3b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "pankajmathur/orca_mini_3b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use pankajmathur/orca_mini_3b with Docker Model Runner:
docker model run hf.co/pankajmathur/orca_mini_3b
add AIBOM
#13
by sabato-nocera - opened
- pankajmathur_orca_mini_3b.json +243 -0
pankajmathur_orca_mini_3b.json
ADDED
|
@@ -0,0 +1,243 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bomFormat": "CycloneDX",
|
| 3 |
+
"specVersion": "1.6",
|
| 4 |
+
"serialNumber": "urn:uuid:1888515b-dc4f-45ab-8c2d-b442c0d24934",
|
| 5 |
+
"version": 1,
|
| 6 |
+
"metadata": {
|
| 7 |
+
"timestamp": "2025-06-05T09:37:53.860418+00:00",
|
| 8 |
+
"component": {
|
| 9 |
+
"type": "machine-learning-model",
|
| 10 |
+
"bom-ref": "pankajmathur/orca_mini_3b-eafe2e45-5a59-5b43-9c97-86c388bed7b9",
|
| 11 |
+
"name": "pankajmathur/orca_mini_3b",
|
| 12 |
+
"externalReferences": [
|
| 13 |
+
{
|
| 14 |
+
"url": "https://huggingface.co/pankajmathur/orca_mini_3b",
|
| 15 |
+
"type": "documentation"
|
| 16 |
+
}
|
| 17 |
+
],
|
| 18 |
+
"modelCard": {
|
| 19 |
+
"modelParameters": {
|
| 20 |
+
"task": "text-generation",
|
| 21 |
+
"architectureFamily": "llama",
|
| 22 |
+
"modelArchitecture": "LlamaForCausalLM",
|
| 23 |
+
"datasets": [
|
| 24 |
+
{
|
| 25 |
+
"ref": "psmathur/alpaca_orca-0d13688f-ffdd-5fd5-9522-083dd42cdac9"
|
| 26 |
+
},
|
| 27 |
+
{
|
| 28 |
+
"ref": "psmathur/dolly-v2_orca-ec6d4ce8-7474-520d-ac1e-080f58c05b6c"
|
| 29 |
+
},
|
| 30 |
+
{
|
| 31 |
+
"ref": "psmathur/WizardLM_Orca-f084d080-d716-5a1d-bca0-b551ab1587aa"
|
| 32 |
+
}
|
| 33 |
+
]
|
| 34 |
+
},
|
| 35 |
+
"properties": [
|
| 36 |
+
{
|
| 37 |
+
"name": "library_name",
|
| 38 |
+
"value": "transformers"
|
| 39 |
+
}
|
| 40 |
+
],
|
| 41 |
+
"quantitativeAnalysis": {
|
| 42 |
+
"performanceMetrics": [
|
| 43 |
+
{
|
| 44 |
+
"slice": "dataset: ai2_arc, split: test, config: ARC-Challenge",
|
| 45 |
+
"type": "acc_norm",
|
| 46 |
+
"value": 41.55
|
| 47 |
+
},
|
| 48 |
+
{
|
| 49 |
+
"slice": "dataset: hellaswag, split: validation",
|
| 50 |
+
"type": "acc_norm",
|
| 51 |
+
"value": 61.52
|
| 52 |
+
},
|
| 53 |
+
{
|
| 54 |
+
"slice": "dataset: cais/mmlu, split: test, config: all",
|
| 55 |
+
"type": "acc",
|
| 56 |
+
"value": 26.79
|
| 57 |
+
},
|
| 58 |
+
{
|
| 59 |
+
"slice": "dataset: truthful_qa, split: validation, config: multiple_choice",
|
| 60 |
+
"type": "mc2",
|
| 61 |
+
"value": 42.42
|
| 62 |
+
},
|
| 63 |
+
{
|
| 64 |
+
"slice": "dataset: winogrande, split: validation, config: winogrande_xl",
|
| 65 |
+
"type": "acc",
|
| 66 |
+
"value": 61.8
|
| 67 |
+
},
|
| 68 |
+
{
|
| 69 |
+
"slice": "dataset: gsm8k, split: test, config: main",
|
| 70 |
+
"type": "acc",
|
| 71 |
+
"value": 0.08
|
| 72 |
+
}
|
| 73 |
+
]
|
| 74 |
+
}
|
| 75 |
+
},
|
| 76 |
+
"authors": [
|
| 77 |
+
{
|
| 78 |
+
"name": "pankajmathur"
|
| 79 |
+
}
|
| 80 |
+
],
|
| 81 |
+
"licenses": [
|
| 82 |
+
{
|
| 83 |
+
"license": {
|
| 84 |
+
"id": "CC-BY-NC-SA-4.0",
|
| 85 |
+
"url": "https://spdx.org/licenses/CC-BY-NC-SA-4.0.html"
|
| 86 |
+
}
|
| 87 |
+
}
|
| 88 |
+
],
|
| 89 |
+
"tags": [
|
| 90 |
+
"transformers",
|
| 91 |
+
"pytorch",
|
| 92 |
+
"safetensors",
|
| 93 |
+
"llama",
|
| 94 |
+
"text-generation",
|
| 95 |
+
"en",
|
| 96 |
+
"dataset:psmathur/alpaca_orca",
|
| 97 |
+
"dataset:psmathur/dolly-v2_orca",
|
| 98 |
+
"dataset:psmathur/WizardLM_Orca",
|
| 99 |
+
"arxiv:2306.02707",
|
| 100 |
+
"license:cc-by-nc-sa-4.0",
|
| 101 |
+
"model-index",
|
| 102 |
+
"autotrain_compatible",
|
| 103 |
+
"text-generation-inference",
|
| 104 |
+
"endpoints_compatible",
|
| 105 |
+
"region:us"
|
| 106 |
+
]
|
| 107 |
+
}
|
| 108 |
+
},
|
| 109 |
+
"components": [
|
| 110 |
+
{
|
| 111 |
+
"type": "data",
|
| 112 |
+
"bom-ref": "psmathur/alpaca_orca-0d13688f-ffdd-5fd5-9522-083dd42cdac9",
|
| 113 |
+
"name": "psmathur/alpaca_orca",
|
| 114 |
+
"data": [
|
| 115 |
+
{
|
| 116 |
+
"type": "dataset",
|
| 117 |
+
"bom-ref": "psmathur/alpaca_orca-0d13688f-ffdd-5fd5-9522-083dd42cdac9",
|
| 118 |
+
"name": "psmathur/alpaca_orca",
|
| 119 |
+
"contents": {
|
| 120 |
+
"url": "https://huggingface.co/datasets/psmathur/alpaca_orca",
|
| 121 |
+
"properties": [
|
| 122 |
+
{
|
| 123 |
+
"name": "task_categories",
|
| 124 |
+
"value": "text-generation"
|
| 125 |
+
},
|
| 126 |
+
{
|
| 127 |
+
"name": "language",
|
| 128 |
+
"value": "en"
|
| 129 |
+
},
|
| 130 |
+
{
|
| 131 |
+
"name": "size_categories",
|
| 132 |
+
"value": "10K<n<100K"
|
| 133 |
+
},
|
| 134 |
+
{
|
| 135 |
+
"name": "license",
|
| 136 |
+
"value": "cc-by-nc-sa-4.0"
|
| 137 |
+
}
|
| 138 |
+
]
|
| 139 |
+
},
|
| 140 |
+
"governance": {
|
| 141 |
+
"owners": [
|
| 142 |
+
{
|
| 143 |
+
"organization": {
|
| 144 |
+
"name": "pankajmathur",
|
| 145 |
+
"url": "https://huggingface.co/pankajmathur"
|
| 146 |
+
}
|
| 147 |
+
}
|
| 148 |
+
]
|
| 149 |
+
},
|
| 150 |
+
"description": "Explain tuned Alpaca dataset ~52K created using approaches from Orca Research Paper. \nWe leverage all of the 15 system instructions provided in Orca Research Paper. to generate custom datasets, in contrast to vanilla instruction tuning approaches used by original datasets.\nThis helps student models like orca_mini_13b to learn thought process from teacher model, which is ChatGPT (gpt-3.5-turbo-0301 version).\nPlease see how the System prompt is added before each instruction.\n"
|
| 151 |
+
}
|
| 152 |
+
]
|
| 153 |
+
},
|
| 154 |
+
{
|
| 155 |
+
"type": "data",
|
| 156 |
+
"bom-ref": "psmathur/dolly-v2_orca-ec6d4ce8-7474-520d-ac1e-080f58c05b6c",
|
| 157 |
+
"name": "psmathur/dolly-v2_orca",
|
| 158 |
+
"data": [
|
| 159 |
+
{
|
| 160 |
+
"type": "dataset",
|
| 161 |
+
"bom-ref": "psmathur/dolly-v2_orca-ec6d4ce8-7474-520d-ac1e-080f58c05b6c",
|
| 162 |
+
"name": "psmathur/dolly-v2_orca",
|
| 163 |
+
"contents": {
|
| 164 |
+
"url": "https://huggingface.co/datasets/psmathur/dolly-v2_orca",
|
| 165 |
+
"properties": [
|
| 166 |
+
{
|
| 167 |
+
"name": "task_categories",
|
| 168 |
+
"value": "text-generation"
|
| 169 |
+
},
|
| 170 |
+
{
|
| 171 |
+
"name": "language",
|
| 172 |
+
"value": "en"
|
| 173 |
+
},
|
| 174 |
+
{
|
| 175 |
+
"name": "size_categories",
|
| 176 |
+
"value": "10K<n<100K"
|
| 177 |
+
},
|
| 178 |
+
{
|
| 179 |
+
"name": "license",
|
| 180 |
+
"value": "cc-by-nc-sa-4.0"
|
| 181 |
+
}
|
| 182 |
+
]
|
| 183 |
+
},
|
| 184 |
+
"governance": {
|
| 185 |
+
"owners": [
|
| 186 |
+
{
|
| 187 |
+
"organization": {
|
| 188 |
+
"name": "pankajmathur",
|
| 189 |
+
"url": "https://huggingface.co/pankajmathur"
|
| 190 |
+
}
|
| 191 |
+
}
|
| 192 |
+
]
|
| 193 |
+
},
|
| 194 |
+
"description": "Explain tuned Dolly-V2 dataset ~15K created using approaches from Orca Research Paper.\nWe leverage all of the 15 system instructions provided in Orca Research Paper to generate explain tuned datasets, in contrast to vanilla instruction tuning approaches used by original datasets.\nThis helps student models like orca_mini_13b, orca_mini_7b or orca_mini_3b to learn thought process from teacher model, which is ChatGPT (gpt-3.5-turbo-0301 version).\nPlease see how the System prompt is added before\u2026 See the full description on the dataset page: https://huggingface.co/datasets/pankajmathur/dolly-v2_orca."
|
| 195 |
+
}
|
| 196 |
+
]
|
| 197 |
+
},
|
| 198 |
+
{
|
| 199 |
+
"type": "data",
|
| 200 |
+
"bom-ref": "psmathur/WizardLM_Orca-f084d080-d716-5a1d-bca0-b551ab1587aa",
|
| 201 |
+
"name": "psmathur/WizardLM_Orca",
|
| 202 |
+
"data": [
|
| 203 |
+
{
|
| 204 |
+
"type": "dataset",
|
| 205 |
+
"bom-ref": "psmathur/WizardLM_Orca-f084d080-d716-5a1d-bca0-b551ab1587aa",
|
| 206 |
+
"name": "psmathur/WizardLM_Orca",
|
| 207 |
+
"contents": {
|
| 208 |
+
"url": "https://huggingface.co/datasets/psmathur/WizardLM_Orca",
|
| 209 |
+
"properties": [
|
| 210 |
+
{
|
| 211 |
+
"name": "task_categories",
|
| 212 |
+
"value": "text-generation"
|
| 213 |
+
},
|
| 214 |
+
{
|
| 215 |
+
"name": "language",
|
| 216 |
+
"value": "en"
|
| 217 |
+
},
|
| 218 |
+
{
|
| 219 |
+
"name": "size_categories",
|
| 220 |
+
"value": "10K<n<100K"
|
| 221 |
+
},
|
| 222 |
+
{
|
| 223 |
+
"name": "license",
|
| 224 |
+
"value": "cc-by-nc-sa-4.0"
|
| 225 |
+
}
|
| 226 |
+
]
|
| 227 |
+
},
|
| 228 |
+
"governance": {
|
| 229 |
+
"owners": [
|
| 230 |
+
{
|
| 231 |
+
"organization": {
|
| 232 |
+
"name": "pankajmathur",
|
| 233 |
+
"url": "https://huggingface.co/pankajmathur"
|
| 234 |
+
}
|
| 235 |
+
}
|
| 236 |
+
]
|
| 237 |
+
},
|
| 238 |
+
"description": "Explain tuned WizardLM dataset ~55K created using approaches from Orca Research Paper.\nWe leverage all of the 15 system instructions provided in Orca Research Paper. to generate custom datasets, in contrast to vanilla instruction tuning approaches used by original datasets.\nThis helps student models like orca_mini_13b to learn thought process from teacher model, which is ChatGPT (gpt-3.5-turbo-0301 version).\nPlease see how the System prompt is added before each instruction.\n"
|
| 239 |
+
}
|
| 240 |
+
]
|
| 241 |
+
}
|
| 242 |
+
]
|
| 243 |
+
}
|