Instructions to use AstraMindAI/AstraQuasar-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AstraMindAI/AstraQuasar-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AstraMindAI/AstraQuasar-4B", trust_remote_code=True, device_map="auto")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("AstraMindAI/AstraQuasar-4B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("AstraMindAI/AstraQuasar-4B", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use AstraMindAI/AstraQuasar-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AstraMindAI/AstraQuasar-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AstraMindAI/AstraQuasar-4B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/AstraMindAI/AstraQuasar-4B

SGLang

How to use AstraMindAI/AstraQuasar-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AstraMindAI/AstraQuasar-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AstraMindAI/AstraQuasar-4B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AstraMindAI/AstraQuasar-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AstraMindAI/AstraQuasar-4B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use AstraMindAI/AstraQuasar-4B with Docker Model Runner:
```
docker model run hf.co/AstraMindAI/AstraQuasar-4B
```

mlinmg commited on Feb 15, 2024

Commit

b2f3be7

verified ·

1 Parent(s): 97bfc8b

Update LICENSE

Browse files

Files changed (1) hide show

LICENSE +8 -10

LICENSE CHANGED Viewed

@@ -1,6 +1,6 @@
-   Apache License
-                           Version 2.0, January 2004
-                        http://www.apache.org/licenses/
    TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
@@ -173,10 +173,8 @@
       incurred by, or claims asserted against, such Contributor by reason
       of your accepting any such warranty or additional liability.
-  10. Addendum to the Apache License, Version 2.0 - Flexible Termination Provision
-    This Addendum is an integral part of the Apache License, Version 2.0
-    (the "License") under which [Your Project Name] ("The Software")
-    is licensed. All terms of the Apache License, Version 2.0, apply to
     the Software, except as modified herein:
     Notwithstanding any other provision of this License, AstraMind reserves
     the right to modify the terms of the License for future versions of
@@ -189,14 +187,14 @@
     The goal of this provision is to allow AstraMind the flexibility to respond
     to changing legal, technical, or business environments.
     For clarity, this provision does not affect the License as it applies to versions
-    of The Software that have already been released under the Apache License,
-    Version 2.0, prior to any such modification or termination of the License
     for future versions.
 END OF TERMS AND CONDITIONS
    Copyright 2024 AstraMindAI
-   Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

+   Quasar License
+                           Version 1.0, February 2024
    TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
       incurred by, or claims asserted against, such Contributor by reason
       of your accepting any such warranty or additional liability.
+  10. Flexible Termination Provision
+    All terms of the Quasar License, Version 1.0, apply to
     the Software, except as modified herein:
     Notwithstanding any other provision of this License, AstraMind reserves
     the right to modify the terms of the License for future versions of
     The goal of this provision is to allow AstraMind the flexibility to respond
     to changing legal, technical, or business environments.
     For clarity, this provision does not affect the License as it applies to versions
+    of The Software that have already been released under the Quasar License,
+    Version 1.0, prior to any such modification or termination of the License
     for future versions.
 END OF TERMS AND CONDITIONS
    Copyright 2024 AstraMindAI
+   Licensed under the Quasar License, Version 1.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at