How to use abdelkader-dev/Quark-pre-30M with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="abdelkader-dev/Quark-pre-30M")
# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("abdelkader-dev/Quark-pre-30M") model = AutoModelForCausalLM.from_pretrained("abdelkader-dev/Quark-pre-30M", device_map="auto")
How to use abdelkader-dev/Quark-pre-30M with vLLM:
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "abdelkader-dev/Quark-pre-30M" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "abdelkader-dev/Quark-pre-30M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'
docker model run hf.co/abdelkader-dev/Quark-pre-30M
How to use abdelkader-dev/Quark-pre-30M with SGLang:
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "abdelkader-dev/Quark-pre-30M" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "abdelkader-dev/Quark-pre-30M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "abdelkader-dev/Quark-pre-30M" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "abdelkader-dev/Quark-pre-30M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'
How to use abdelkader-dev/Quark-pre-30M with Docker Model Runner:
# Install vLLM from pip: pip install vllm
# Start the vLLM server: vllm serve "abdelkader-dev/Quark-pre-30M"
# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "abdelkader-dev/Quark-pre-30M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'
Quark-pre-30M is a pre-trained model intended for fine-tuning. It uses the GPT2 model's tokenizer and is ready for training on data related to classification, simple summarization, answering short questions, etc.
Files info
Base model
Install from pip and serve model
# Install vLLM from pip: pip install vllm# Start the vLLM server: vllm serve "abdelkader-dev/Quark-pre-30M"# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "abdelkader-dev/Quark-pre-30M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'