Instructions to use youssefedweqd/working with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use youssefedweqd/working with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="youssefedweqd/working")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("youssefedweqd/working", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use youssefedweqd/working with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "youssefedweqd/working"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "youssefedweqd/working",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/youssefedweqd/working

SGLang

How to use youssefedweqd/working with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "youssefedweqd/working" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "youssefedweqd/working",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "youssefedweqd/working" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "youssefedweqd/working",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use youssefedweqd/working with Docker Model Runner:
```
docker model run hf.co/youssefedweqd/working
```

working / Model /LLaMA-Factory /docker /docker-npu /Dockerfile

youssefedweqd

Training in progress, step 1000

cd63278 verified 11 months ago

raw

history blame contribute delete

1.47 kB

	# https://hub.docker.com/r/ascendai/cann/tags
	ARG BASE_IMAGE=ascendai/cann:8.0.0-910b-ubuntu22.04-py3.11
	FROM ${BASE_IMAGE}

	# Installation arguments
	ARG PIP_INDEX=https://pypi.org/simple
	ARG EXTRAS=torch-npu,metrics
	ARG HTTP_PROXY=""

	# Define environments
	ENV MAX_JOBS=16
	ENV FLASH_ATTENTION_FORCE_BUILD=TRUE
	ENV VLLM_WORKER_MULTIPROC_METHOD=spawn
	ENV DEBIAN_FRONTEND=noninteractive
	ENV NODE_OPTIONS=""
	ENV PIP_ROOT_USER_ACTION=ignore
	ENV http_proxy="${HTTP_PROXY}"
	ENV https_proxy="${HTTP_PROXY}"

	# Use Bash instead of default /bin/sh
	SHELL ["/bin/bash", "-c"]

	# Set the working directory
	WORKDIR /app

	# Change pip source
	RUN pip config set global.index-url "${PIP_INDEX}" && \
	pip config set global.extra-index-url "${PIP_INDEX}" && \
	pip install --no-cache-dir --upgrade pip packaging wheel setuptools

	# Install the requirements
	COPY requirements.txt /app
	RUN pip install --no-cache-dir -r requirements.txt

	# Copy the rest of the application into the image
	COPY . /app

	# Install LLaMA Factory
	RUN pip install --no-cache-dir -e ".[${EXTRAS}]" --no-build-isolation

	# Set up volumes
	# VOLUME [ "/root/.cache/huggingface", "/app/shared_data", "/app/output" ]

	# Expose port 7860 for LLaMA Board
	ENV GRADIO_SERVER_PORT=7860
	EXPOSE 7860

	# Expose port 8000 for API service
	ENV API_PORT=8000
	EXPOSE 8000

	# unset proxy
	ENV http_proxy=
	ENV https_proxy=

	# Reset pip config
	RUN pip config unset global.index-url && \
	pip config unset global.extra-index-url