Instructions to use shailja/fine-tuned-codegen-2B-Verilog with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use shailja/fine-tuned-codegen-2B-Verilog with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="shailja/fine-tuned-codegen-2B-Verilog")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("shailja/fine-tuned-codegen-2B-Verilog")
model = AutoModelForCausalLM.from_pretrained("shailja/fine-tuned-codegen-2B-Verilog")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use shailja/fine-tuned-codegen-2B-Verilog with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "shailja/fine-tuned-codegen-2B-Verilog"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shailja/fine-tuned-codegen-2B-Verilog",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/shailja/fine-tuned-codegen-2B-Verilog

SGLang

How to use shailja/fine-tuned-codegen-2B-Verilog with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "shailja/fine-tuned-codegen-2B-Verilog" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shailja/fine-tuned-codegen-2B-Verilog",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "shailja/fine-tuned-codegen-2B-Verilog" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shailja/fine-tuned-codegen-2B-Verilog",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use shailja/fine-tuned-codegen-2B-Verilog with Docker Model Runner:
```
docker model run hf.co/shailja/fine-tuned-codegen-2B-Verilog
```

what's the purpose of the model? Is it to help generate verilog design?

by StevenGon - opened Aug 30, 2023

Discussion

StevenGon

Aug 30, 2023

The model is too large to run demo in hugging face and I am not sure what is this? Thanks.

shailja

Owner Aug 30, 2023

Have you considered running any other LLM of the same scale? If not, the overhead will be the same. However, you can reduce the compute and memory footprint by using one of our in-house developed frameworks called fauxpilot, details in my GitHub page

shailja

Owner Aug 30, 2023

Yes, the model is used to generate Verilog for a given high level specification in natural language

noobmldude

Aug 30, 2023

It would really help to know details of how this model was trained.
Details like:

what was the dataset used for fine tuning
Total tokens used for fine tuning
Tasks this model can already be used for

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment