Instructions to use Sin/DialoGPT-small-zai with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Sin/DialoGPT-small-zai with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Sin/DialoGPT-small-zai")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Sin/DialoGPT-small-zai") model = AutoModelForCausalLM.from_pretrained("Sin/DialoGPT-small-zai") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Sin/DialoGPT-small-zai with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Sin/DialoGPT-small-zai" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Sin/DialoGPT-small-zai", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Sin/DialoGPT-small-zai
- SGLang
How to use Sin/DialoGPT-small-zai with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Sin/DialoGPT-small-zai" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Sin/DialoGPT-small-zai", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Sin/DialoGPT-small-zai" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Sin/DialoGPT-small-zai", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Sin/DialoGPT-small-zai with Docker Model Runner:
docker model run hf.co/Sin/DialoGPT-small-zai
| <!-- You may freely edit this file. See commented blocks below for --> | |
| <!-- some examples of how to customize the build. --> | |
| <!-- (If you delete it and reopen the project it will be recreated.) --> | |
| <!-- By default, only the Clean and Build commands use this build script. --> | |
| <!-- Commands such as Run, Debug, and Test only use this build script if --> | |
| <!-- the Compile on Save feature is turned off for the project. --> | |
| <!-- You can turn off the Compile on Save (or Deploy on Save) setting --> | |
| <!-- in the project's Project Properties dialog box.--> | |
| <project name="TestBufferedReader" default="default" basedir="."> | |
| <description>Builds, tests, and runs the project TestBufferedReader.</description> | |
| <import file="nbproject/build-impl.xml"/> | |
| <!-- | |
| There exist several targets which are by default empty and which can be | |
| used for execution of your tasks. These targets are usually executed | |
| before and after some main targets. They are: | |
| -pre-init: called before initialization of project properties | |
| -post-init: called after initialization of project properties | |
| -pre-compile: called before javac compilation | |
| -post-compile: called after javac compilation | |
| -pre-compile-single: called before javac compilation of single file | |
| -post-compile-single: called after javac compilation of single file | |
| -pre-compile-test: called before javac compilation of JUnit tests | |
| -post-compile-test: called after javac compilation of JUnit tests | |
| -pre-compile-test-single: called before javac compilation of single JUnit test | |
| -post-compile-test-single: called after javac compilation of single JUunit test | |
| -pre-jar: called before JAR building | |
| -post-jar: called after JAR building | |
| -post-clean: called after cleaning build products | |
| (Targets beginning with '-' are not intended to be called on their own.) | |
| Example of inserting an obfuscator after compilation could look like this: | |
| <target name="-post-compile"> | |
| <obfuscate> | |
| <fileset dir="${build.classes.dir}"/> | |
| </obfuscate> | |
| </target> | |
| For list of available properties check the imported | |
| nbproject/build-impl.xml file. | |
| Another way to customize the build is by overriding existing main targets. | |
| The targets of interest are: | |
| -init-macrodef-javac: defines macro for javac compilation | |
| -init-macrodef-junit: defines macro for junit execution | |
| -init-macrodef-debug: defines macro for class debugging | |
| -init-macrodef-java: defines macro for class execution | |
| -do-jar: JAR building | |
| run: execution of project | |
| -javadoc-build: Javadoc generation | |
| test-report: JUnit report generation | |
| An example of overriding the target for project execution could look like this: | |
| <target name="run" depends="TestBufferedReader-impl.jar"> | |
| <exec dir="bin" executable="launcher.exe"> | |
| <arg file="${dist.jar}"/> | |
| </exec> | |
| </target> | |
| Notice that the overridden target depends on the jar target and not only on | |
| the compile target as the regular run target does. Again, for a list of available | |
| properties which you can use, check the target you are overriding in the | |
| nbproject/build-impl.xml file. | |
| --> | |
| </project> | |