Text Generation
Transformers
Safetensors
gemma2
llama-factory
full
Generated from Trainer
conversational
text-generation-inference
Instructions to use tzchen07/g2_X9g with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tzchen07/g2_X9g with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="tzchen07/g2_X9g") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForMultimodalLM tokenizer = AutoTokenizer.from_pretrained("tzchen07/g2_X9g") model = AutoModelForMultimodalLM.from_pretrained("tzchen07/g2_X9g") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use tzchen07/g2_X9g with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "tzchen07/g2_X9g" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tzchen07/g2_X9g", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/tzchen07/g2_X9g
- SGLang
How to use tzchen07/g2_X9g with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "tzchen07/g2_X9g" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tzchen07/g2_X9g", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "tzchen07/g2_X9g" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tzchen07/g2_X9g", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use tzchen07/g2_X9g with Docker Model Runner:
docker model run hf.co/tzchen07/g2_X9g
| {"current_steps": 205, "total_steps": 446, "loss": 0.0637, "lr": 4.444444444444445e-07, "epoch": 0.9197980931015143, "percentage": 45.96, "elapsed_time": "0:01:46", "remaining_time": "0:02:04"} | |
| {"current_steps": 210, "total_steps": 446, "loss": 0.0608, "lr": 1.0000000000000002e-06, "epoch": 0.9422321929332585, "percentage": 47.09, "elapsed_time": "0:03:33", "remaining_time": "0:03:59"} | |
| {"current_steps": 215, "total_steps": 446, "loss": 0.0747, "lr": 1.5555555555555558e-06, "epoch": 0.9646662927650028, "percentage": 48.21, "elapsed_time": "0:05:19", "remaining_time": "0:05:43"} | |
| {"current_steps": 220, "total_steps": 446, "loss": 0.0801, "lr": 2.1111111111111114e-06, "epoch": 0.9871003925967471, "percentage": 49.33, "elapsed_time": "0:07:03", "remaining_time": "0:07:14"} | |
| {"current_steps": 225, "total_steps": 446, "loss": 0.0655, "lr": 2.666666666666667e-06, "epoch": 1.0089736399326976, "percentage": 50.45, "elapsed_time": "0:08:46", "remaining_time": "0:08:36"} | |
| {"current_steps": 230, "total_steps": 446, "loss": 0.0574, "lr": 3.2222222222222227e-06, "epoch": 1.0314077397644419, "percentage": 51.57, "elapsed_time": "0:10:29", "remaining_time": "0:09:51"} | |
| {"current_steps": 235, "total_steps": 446, "loss": 0.0608, "lr": 3.777777777777778e-06, "epoch": 1.0538418395961862, "percentage": 52.69, "elapsed_time": "0:12:17", "remaining_time": "0:11:02"} | |
| {"current_steps": 240, "total_steps": 446, "loss": 0.0568, "lr": 4.333333333333334e-06, "epoch": 1.0762759394279304, "percentage": 53.81, "elapsed_time": "0:14:00", "remaining_time": "0:12:01"} | |
| {"current_steps": 245, "total_steps": 446, "loss": 0.0525, "lr": 4.888888888888889e-06, "epoch": 1.0987100392596747, "percentage": 54.93, "elapsed_time": "0:15:44", "remaining_time": "0:12:54"} | |
| {"current_steps": 250, "total_steps": 446, "loss": 0.0593, "lr": 4.998772545353743e-06, "epoch": 1.121144139091419, "percentage": 56.05, "elapsed_time": "0:17:27", "remaining_time": "0:13:41"} | |
| {"current_steps": 255, "total_steps": 446, "loss": 0.049, "lr": 4.99378807653289e-06, "epoch": 1.1435782389231632, "percentage": 57.17, "elapsed_time": "0:19:11", "remaining_time": "0:14:22"} | |
| {"current_steps": 260, "total_steps": 446, "loss": 0.0569, "lr": 4.984977519145677e-06, "epoch": 1.1660123387549075, "percentage": 58.3, "elapsed_time": "0:20:58", "remaining_time": "0:15:00"} | |
| {"current_steps": 265, "total_steps": 446, "loss": 0.0542, "lr": 4.9723543907693185e-06, "epoch": 1.1884464385866518, "percentage": 59.42, "elapsed_time": "0:22:43", "remaining_time": "0:15:31"} | |
| {"current_steps": 270, "total_steps": 446, "loss": 0.0548, "lr": 4.955938058409217e-06, "epoch": 1.210880538418396, "percentage": 60.54, "elapsed_time": "0:24:30", "remaining_time": "0:15:58"} | |
| {"current_steps": 275, "total_steps": 446, "loss": 0.0538, "lr": 4.935753708785181e-06, "epoch": 1.2333146382501403, "percentage": 61.66, "elapsed_time": "0:26:13", "remaining_time": "0:16:18"} | |
| {"current_steps": 280, "total_steps": 446, "loss": 0.0594, "lr": 4.911832309688758e-06, "epoch": 1.2557487380818846, "percentage": 62.78, "elapsed_time": "0:27:59", "remaining_time": "0:16:35"} | |
| {"current_steps": 285, "total_steps": 446, "loss": 0.061, "lr": 4.884210562470972e-06, "epoch": 1.2781828379136286, "percentage": 63.9, "elapsed_time": "0:29:41", "remaining_time": "0:16:46"} | |
| {"current_steps": 290, "total_steps": 446, "loss": 0.0643, "lr": 4.852930845733358e-06, "epoch": 1.300616937745373, "percentage": 65.02, "elapsed_time": "0:31:23", "remaining_time": "0:16:53"} | |
| {"current_steps": 295, "total_steps": 446, "loss": 0.0572, "lr": 4.818041150308695e-06, "epoch": 1.3230510375771172, "percentage": 66.14, "elapsed_time": "0:33:08", "remaining_time": "0:16:58"} | |
| {"current_steps": 300, "total_steps": 446, "loss": 0.0606, "lr": 4.779595005631185e-06, "epoch": 1.3454851374088614, "percentage": 67.26, "elapsed_time": "0:34:54", "remaining_time": "0:16:59"} | |
| {"current_steps": 305, "total_steps": 446, "loss": 0.0658, "lr": 4.737651397609031e-06, "epoch": 1.3679192372406057, "percentage": 68.39, "elapsed_time": "0:36:43", "remaining_time": "0:16:58"} | |
| {"current_steps": 310, "total_steps": 446, "loss": 0.0625, "lr": 4.692274678125461e-06, "epoch": 1.39035333707235, "percentage": 69.51, "elapsed_time": "0:38:32", "remaining_time": "0:16:54"} | |
| {"current_steps": 315, "total_steps": 446, "loss": 0.049, "lr": 4.643534466306983e-06, "epoch": 1.4127874369040943, "percentage": 70.63, "elapsed_time": "0:40:17", "remaining_time": "0:16:45"} | |
| {"current_steps": 320, "total_steps": 446, "loss": 0.0551, "lr": 4.591505541710419e-06, "epoch": 1.4352215367358385, "percentage": 71.75, "elapsed_time": "0:41:57", "remaining_time": "0:16:31"} | |
| {"current_steps": 325, "total_steps": 446, "loss": 0.0553, "lr": 4.536267729592529e-06, "epoch": 1.4576556365675828, "percentage": 72.87, "elapsed_time": "0:43:42", "remaining_time": "0:16:16"} | |
| {"current_steps": 330, "total_steps": 446, "loss": 0.0585, "lr": 4.477905778438303e-06, "epoch": 1.4800897363993268, "percentage": 73.99, "elapsed_time": "0:45:25", "remaining_time": "0:15:58"} | |
| {"current_steps": 335, "total_steps": 446, "loss": 0.051, "lr": 4.416509229935797e-06, "epoch": 1.502523836231071, "percentage": 75.11, "elapsed_time": "0:47:12", "remaining_time": "0:15:38"} | |
| {"current_steps": 340, "total_steps": 446, "loss": 0.0618, "lr": 4.352172281596999e-06, "epoch": 1.5249579360628154, "percentage": 76.23, "elapsed_time": "0:49:00", "remaining_time": "0:15:16"} | |
| {"current_steps": 345, "total_steps": 446, "loss": 0.0601, "lr": 4.2849936422355235e-06, "epoch": 1.5473920358945596, "percentage": 77.35, "elapsed_time": "0:50:44", "remaining_time": "0:14:51"} | |
| {"current_steps": 350, "total_steps": 446, "loss": 0.0585, "lr": 4.215076380522832e-06, "epoch": 1.569826135726304, "percentage": 78.48, "elapsed_time": "0:52:31", "remaining_time": "0:14:24"} | |
| {"current_steps": 355, "total_steps": 446, "loss": 0.0531, "lr": 4.142527766855376e-06, "epoch": 1.5922602355580482, "percentage": 79.6, "elapsed_time": "0:54:13", "remaining_time": "0:13:54"} | |
| {"current_steps": 360, "total_steps": 446, "loss": 0.0505, "lr": 4.067459108775233e-06, "epoch": 1.6146943353897925, "percentage": 80.72, "elapsed_time": "0:55:58", "remaining_time": "0:13:22"} | |
| {"current_steps": 365, "total_steps": 446, "loss": 0.0519, "lr": 3.989985580196767e-06, "epoch": 1.6371284352215367, "percentage": 81.84, "elapsed_time": "0:57:42", "remaining_time": "0:12:48"} | |
| {"current_steps": 370, "total_steps": 446, "loss": 0.0639, "lr": 3.910226044701323e-06, "epoch": 1.659562535053281, "percentage": 82.96, "elapsed_time": "0:59:25", "remaining_time": "0:12:12"} | |
| {"current_steps": 375, "total_steps": 446, "loss": 0.0576, "lr": 3.828302873171048e-06, "epoch": 1.6819966348850253, "percentage": 84.08, "elapsed_time": "1:01:09", "remaining_time": "0:11:34"} | |
| {"current_steps": 380, "total_steps": 446, "loss": 0.0569, "lr": 3.744341756041655e-06, "epoch": 1.7044307347167695, "percentage": 85.2, "elapsed_time": "1:02:54", "remaining_time": "0:10:55"} | |
| {"current_steps": 385, "total_steps": 446, "loss": 0.0458, "lr": 3.65847151046216e-06, "epoch": 1.7268648345485138, "percentage": 86.32, "elapsed_time": "1:04:36", "remaining_time": "0:10:14"} | |
| {"current_steps": 390, "total_steps": 446, "loss": 0.0649, "lr": 3.5708238826574693e-06, "epoch": 1.749298934380258, "percentage": 87.44, "elapsed_time": "1:06:18", "remaining_time": "0:09:31"} | |
| {"current_steps": 395, "total_steps": 446, "loss": 0.048, "lr": 3.4815333457970425e-06, "epoch": 1.7717330342120023, "percentage": 88.57, "elapsed_time": "1:08:03", "remaining_time": "0:08:47"} | |
| {"current_steps": 400, "total_steps": 446, "loss": 0.055, "lr": 3.3907368936797443e-06, "epoch": 1.7941671340437466, "percentage": 89.69, "elapsed_time": "1:09:47", "remaining_time": "0:08:01"} | |
| {"current_steps": 405, "total_steps": 446, "loss": 0.0548, "lr": 3.2985738305514246e-06, "epoch": 1.8166012338754909, "percentage": 90.81, "elapsed_time": "1:12:19", "remaining_time": "0:07:19"} | |
| {"current_steps": 410, "total_steps": 446, "loss": 0.0517, "lr": 3.2051855573777045e-06, "epoch": 1.8390353337072352, "percentage": 91.93, "elapsed_time": "1:14:02", "remaining_time": "0:06:30"} | |
| {"current_steps": 415, "total_steps": 446, "loss": 0.0506, "lr": 3.110715354899873e-06, "epoch": 1.8614694335389792, "percentage": 93.05, "elapsed_time": "1:15:46", "remaining_time": "0:05:39"} | |
| {"current_steps": 420, "total_steps": 446, "loss": 0.0479, "lr": 3.015308163806741e-06, "epoch": 1.8839035333707235, "percentage": 94.17, "elapsed_time": "1:17:27", "remaining_time": "0:04:47"} | |
| {"current_steps": 425, "total_steps": 446, "loss": 0.0521, "lr": 2.919110362359727e-06, "epoch": 1.9063376332024677, "percentage": 95.29, "elapsed_time": "1:19:10", "remaining_time": "0:03:54"} | |
| {"current_steps": 430, "total_steps": 446, "loss": 0.0487, "lr": 2.8222695418123508e-06, "epoch": 1.928771733034212, "percentage": 96.41, "elapsed_time": "1:20:55", "remaining_time": "0:03:00"} | |
| {"current_steps": 435, "total_steps": 446, "loss": 0.0525, "lr": 2.724934279968692e-06, "epoch": 1.9512058328659563, "percentage": 97.53, "elapsed_time": "1:22:42", "remaining_time": "0:02:05"} | |
| {"current_steps": 440, "total_steps": 446, "loss": 0.048, "lr": 2.6272539132282413e-06, "epoch": 1.9736399326977005, "percentage": 98.65, "elapsed_time": "1:24:29", "remaining_time": "0:01:09"} | |
| {"current_steps": 445, "total_steps": 446, "loss": 0.0482, "lr": 2.5293783074668787e-06, "epoch": 1.9960740325294446, "percentage": 99.78, "elapsed_time": "1:26:14", "remaining_time": "0:00:11"} | |
| {"current_steps": 446, "total_steps": 446, "epoch": 2.0, "percentage": 100.0, "elapsed_time": "1:27:21", "remaining_time": "0:00:00"} | |