Text Generation
Transformers
Safetensors
qwen2
phai-ide
science
code
tool-use
sft
lora
conversational
text-generation-inference
Instructions to use AItonomy/PhAI-IDE-72B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AItonomy/PhAI-IDE-72B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="AItonomy/PhAI-IDE-72B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("AItonomy/PhAI-IDE-72B") model = AutoModelForCausalLM.from_pretrained("AItonomy/PhAI-IDE-72B", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use AItonomy/PhAI-IDE-72B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "AItonomy/PhAI-IDE-72B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AItonomy/PhAI-IDE-72B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/AItonomy/PhAI-IDE-72B
- SGLang
How to use AItonomy/PhAI-IDE-72B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "AItonomy/PhAI-IDE-72B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AItonomy/PhAI-IDE-72B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "AItonomy/PhAI-IDE-72B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AItonomy/PhAI-IDE-72B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use AItonomy/PhAI-IDE-72B with Docker Model Runner:
docker model run hf.co/AItonomy/PhAI-IDE-72B
Simplify published comparisons to complete benchmark configurations
Browse filesUse six compact benchmark rows with twelve similarly sized published-model references. Remove item-count annotations and retain comparisons backed by complete benchmark configurations. Preserve the ScienceAccelBench section unchanged.
README.md
CHANGED
|
@@ -54,18 +54,14 @@ Scores (%), grouped by benchmark and model size. Each reference entry gives its
|
|
| 54 |
|
| 55 |
| PhAI-IDE | Benchmark | Score | Reference models: score (difference) |
|
| 56 |
| --- | --- | ---: | --- |
|
| 57 |
-
| 4B | GSM8K | **89.06†** | [Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct#instruction-tuned-models) (3B): 77.7 (**+11.36**); [Phi-3-mini](https://arxiv.org/html/2404.14219v1) (3.8B): 82.5 (**+6.56**); [Qwen2.5-3B-Instruct](https://arxiv.org/html/2412.15115v2#S5.SS2.SSS1) (3B): 86.7 (**+2.36**); [WizardMath-Llama-RL (Llama 3.2)](https://arxiv.org/html/2308.09583v2) (3B): 85.5 (**+3.56**) |
|
| 58 |
| 4B | BBH multistep-arithmetic-two | **97.60** | [Llama-3.2-3B-Instruct](https://huggingface.co/spaces/steampunque/benchlm/blob/d45e8600172857935610426f797a4429f2f136d6/README.md) (3.21B): 53.2 (**+44.40**); [Phi-3.5-mini-8k-instruct](https://huggingface.co/spaces/steampunque/benchlm/blob/d45e8600172857935610426f797a4429f2f136d6/README.md) (3.82B): 95.6 (**+2.00**) |
|
| 59 |
-
| 4B | MATH-500 | **87.50†** | [Llama-3.2-3B-Instruct](https://huggingface.co/microsoft/Phi-4-mini-reasoning/blob/661e72224c678e7360429603fd898712ee0c74b9/README.md) (3B): 44.4 (**+43.10**); [Phi-4-Mini](https://huggingface.co/microsoft/Phi-4-mini-reasoning/blob/661e72224c678e7360429603fd898712ee0c74b9/README.md) (3.8B): 71.8 (**+15.70**) |
|
| 60 |
-
| 9B | GSM8K | **92.97†** | [Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct#instruction-tuned-models) (8B): 84.5 (**+8.47**); [Qwen2.5-7B-Instruct](https://arxiv.org/html/2412.15115v2#S5.SS2.SSS1) (7B): 91.6 (**+1.37**); [Gemma-2-9B-IT](https://arxiv.org/html/2412.15115v2#S5.SS2.SSS1) (9B): 76.7 (**+16.27**); [SciTulu-7B](https://arxiv.org/html/2406.07835v2#A3) (7B): 29.5 (**+63.47**) |
|
| 61 |
| 9B | BBH word-sorting | **60.40** | [Llama-3.1-8B-Instruct](https://huggingface.co/spaces/steampunque/benchlm/blob/d45e8600172857935610426f797a4429f2f136d6/README.md) (8.03B): 51.2 (**+9.20**); [Qwen2.5-7B-Instruct](https://huggingface.co/spaces/steampunque/benchlm/blob/d45e8600172857935610426f797a4429f2f136d6/README.md) (7.62B): 15.6 (**+44.80**) |
|
| 62 |
| 9B | MATH-500 | **92.20** | [InternLM3-8B-Instruct](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm3-8b-instruct-gptq-int4) (8B): 83 (**+9.20**); [Qwen2.5-7B-Instruct](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm3-8b-instruct-gptq-int4) (7B): 72.4 (**+19.80**); [Llama-3.1-8B-Instruct](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm3-8b-instruct-gptq-int4) (8B): 48.4 (**+43.80**) |
|
| 63 |
-
| 72B | GSM8K | **93.75†** | [Llama-3-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct#instruction-tuned-models) (70B): 93 (**+0.75**); [DeepSeek-LLM-67B-Chat](https://github.com/deepseek-ai/DeepSeek-LLM#3-evaluation-results) (67B): 84.1 (**+9.65**); [Qwen2-72B-Instruct](https://arxiv.org/html/2412.15115v2#S5.SS2.SSS1) (72B): 93.2 (**+0.55**); [SciTulu-70B](https://arxiv.org/html/2406.07835v2#A3) (70B): 67.5 (**+26.25**); [WizardMath-Llama-RL (Llama 2)](https://arxiv.org/html/2308.09583v2) (70B): 92.8 (**+0.95**) |
|
| 64 |
| 72B | AQuA-RAT | **77.56** | [Llama-2-70B-Chat](https://openreview.net/pdf?id=FvfhHucpLd) (70B): 31.32 (**+46.24**) |
|
| 65 |
| 72B | ARC-Easy | **84.64** | [Llama-2-70B](https://github.com/deepseek-ai/DeepSeek-LLM/blob/main/evaluation/more_results.md) (70B): 76.5 (**+8.14**); [DeepSeek-LLM-67B-Chat](https://github.com/deepseek-ai/DeepSeek-LLM/blob/main/evaluation/more_results.md) (67B): 81.6 (**+3.04**) |
|
| 66 |
| 72B | ARC-Challenge | **64.42** | [Llama-2-70B](https://github.com/deepseek-ai/DeepSeek-LLM/blob/main/evaluation/more_results.md) (70B): 59.5 (**+4.92**); [DeepSeek-LLM-67B-Chat](https://github.com/deepseek-ai/DeepSeek-LLM/blob/main/evaluation/more_results.md) (67B): 64.1 (**+0.32**) |
|
| 67 |
|
| 68 |
-
|
| 69 |
|
| 70 |
## Quick start
|
| 71 |
|
|
|
|
| 54 |
|
| 55 |
| PhAI-IDE | Benchmark | Score | Reference models: score (difference) |
|
| 56 |
| --- | --- | ---: | --- |
|
|
|
|
| 57 |
| 4B | BBH multistep-arithmetic-two | **97.60** | [Llama-3.2-3B-Instruct](https://huggingface.co/spaces/steampunque/benchlm/blob/d45e8600172857935610426f797a4429f2f136d6/README.md) (3.21B): 53.2 (**+44.40**); [Phi-3.5-mini-8k-instruct](https://huggingface.co/spaces/steampunque/benchlm/blob/d45e8600172857935610426f797a4429f2f136d6/README.md) (3.82B): 95.6 (**+2.00**) |
|
|
|
|
|
|
|
| 58 |
| 9B | BBH word-sorting | **60.40** | [Llama-3.1-8B-Instruct](https://huggingface.co/spaces/steampunque/benchlm/blob/d45e8600172857935610426f797a4429f2f136d6/README.md) (8.03B): 51.2 (**+9.20**); [Qwen2.5-7B-Instruct](https://huggingface.co/spaces/steampunque/benchlm/blob/d45e8600172857935610426f797a4429f2f136d6/README.md) (7.62B): 15.6 (**+44.80**) |
|
| 59 |
| 9B | MATH-500 | **92.20** | [InternLM3-8B-Instruct](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm3-8b-instruct-gptq-int4) (8B): 83 (**+9.20**); [Qwen2.5-7B-Instruct](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm3-8b-instruct-gptq-int4) (7B): 72.4 (**+19.80**); [Llama-3.1-8B-Instruct](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm3-8b-instruct-gptq-int4) (8B): 48.4 (**+43.80**) |
|
|
|
|
| 60 |
| 72B | AQuA-RAT | **77.56** | [Llama-2-70B-Chat](https://openreview.net/pdf?id=FvfhHucpLd) (70B): 31.32 (**+46.24**) |
|
| 61 |
| 72B | ARC-Easy | **84.64** | [Llama-2-70B](https://github.com/deepseek-ai/DeepSeek-LLM/blob/main/evaluation/more_results.md) (70B): 76.5 (**+8.14**); [DeepSeek-LLM-67B-Chat](https://github.com/deepseek-ai/DeepSeek-LLM/blob/main/evaluation/more_results.md) (67B): 81.6 (**+3.04**) |
|
| 62 |
| 72B | ARC-Challenge | **64.42** | [Llama-2-70B](https://github.com/deepseek-ai/DeepSeek-LLM/blob/main/evaluation/more_results.md) (70B): 59.5 (**+4.92**); [DeepSeek-LLM-67B-Chat](https://github.com/deepseek-ai/DeepSeek-LLM/blob/main/evaluation/more_results.md) (67B): 64.1 (**+0.32**) |
|
| 63 |
|
| 64 |
+
Reference scores come from the linked publications, model cards, and independent evaluation reports; evaluation settings and sample counts vary by source. Differences describe reported scores across evaluations, rather than matched-protocol head-to-head gains. BBH entries refer to the named tasks.
|
| 65 |
|
| 66 |
## Quick start
|
| 67 |
|