Instructions to use lianghsun/Llama-3.1-DeepFox-70B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lianghsun/Llama-3.1-DeepFox-70B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lianghsun/Llama-3.1-DeepFox-70B-Instruct")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("lianghsun/Llama-3.1-DeepFox-70B-Instruct", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use lianghsun/Llama-3.1-DeepFox-70B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lianghsun/Llama-3.1-DeepFox-70B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/Llama-3.1-DeepFox-70B-Instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/lianghsun/Llama-3.1-DeepFox-70B-Instruct

SGLang

How to use lianghsun/Llama-3.1-DeepFox-70B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lianghsun/Llama-3.1-DeepFox-70B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/Llama-3.1-DeepFox-70B-Instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lianghsun/Llama-3.1-DeepFox-70B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/Llama-3.1-DeepFox-70B-Instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use lianghsun/Llama-3.1-DeepFox-70B-Instruct with Docker Model Runner:
```
docker model run hf.co/lianghsun/Llama-3.1-DeepFox-70B-Instruct
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Model Card for Llama-3.1-DeepFox-70B-Instruct

Llama-3.1-DeepFox-70B-Instruct 是以 Llama 3.1 70B 為基底，針對中華民國台灣語境完成持續預訓練與指令微調的繁體中文大模型，提供 70B 規模下的台灣語境對話與專業領域回答能力。

⚠️ 規格重點：本模型為 70B 參數、純文本單模態，部署需多張高記憶體 GPU 或量化版本。

Model Details

70B 級模型在能力上明顯優於中小型模型，但繁中與台灣語境的覆蓋仍是大模型的弱點。Llama-3.1-DeepFox-70B-Instruct 把繁中與台灣本地語料注入 Llama-3.1-70B，並完成指令微調，目標是在大型模型規模下提供穩定可用的繁中對話與在地語境理解能力。

核心特點 (Key Features)

70B 級繁中對話：能力顯著優於中小型模型，適合高品質要求的應用情境。
台灣語境對齊：訓練資料以繁中與台灣常見任務為主。
企業級部署：模型容量可支撐長文本、多輪對話與複雜推理。

Model Description

Developed by: Liang Hsun Huang
Funded by: APMIC
Base model: meta-llama/Llama-3.1-70B
Model type: LlamaForCausalLM (Transformers)
Language(s) (NLP): Traditional Chinese, English
License: llama3.1

Model Sources

Repository: lianghsun/Llama-3.1-DeepFox-70B-Instruct

Citation

@misc{llama_3_1_deepfox_70b_instruct,
  title        = {Llama-3.1-DeepFox-70B-Instruct: A 70B Traditional Chinese Instruction-Tuned Llama-3.1 Model for Taiwan},
  author       = {Huang, Liang Hsun},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/lianghsun/Llama-3.1-DeepFox-70B-Instruct}}
}

Acknowledge

特此感謝 APMIC 的算力支援。

Model Card Authors

Huang Liang Hsun

Model Card Contact

Huang Liang Hsun

Downloads last month: -; Downloads are not tracked for this model. How to track

Model tree for lianghsun/Llama-3.1-DeepFox-70B-Instruct

Base model

meta-llama/Llama-3.1-70B

Finetuned

(59)

this model