Instructions to use ByteDance-Seed/Cola2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ByteDance-Seed/Cola2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ByteDance-Seed/Cola2", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("ByteDance-Seed/Cola2", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ByteDance-Seed/Cola2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ByteDance-Seed/Cola2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ByteDance-Seed/Cola2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ByteDance-Seed/Cola2

SGLang

How to use ByteDance-Seed/Cola2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ByteDance-Seed/Cola2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ByteDance-Seed/Cola2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ByteDance-Seed/Cola2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ByteDance-Seed/Cola2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ByteDance-Seed/Cola2 with Docker Model Runner:
```
docker model run hf.co/ByteDance-Seed/Cola2
```

Cola2 / eval_data /manifest.json

HongcanGuo04

Upload folder using huggingface_hub

a310630 verified about 10 hours ago

Raw History Blame Contribute Delete

1.37 kB

{
  "format": "cola2-generate-task-jsonl-v1",
  "tasks": {
    "arc_easy": {
      "file": "arc_easy.jsonl",
      "samples": 2375,
      "sha256": "82ca2652c55e896130fa0ca4adedb5ce404d8bb16d118b4b1b382bd3907c4024"
    },
    "hellaswag": {
      "file": "hellaswag.jsonl",
      "samples": 10042,
      "sha256": "e45c5a826dccfaa2f6f4b3db84ac121e33e3f71703f43b18b38b58d6325db001"
    },
    "lambada": {
      "file": "lambada.jsonl",
      "samples": 5153,
      "sha256": "12006b30c189194ddcedfc6ad11c57a7cd107a97340b85199e9fe1766dae7f03"
    },
    "mmlu": {
      "file": "mmlu.jsonl",
      "samples": 14042,
      "sha256": "ce429dd9fdc44d56f2299b63cc934df1d88b8cdb7709c72ee24d8323aba95fdd"
    },
    "narrativeqa": {
      "file": "narrativeqa.jsonl",
      "samples": 1000,
      "sha256": "28daf47f744fc956892402e62a6ac9c41ca4babc8edf5f7105cfb6bb0eba5ff8"
    },
    "race": {
      "file": "race.jsonl",
      "samples": 4887,
      "sha256": "1d742781cac6f9237a33b49654815e119ac6f845ef7613ad9b9e0860b6596c0d"
    },
    "story_cloze": {
      "file": "story_cloze.jsonl",
      "samples": 468,
      "sha256": "d6555a855ce1a23f23ac2474a1e9dfda7239ec368b6a5962434db5c2f678c014"
    },
    "truthfulqa": {
      "file": "truthfulqa.jsonl",
      "samples": 817,
      "sha256": "ccef0b9ecae13cc5b83c24430c6cfee37d3872455b428c4630bae434a27ec935"
    }
  }
}

	{
	"format": "cola2-generate-task-jsonl-v1",
	"tasks": {
	"arc_easy": {
	"file": "arc_easy.jsonl",
	"samples": 2375,
	"sha256": "82ca2652c55e896130fa0ca4adedb5ce404d8bb16d118b4b1b382bd3907c4024"
	},
	"hellaswag": {
	"file": "hellaswag.jsonl",
	"samples": 10042,
	"sha256": "e45c5a826dccfaa2f6f4b3db84ac121e33e3f71703f43b18b38b58d6325db001"
	},
	"lambada": {
	"file": "lambada.jsonl",
	"samples": 5153,
	"sha256": "12006b30c189194ddcedfc6ad11c57a7cd107a97340b85199e9fe1766dae7f03"
	},
	"mmlu": {
	"file": "mmlu.jsonl",
	"samples": 14042,
	"sha256": "ce429dd9fdc44d56f2299b63cc934df1d88b8cdb7709c72ee24d8323aba95fdd"
	},
	"narrativeqa": {
	"file": "narrativeqa.jsonl",
	"samples": 1000,
	"sha256": "28daf47f744fc956892402e62a6ac9c41ca4babc8edf5f7105cfb6bb0eba5ff8"
	},
	"race": {
	"file": "race.jsonl",
	"samples": 4887,
	"sha256": "1d742781cac6f9237a33b49654815e119ac6f845ef7613ad9b9e0860b6596c0d"
	},
	"story_cloze": {
	"file": "story_cloze.jsonl",
	"samples": 468,
	"sha256": "d6555a855ce1a23f23ac2474a1e9dfda7239ec368b6a5962434db5c2f678c014"
	},
	"truthfulqa": {
	"file": "truthfulqa.jsonl",
	"samples": 817,
	"sha256": "ccef0b9ecae13cc5b83c24430c6cfee37d3872455b428c4630bae434a27ec935"
	}
	}
	}