Instructions to use inception42/Jais-2-70B-Chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use inception42/Jais-2-70B-Chat with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="inception42/Jais-2-70B-Chat")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("inception42/Jais-2-70B-Chat")
model = AutoModelForCausalLM.from_pretrained("inception42/Jais-2-70B-Chat", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use inception42/Jais-2-70B-Chat with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "inception42/Jais-2-70B-Chat"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inception42/Jais-2-70B-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/inception42/Jais-2-70B-Chat

SGLang

How to use inception42/Jais-2-70B-Chat with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "inception42/Jais-2-70B-Chat" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inception42/Jais-2-70B-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "inception42/Jais-2-70B-Chat" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inception42/Jais-2-70B-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use inception42/Jais-2-70B-Chat with Docker Model Runner:
```
docker model run hf.co/inception42/Jais-2-70B-Chat
```

Native-speaker Arabic dialect eval for Jais 2

by pferg42 - opened 29 days ago

Discussion

pferg42

29 days ago

Hi Jais team,

Congratulations on Jais 2. Building a 70B Arabic-first open-weight model tuned for Modern Standard Arabic
and regional dialects, with code-switching and informal tone, and pairing it with a safety framework
grounded in evaluation and continuous user feedback, points straight at the layer we can help with: the
hardest part of Arabic evaluation is native-speaker judgment of MSA-versus-dialect fluency and cultural
correctness across Gulf, Levantine, and Maghrebi variants, and that is a layer the English-first eval
shops cannot staff well.

That is what Boltwork runs. We are a human-data network of native speakers in long-tail and
harder-to-source languages, with quality built into the pipeline: hidden gold checks, multi-judge consensus
across distinct workers, per-worker trust scores, and a written rationale plus an error-category tag and
provenance on every judgment, so the labels are auditable rather than a bare score. Clean JSONL out.

The offer is free and low-effort: send 100 to 300 of your own items (preference pairs, output ratings, or
fluency judgments) in the Arabic variants you care most about, and we return labeled JSONL in about a week
with full QC stats. On our pilot set we run ~92% gold accuracy and ~95% inter-annotator agreement. No cost,
no commitment. The cleanest read is on the dialects where automated scoring struggles most.

One operational detail that helps us reach and keep native speakers across the Arabic-speaking world: we
pay them instantly in Bitcoin over Lightning, which sidesteps the payout and KYC friction that usually
breaks the unit economics at the long tail.

Methodology and deliverable format: https://lightningfaucet.com/boltwork/data/. Reply and I will send a
redacted sample record.

Would your evaluation or research team be open to a short conversation?

Paul Ferguson, Boltwork

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment