Instructions to use ubitech-edg/mistral-12b-cpt with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ubitech-edg/mistral-12b-cpt with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ubitech-edg/mistral-12b-cpt")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ubitech-edg/mistral-12b-cpt")
model = AutoModelForCausalLM.from_pretrained("ubitech-edg/mistral-12b-cpt")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

NeMo
How to use ubitech-edg/mistral-12b-cpt with NeMo:
```
# tag did not correspond to a valid NeMo domain.
```
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use ubitech-edg/mistral-12b-cpt with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ubitech-edg/mistral-12b-cpt"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ubitech-edg/mistral-12b-cpt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ubitech-edg/mistral-12b-cpt

SGLang

How to use ubitech-edg/mistral-12b-cpt with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ubitech-edg/mistral-12b-cpt" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ubitech-edg/mistral-12b-cpt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ubitech-edg/mistral-12b-cpt" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ubitech-edg/mistral-12b-cpt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ubitech-edg/mistral-12b-cpt with Docker Model Runner:
```
docker model run hf.co/ubitech-edg/mistral-12b-cpt
```

Mistral 12B — CPT (Continual Pretraining with LoRA)

Model type: Causal Language Model
Base model: mistralai/Mistral-Nemo-Instruct-2407
License: Apache 2.0
Framework: Axolotl

Overview

mistral-12b-cpt is a continual-pretrained version of the Mistral-12B Nemo Instruct model.
This CPT phase extends the model’s factual and energy domain understanding using scientific, governmental, news, and encyclopedic text.

Training was executed on the Leonardo EuroHPC system using Axolotl with DeepSpeed ZeRO-1 for efficient large-scale distributed fine-tuning.

Training Setup

Objective: Unsupervised continual pretraining (language modeling)
Adapter type: LoRA
Precision: bfloat16
Hardware: 8 nodes × 2 × NVIDIA A100 64 GB GPUs
Framework: Axolotl + DeepSpeed + PyTorch 2.5.1 + CUDA 12.1
Runtime: 24 h
Checkpoints: 5 per epoch

Dataset

Dataset	Description
`arxiv.jsonl`	Scientific and technical papers
`gov.jsonl`	Government and policy documents
`news.jsonl`	News articles
`wiki.jsonl`	Wikipedia text

Hyperparameters

Parameter	Value
Sequence length	2048
Micro batch size	2
Gradient accumulation	2
Epochs	10
Max steps	10000
Learning rate	0.0002
LR scheduler	cosine
Optimizer	AdamW (8-bit)
Warmup steps	10
Weight decay	0.0
LoRA rank (r)	16
LoRA alpha	32
LoRA dropout	0.05
LoRA targets	q_proj, k_proj, v_proj, o_proj
Gradient checkpointing	✅
Flash attention	✅
Loss watchdog (threshold/patience)	5.0 / 3

Tokenizer

Tokenizer type: AutoTokenizer
Pad token: <|end_of_text|>

Downloads last month: 2

Model tree for ubitech-edg/mistral-12b-cpt

Base model

mistralai/Mistral-Nemo-Base-2407

Finetuned

mistralai/Mistral-Nemo-Instruct-2407

Adapter

(66)

this model

Adapters

1 model

ubitech-edg
/

mistral-12b-cpt