Text Generation
Transformers
Safetensors
English
qwen3
continued-pretraining
cpt
domain-adaptation
llm
educational
conversational
text-generation-inference
Instructions to use vinmlops/cpt-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use vinmlops/cpt-v2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="vinmlops/cpt-v2") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("vinmlops/cpt-v2") model = AutoModelForCausalLM.from_pretrained("vinmlops/cpt-v2", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use vinmlops/cpt-v2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "vinmlops/cpt-v2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "vinmlops/cpt-v2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/vinmlops/cpt-v2
- SGLang
How to use vinmlops/cpt-v2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "vinmlops/cpt-v2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "vinmlops/cpt-v2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "vinmlops/cpt-v2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "vinmlops/cpt-v2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use vinmlops/cpt-v2 with Docker Model Runner:
docker model run hf.co/vinmlops/cpt-v2
Delete base_vs_cpt.json
Browse files- base_vs_cpt.json +0 -39
base_vs_cpt.json
DELETED
|
@@ -1,39 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"method": "full",
|
| 3 |
-
"base": [
|
| 4 |
-
{
|
| 5 |
-
"prompt": "What is attention in a transformer?",
|
| 6 |
-
"answer": "How does it work? What is the role of attention in a transformer? How does attention work in a transformer? What is the difference between attention and a transformer? How does attention work in a transformer? What is the role of attention in a transformer? How does attention work in a transformer? What is the difference between attention and a transformer? How does attention work in a transformer? What is the role of attention in a transformer? How does attention work in a transformer? What is the difference between attention and a transformer? How does attention work in a transformer? What is the role of attention in a transformer? How does attention work in"
|
| 7 |
-
},
|
| 8 |
-
{
|
| 9 |
-
"prompt": "Explain what a tokenizer does, simply.",
|
| 10 |
-
"answer": "A tokenizer is a tool that breaks down a text into smaller units, such as words or phrases, for easier processing and analysis."
|
| 11 |
-
},
|
| 12 |
-
{
|
| 13 |
-
"prompt": "Give me three tips for fine-tuning an LLM.",
|
| 14 |
-
"answer": "The first tip should be about the model architecture. The second tip should be about the training data. The third tip should be about the training process. The first tip for fine-tuning an LLM is to use a model architecture that is well-suited for the specific task at hand. This may involve using a transformer-based architecture, such as BERT or GPT, or a more specialized architecture, such as a sequence-to-sequence model or a graph neural network. The choice of architecture will depend on the nature of the task and the available data.\n\nThe second tip for fine-tuning an LLM is to use high-quality training"
|
| 15 |
-
},
|
| 16 |
-
{
|
| 17 |
-
"prompt": "Who are you?",
|
| 18 |
-
"answer": "What is your background? What is your role in the company? What is your job title? What is your job description? What is your salary? What is your job performance? What is your job performance evaluation? What is your job performance rating? What is your job performance rating? What is your job performance rating? What is your job performance rating? What is your job performance rating? What is your job performance rating? What is your job performance rating? What is your job performance rating? What is your job performance rating? What is your job performance rating? What is your job performance rating? What is your job performance rating? What"
|
| 19 |
-
}
|
| 20 |
-
],
|
| 21 |
-
"cpt": [
|
| 22 |
-
{
|
| 23 |
-
"prompt": "What is attention in a transformer?",
|
| 24 |
-
"answer": "How does it work? How can we implement it in PyTorch?\n\nAttention is a mechanism in transformers that allows the model to focus on different parts of the input sequence when generating each output token. It works by computing a weighted sum of the input sequence, where the weights are determined by the attention scores between the input and output sequences. In PyTorch, we can implement attention using the `torch.nn.MultiheadAttention` module, which provides a flexible and efficient way to compute attention scores and weights. Here's an example of how to use it:\n\n```python\nimport torch\nfrom torch.nn import MultiheadAttention\n\n# Define"
|
| 25 |
-
},
|
| 26 |
-
{
|
| 27 |
-
"prompt": "Explain what a tokenizer does, simply.",
|
| 28 |
-
"answer": "Tokenizers are responsible for breaking down text into smaller units, such as words or subwords, which can then be processed by machine learning models."
|
| 29 |
-
},
|
| 30 |
-
{
|
| 31 |
-
"prompt": "Give me three tips for fine-tuning an LLM.",
|
| 32 |
-
"answer": "The first tip should be about the model architecture, the second should be about the training data, and the third should be about the training process. The tips should be in the form of a list, with each tip on a new line. The tips should be in the order of importance, with the most important tip first. The tips should be in the form of a list, with each tip on a new line. The tips should be in the form of a list, with each tip on a new line. The tips should be in the form of a list, with each tip on a new line. The tips should be in the"
|
| 33 |
-
},
|
| 34 |
-
{
|
| 35 |
-
"prompt": "Who are you?",
|
| 36 |
-
"answer": "What do you do? What do you like? What do you hate? What do you want? What do you need? What do you fear? What do you hope? What do you believe? What do you know? What do you think? What do you feel? What do you do? What do you do? What do you do? What do you do? What do you do? What do you do? What do you do? What do you do? What do you do? What do you do? What do you do? What do you do? What do you do? What do you do? What do you"
|
| 37 |
-
}
|
| 38 |
-
]
|
| 39 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|