Instructions to use deevade/DeepSeek-V2-Lite-Chat-finetuned with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use deevade/DeepSeek-V2-Lite-Chat-finetuned with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-V2-Lite-Chat")
model = PeftModel.from_pretrained(base_model, "deevade/DeepSeek-V2-Lite-Chat-finetuned")

llama-cpp-python

How to use deevade/DeepSeek-V2-Lite-Chat-finetuned with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="deevade/DeepSeek-V2-Lite-Chat-finetuned",
	filename="DeepSeek-V2-Lite-Chat-Q2-finetuned.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use deevade/DeepSeek-V2-Lite-Chat-finetuned with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf deevade/DeepSeek-V2-Lite-Chat-finetuned
# Run inference directly in the terminal:
llama cli -hf deevade/DeepSeek-V2-Lite-Chat-finetuned

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf deevade/DeepSeek-V2-Lite-Chat-finetuned
# Run inference directly in the terminal:
llama cli -hf deevade/DeepSeek-V2-Lite-Chat-finetuned

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf deevade/DeepSeek-V2-Lite-Chat-finetuned
# Run inference directly in the terminal:
./llama-cli -hf deevade/DeepSeek-V2-Lite-Chat-finetuned

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf deevade/DeepSeek-V2-Lite-Chat-finetuned
# Run inference directly in the terminal:
./build/bin/llama-cli -hf deevade/DeepSeek-V2-Lite-Chat-finetuned

Use Docker

docker model run hf.co/deevade/DeepSeek-V2-Lite-Chat-finetuned

LM Studio
Jan
Ollama
How to use deevade/DeepSeek-V2-Lite-Chat-finetuned with Ollama:
```
ollama run hf.co/deevade/DeepSeek-V2-Lite-Chat-finetuned
```

Unsloth Studio

How to use deevade/DeepSeek-V2-Lite-Chat-finetuned with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for deevade/DeepSeek-V2-Lite-Chat-finetuned to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for deevade/DeepSeek-V2-Lite-Chat-finetuned to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for deevade/DeepSeek-V2-Lite-Chat-finetuned to start chatting

Atomic Chat new
Docker Model Runner
How to use deevade/DeepSeek-V2-Lite-Chat-finetuned with Docker Model Runner:
```
docker model run hf.co/deevade/DeepSeek-V2-Lite-Chat-finetuned
```

Lemonade

How to use deevade/DeepSeek-V2-Lite-Chat-finetuned with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull deevade/DeepSeek-V2-Lite-Chat-finetuned

Run and chat with the model

lemonade run user.DeepSeek-V2-Lite-Chat-finetuned-{{QUANT_TAG}}

List all available models

lemonade list

See axolotl config

axolotl version: 0.7.0

trust_remote_code: true
adapter: lora
base_model: deepseek-ai/DeepSeek-V2-Lite-Chat
bf16: auto
bnb_config_kwargs:
  # These are default values
  llm_int8_has_fp16_weight: false
  bnb_4bit_quant_type: nf4
  bnb_4bit_use_double_quant: true
dataset_processes: 32
datasets:
- path: ./train.jsonl
  type: chat_template
  field_messages: messages
gradient_accumulation_steps: 1
gradient_checkpointing: true
learning_rate: 0.0002
lisa_layers_attribute: model.layers
load_best_model_at_end: false
lora_alpha: 16
lora_dropout: 0.05
lora_r: 8
lora_target_modules:
- q_proj
- v_proj
- k_proj
- o_proj
- gate_proj
- down_proj
- up_proj
loraplus_lr_embedding: 1.0e-06
lr_scheduler: cosine
max_prompt_len: 1024
mean_resizing_embeddings: false
micro_batch_size: 1
num_epochs: 1.0
optimizer: adamw_bnb_8bit
output_dir: ./outputs/mymodel
pretrain_multipack_attn: true
pretrain_multipack_buffer_size: 10000
qlora_sharded_model_loading: false
ray_num_workers: 1
resources_per_worker:
  GPU: 1
sample_packing_bin_size: 200
sample_packing_group_size: 100000
save_only_model: false
save_safetensors: true
sequence_len: 4096
shuffle_merged_datasets: true
skip_prepare_dataset: false
strict: false
train_on_inputs: false
trl:
  log_completions: false
  ref_model_mixup_alpha: 0.9
  ref_model_sync_steps: 64
  sync_ref_model: false
  use_vllm: false
  vllm_device: auto
  vllm_dtype: auto
  vllm_gpu_memory_utilization: 0.9
use_ray: false
val_set_size: 0.0
weight_decay: 0.0

outputs/mymodel

This model is a fine-tuned version of deepseek-ai/DeepSeek-V2-Lite-Chat on the ./train.jsonl dataset.

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0002
train_batch_size: 1
eval_batch_size: 1
seed: 42
optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 27
num_epochs: 1.0

Training results

Framework versions

PEFT 0.14.0
Transformers 4.48.3
Pytorch 2.4.0+cu121
Datasets 3.2.0
Tokenizers 0.21.1

Downloads last month: 38

GGUF

Model size

16B params

Architecture

deepseek2

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for deevade/DeepSeek-V2-Lite-Chat-finetuned

Base model

deepseek-ai/DeepSeek-V2-Lite-Chat

Adapter

(5)

this model