Instructions to use lxcxjxhx/MSN_demo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lxcxjxhx/MSN_demo with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="lxcxjxhx/MSN_demo",
	filename="DSR1-Q7-4S.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use lxcxjxhx/MSN_demo with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf lxcxjxhx/MSN_demo
# Run inference directly in the terminal:
llama-cli -hf lxcxjxhx/MSN_demo

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf lxcxjxhx/MSN_demo
# Run inference directly in the terminal:
llama-cli -hf lxcxjxhx/MSN_demo

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf lxcxjxhx/MSN_demo
# Run inference directly in the terminal:
./llama-cli -hf lxcxjxhx/MSN_demo

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf lxcxjxhx/MSN_demo
# Run inference directly in the terminal:
./build/bin/llama-cli -hf lxcxjxhx/MSN_demo

Use Docker

docker model run hf.co/lxcxjxhx/MSN_demo

LM Studio
Jan
Ollama
How to use lxcxjxhx/MSN_demo with Ollama:
```
ollama run hf.co/lxcxjxhx/MSN_demo
```

Unsloth Studio new

How to use lxcxjxhx/MSN_demo with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for lxcxjxhx/MSN_demo to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for lxcxjxhx/MSN_demo to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for lxcxjxhx/MSN_demo to start chatting

Docker Model Runner
How to use lxcxjxhx/MSN_demo with Docker Model Runner:
```
docker model run hf.co/lxcxjxhx/MSN_demo
```

Lemonade

How to use lxcxjxhx/MSN_demo with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull lxcxjxhx/MSN_demo

Run and chat with the model

lemonade run user.MSN_demo-{{QUANT_TAG}}

List all available models

lemonade list

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

---license: apache-2.0

Model Description

This model is a fine-tuning model based on Deepseek ai/Deepseek R1 and is currently in its early stages.
This model is committed to minimizing the model size as much as possible and providing models for a single professional field.

（This model is only used for learning purposes）

Model Details

DEMO 1:Attempt to reduce the size of the local model

DEMO 2:Reupload the usable quantized gguf model and attach a Llama.cpp compiled based on mysys2 for calling during model migration.

1.0：For the first fine-tuning and training, we attempted to use a smaller model and added a dataset from
huzaifa525 Medical_Sntelligence-Dataset_40k-Rows_of-Disease_info_Treatments_and_Sedical_QA based on the Lora training method

Downloads last month: 6

GGUF

Model size

8B params

Architecture

qwen2

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including lxcxjxhx/MSN_demo

mine

Collection

1 item • Updated Jan 23, 2025