Instructions to use prasannaJagadesh/my-model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use prasannaJagadesh/my-model with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="prasannaJagadesh/my-model", filename="gguf/SmolLM2-360M-Instruct.Q4_K_S.gguf", )
llm.create_chat_completion( messages = "No input example has been defined for this model task." )
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use prasannaJagadesh/my-model with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf prasannaJagadesh/my-model:Q4_K_S # Run inference directly in the terminal: llama cli -hf prasannaJagadesh/my-model:Q4_K_S
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf prasannaJagadesh/my-model:Q4_K_S # Run inference directly in the terminal: llama cli -hf prasannaJagadesh/my-model:Q4_K_S
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf prasannaJagadesh/my-model:Q4_K_S # Run inference directly in the terminal: ./llama-cli -hf prasannaJagadesh/my-model:Q4_K_S
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf prasannaJagadesh/my-model:Q4_K_S # Run inference directly in the terminal: ./build/bin/llama-cli -hf prasannaJagadesh/my-model:Q4_K_S
Use Docker
docker model run hf.co/prasannaJagadesh/my-model:Q4_K_S
- LM Studio
- Jan
- Ollama
How to use prasannaJagadesh/my-model with Ollama:
ollama run hf.co/prasannaJagadesh/my-model:Q4_K_S
- Unsloth Studio
How to use prasannaJagadesh/my-model with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for prasannaJagadesh/my-model to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for prasannaJagadesh/my-model to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for prasannaJagadesh/my-model to start chatting
- Atomic Chat new
- Docker Model Runner
How to use prasannaJagadesh/my-model with Docker Model Runner:
docker model run hf.co/prasannaJagadesh/my-model:Q4_K_S
- Lemonade
How to use prasannaJagadesh/my-model with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull prasannaJagadesh/my-model:Q4_K_S
Run and chat with the model
lemonade run user.my-model-Q4_K_S
List all available models
lemonade list
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
Prasanna-SmolLM-360M-3.1
A fine-tuned version of SmolLM2-360M-Instruct trained to be a personal AI assistant that answers questions about my professional background, projects, and skills.
This model is designed to serve as a personal AI assistant on a portfolio website and it's trained for only training purpose of the finetuning & Reward model and It answers questions specifically about myself and refuses off-topic or inappropriate requests.
Model Details
| Parameter | Value |
|---|---|
| Base Model | HuggingFaceTB/SmolLM2-360M-Instruct |
| Parameters | 360M |
| Max Sequence Length | 1024 |
| Fine-Tuning Method | LoRA (via Unsloth) |
| Merge Method | merged_16bit |
| GGUF Quantizations | q8_0 |
LoRA Configuration
| Parameter | Value |
|---|---|
| Rank (r) | 16 |
| Alpha | 32 |
| Dropout | 0.05 |
| Bias | none |
| Gradient Checkpointing | unsloth |
Training Arguments
| Parameter | Value |
|---|---|
| Batch Size (per device) | 8 |
| Gradient Accumulation Steps | 2 |
| Effective Batch Size | 16 |
| Epochs | 3 |
| Learning Rate | 2e-4 |
| Weight Decay | 0.01 |
| LR Scheduler | cosine |
| Optimizer | adamw_8bit |
| Precision | bf16 (if supported, else fp16) |
| Packing | enabled |
| Dataset Workers | 2 |
Dataset
~2K samples curated and reviewed manually, covering:
- Biography & identity
- career & workExp
- technical skills
- tech journey
- contacts & social media
- some Refusal for refuse questions if asked not about me
- NFSW to prevent safety measure
Format
{
"messages": [
{
"role": "system",
"content": "You are Prasanna's AI Assistant. You answer questions about his professional background, projects, and skills."
},
{
"role": "user",
"content": "Who is Prasanna?"
},
{
"role": "assistant",
"content": "Prasanna is a driven Software Engineer based in Chennai, India..."
}
]
}
Usage
Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("prasannaJagadesh/Prasanna-SmolLM-360M-3.1")
tokenizer = AutoTokenizer.from_pretrained("prasannaJagadesh/Prasanna-SmolLM-360M-3.1")
messages = [
{"role": "system", "content": "You are Prasanna's AI Assistant. You answer questions about his professional background, projects, and skills."},
{"role": "user", "content": "Tell me about Prasanna."},
]
input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Warning Limitations
- Only knows about myself, not a general-purpose assistant
- Small model (360M params) very limited reasoning depth compared to larger models
- Best suited for CPU inference on constrained environments (4-8 GB RAM)
- Downloads last month
- 6