Text Generation
Transformers
GGUF
English
llama
llama3
dementia
healthcare
medical
caregiving
alzheimers
memory-care
assistant
fine-tuned
specialized
conversational
4-bit precision
gptq
Instructions to use splendidcomputer/new-dim with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use splendidcomputer/new-dim with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="splendidcomputer/new-dim") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForMultimodalLM tokenizer = AutoTokenizer.from_pretrained("splendidcomputer/new-dim") model = AutoModelForMultimodalLM.from_pretrained("splendidcomputer/new-dim") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - llama-cpp-python
How to use splendidcomputer/new-dim with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="splendidcomputer/new-dim", filename="model.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": "What is the capital of France?" } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use splendidcomputer/new-dim with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf splendidcomputer/new-dim # Run inference directly in the terminal: llama-cli -hf splendidcomputer/new-dim
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf splendidcomputer/new-dim # Run inference directly in the terminal: llama-cli -hf splendidcomputer/new-dim
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf splendidcomputer/new-dim # Run inference directly in the terminal: ./llama-cli -hf splendidcomputer/new-dim
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf splendidcomputer/new-dim # Run inference directly in the terminal: ./build/bin/llama-cli -hf splendidcomputer/new-dim
Use Docker
docker model run hf.co/splendidcomputer/new-dim
- LM Studio
- Jan
- vLLM
How to use splendidcomputer/new-dim with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "splendidcomputer/new-dim" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "splendidcomputer/new-dim", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/splendidcomputer/new-dim
- SGLang
How to use splendidcomputer/new-dim with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "splendidcomputer/new-dim" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "splendidcomputer/new-dim", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "splendidcomputer/new-dim" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "splendidcomputer/new-dim", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use splendidcomputer/new-dim with Ollama:
ollama run hf.co/splendidcomputer/new-dim
- Unsloth Studio
How to use splendidcomputer/new-dim with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for splendidcomputer/new-dim to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for splendidcomputer/new-dim to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for splendidcomputer/new-dim to start chatting
- Docker Model Runner
How to use splendidcomputer/new-dim with Docker Model Runner:
docker model run hf.co/splendidcomputer/new-dim
- Lemonade
How to use splendidcomputer/new-dim with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull splendidcomputer/new-dim
Run and chat with the model
lemonade run user.new-dim-{{QUANT_TAG}}List all available models
lemonade list
File size: 4,990 Bytes
fa9878d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 | # Hugging Face Upload Guide
## Prerequisites
1. **Hugging Face Account**: Create an account at https://huggingface.co
2. **Git LFS**: Install Git Large File Storage for handling large model files
```bash
git lfs install
```
3. **Hugging Face CLI**: Install the Hugging Face CLI
```bash
pip install huggingface_hub[cli]
```
## Step 1: Create a New Model Repository
1. Go to https://huggingface.co/new
2. Choose "Model" as the repository type
3. Name your repository (e.g., `llama3-dementia-care`)
4. Set it to Public or Private as desired
5. Click "Create Repository"
## Step 2: Clone Your Repository
```bash
git clone https://huggingface.co/your-username/llama3-dementia-care
cd llama3-dementia-care
```
## Step 3: Copy Repository Files
Copy all the files from this directory to your cloned Hugging Face repository:
```bash
# From your LLAMA3_DEMENTIA_SHARE directory
cp README.md /path/to/your-username/llama3-dementia-care/
cp config.json /path/to/your-username/llama3-dementia-care/
cp tokenizer_config.json /path/to/your-username/llama3-dementia-care/
cp special_tokens_map.json /path/to/your-username/llama3-dementia-care/
cp Modelfile /path/to/your-username/llama3-dementia-care/
cp model_info.json /path/to/your-username/llama3-dementia-care/
cp usage_example.py /path/to/your-username/llama3-dementia-care/
cp requirements.txt /path/to/your-username/llama3-dementia-care/
cp NOTICE /path/to/your-username/llama3-dementia-care/
cp .gitignore /path/to/your-username/llama3-dementia-care/
```
## Step 4: Add Model Weights (Critical Step)
This is the most complex part. You have several options:
### Option A: Convert Ollama Model (Recommended)
1. Run the export script:
```bash
./export_model.sh
```
2. Use a conversion tool like `ollama-export` or similar to convert your Ollama model to PyTorch format
3. Common conversion commands:
```bash
# Example conversion (may vary based on tool)
ollama export llama3-dementia-care:latest model.gguf
# Then convert GGUF to PyTorch format using appropriate tools
```
### Option B: Use Base Model + Fine-tuning Weights
1. Download the base Llama 3 8B model from Hugging Face
2. Add your fine-tuning weights/adapters
3. Upload the complete model
### Option C: Re-create the Model
1. Start with the official Llama 3 8B model
2. Fine-tune it using your dementia care dataset
3. Upload the fine-tuned result
## Step 5: Set up Git LFS for Large Files
```bash
cd your-username/llama3-dementia-care
git lfs track "*.bin"
git lfs track "*.safetensors"
git lfs track "*.gguf"
git add .gitattributes
```
## Step 6: Commit and Push
```bash
git add .
git commit -m "Add Llama 3 Dementia Care Assistant model"
git push
```
## Step 7: Update Model Card
1. Go to your model page on Hugging Face
2. Edit the README.md if needed
3. Add any additional information about training data, evaluation metrics, etc.
4. Test the inference widget with sample prompts
## Sample Model Files You Need
For a complete Hugging Face model, you typically need:
- ✅ `README.md` (with YAML frontmatter)
- ✅ `config.json`
- ✅ `tokenizer_config.json`
- ✅ `special_tokens_map.json`
- ⚠️ `pytorch_model.bin` or `model.safetensors` (converted model weights)
- ⚠️ `tokenizer.model` or `tokenizer.json` (if needed)
- ✅ Optional: `generation_config.json`, `training_args.bin`
## Testing Your Model
After upload, test your model:
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "your-username/llama3-dementia-care"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
# Test with a dementia care question
prompt = "What are some strategies for managing sundown syndrome?"
# ... rest of inference code
```
## Troubleshooting
### Common Issues:
1. **Large file errors**: Make sure Git LFS is properly configured
2. **Token errors**: Use `huggingface-cli login` to authenticate
3. **Model loading errors**: Ensure all config files are correct
4. **Inference issues**: Test the model locally before uploading
### Getting Help:
- Hugging Face Documentation: https://huggingface.co/docs
- Community Forum: https://discuss.huggingface.co
- Discord: https://discord.gg/huggingface
## Important Notes
1. **License Compliance**: Ensure your model respects the Llama 3 Community License
2. **Attribution**: Always include "Built with Meta Llama 3" as required
3. **Medical Disclaimers**: Include appropriate disclaimers for medical/health content
4. **Model Safety**: Test thoroughly before public release
## Final Checklist
- [ ] Repository created on Hugging Face
- [ ] All configuration files uploaded
- [ ] Model weights converted and uploaded
- [ ] README.md is complete and accurate
- [ ] License information is included
- [ ] Model card is comprehensive
- [ ] Inference widget works
- [ ] Example usage is provided
- [ ] Appropriate disclaimers are included
Good luck with your model upload! 🚀
|