Text Generation
Transformers
GGUF
English
llama
llama3
dementia
healthcare
medical
caregiving
alzheimers
memory-care
assistant
fine-tuned
specialized
conversational
4-bit precision
gptq
Instructions to use splendidcomputer/new-dim with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use splendidcomputer/new-dim with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="splendidcomputer/new-dim") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForMultimodalLM tokenizer = AutoTokenizer.from_pretrained("splendidcomputer/new-dim") model = AutoModelForMultimodalLM.from_pretrained("splendidcomputer/new-dim") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - llama-cpp-python
How to use splendidcomputer/new-dim with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="splendidcomputer/new-dim", filename="model.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": "What is the capital of France?" } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use splendidcomputer/new-dim with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf splendidcomputer/new-dim # Run inference directly in the terminal: llama-cli -hf splendidcomputer/new-dim
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf splendidcomputer/new-dim # Run inference directly in the terminal: llama-cli -hf splendidcomputer/new-dim
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf splendidcomputer/new-dim # Run inference directly in the terminal: ./llama-cli -hf splendidcomputer/new-dim
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf splendidcomputer/new-dim # Run inference directly in the terminal: ./build/bin/llama-cli -hf splendidcomputer/new-dim
Use Docker
docker model run hf.co/splendidcomputer/new-dim
- LM Studio
- Jan
- vLLM
How to use splendidcomputer/new-dim with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "splendidcomputer/new-dim" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "splendidcomputer/new-dim", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/splendidcomputer/new-dim
- SGLang
How to use splendidcomputer/new-dim with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "splendidcomputer/new-dim" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "splendidcomputer/new-dim", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "splendidcomputer/new-dim" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "splendidcomputer/new-dim", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use splendidcomputer/new-dim with Ollama:
ollama run hf.co/splendidcomputer/new-dim
- Unsloth Studio
How to use splendidcomputer/new-dim with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for splendidcomputer/new-dim to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for splendidcomputer/new-dim to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for splendidcomputer/new-dim to start chatting
- Docker Model Runner
How to use splendidcomputer/new-dim with Docker Model Runner:
docker model run hf.co/splendidcomputer/new-dim
- Lemonade
How to use splendidcomputer/new-dim with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull splendidcomputer/new-dim
Run and chat with the model
lemonade run user.new-dim-{{QUANT_TAG}}List all available models
lemonade list
| # Hugging Face Upload Guide | |
| ## Prerequisites | |
| 1. **Hugging Face Account**: Create an account at https://huggingface.co | |
| 2. **Git LFS**: Install Git Large File Storage for handling large model files | |
| ```bash | |
| git lfs install | |
| ``` | |
| 3. **Hugging Face CLI**: Install the Hugging Face CLI | |
| ```bash | |
| pip install huggingface_hub[cli] | |
| ``` | |
| ## Step 1: Create a New Model Repository | |
| 1. Go to https://huggingface.co/new | |
| 2. Choose "Model" as the repository type | |
| 3. Name your repository (e.g., `llama3-dementia-care`) | |
| 4. Set it to Public or Private as desired | |
| 5. Click "Create Repository" | |
| ## Step 2: Clone Your Repository | |
| ```bash | |
| git clone https://huggingface.co/your-username/llama3-dementia-care | |
| cd llama3-dementia-care | |
| ``` | |
| ## Step 3: Copy Repository Files | |
| Copy all the files from this directory to your cloned Hugging Face repository: | |
| ```bash | |
| # From your LLAMA3_DEMENTIA_SHARE directory | |
| cp README.md /path/to/your-username/llama3-dementia-care/ | |
| cp config.json /path/to/your-username/llama3-dementia-care/ | |
| cp tokenizer_config.json /path/to/your-username/llama3-dementia-care/ | |
| cp special_tokens_map.json /path/to/your-username/llama3-dementia-care/ | |
| cp Modelfile /path/to/your-username/llama3-dementia-care/ | |
| cp model_info.json /path/to/your-username/llama3-dementia-care/ | |
| cp usage_example.py /path/to/your-username/llama3-dementia-care/ | |
| cp requirements.txt /path/to/your-username/llama3-dementia-care/ | |
| cp NOTICE /path/to/your-username/llama3-dementia-care/ | |
| cp .gitignore /path/to/your-username/llama3-dementia-care/ | |
| ``` | |
| ## Step 4: Add Model Weights (Critical Step) | |
| This is the most complex part. You have several options: | |
| ### Option A: Convert Ollama Model (Recommended) | |
| 1. Run the export script: | |
| ```bash | |
| ./export_model.sh | |
| ``` | |
| 2. Use a conversion tool like `ollama-export` or similar to convert your Ollama model to PyTorch format | |
| 3. Common conversion commands: | |
| ```bash | |
| # Example conversion (may vary based on tool) | |
| ollama export llama3-dementia-care:latest model.gguf | |
| # Then convert GGUF to PyTorch format using appropriate tools | |
| ``` | |
| ### Option B: Use Base Model + Fine-tuning Weights | |
| 1. Download the base Llama 3 8B model from Hugging Face | |
| 2. Add your fine-tuning weights/adapters | |
| 3. Upload the complete model | |
| ### Option C: Re-create the Model | |
| 1. Start with the official Llama 3 8B model | |
| 2. Fine-tune it using your dementia care dataset | |
| 3. Upload the fine-tuned result | |
| ## Step 5: Set up Git LFS for Large Files | |
| ```bash | |
| cd your-username/llama3-dementia-care | |
| git lfs track "*.bin" | |
| git lfs track "*.safetensors" | |
| git lfs track "*.gguf" | |
| git add .gitattributes | |
| ``` | |
| ## Step 6: Commit and Push | |
| ```bash | |
| git add . | |
| git commit -m "Add Llama 3 Dementia Care Assistant model" | |
| git push | |
| ``` | |
| ## Step 7: Update Model Card | |
| 1. Go to your model page on Hugging Face | |
| 2. Edit the README.md if needed | |
| 3. Add any additional information about training data, evaluation metrics, etc. | |
| 4. Test the inference widget with sample prompts | |
| ## Sample Model Files You Need | |
| For a complete Hugging Face model, you typically need: | |
| - ✅ `README.md` (with YAML frontmatter) | |
| - ✅ `config.json` | |
| - ✅ `tokenizer_config.json` | |
| - ✅ `special_tokens_map.json` | |
| - ⚠️ `pytorch_model.bin` or `model.safetensors` (converted model weights) | |
| - ⚠️ `tokenizer.model` or `tokenizer.json` (if needed) | |
| - ✅ Optional: `generation_config.json`, `training_args.bin` | |
| ## Testing Your Model | |
| After upload, test your model: | |
| ```python | |
| from transformers import AutoTokenizer, AutoModelForCausalLM | |
| model_name = "your-username/llama3-dementia-care" | |
| tokenizer = AutoTokenizer.from_pretrained(model_name) | |
| model = AutoModelForCausalLM.from_pretrained(model_name) | |
| # Test with a dementia care question | |
| prompt = "What are some strategies for managing sundown syndrome?" | |
| # ... rest of inference code | |
| ``` | |
| ## Troubleshooting | |
| ### Common Issues: | |
| 1. **Large file errors**: Make sure Git LFS is properly configured | |
| 2. **Token errors**: Use `huggingface-cli login` to authenticate | |
| 3. **Model loading errors**: Ensure all config files are correct | |
| 4. **Inference issues**: Test the model locally before uploading | |
| ### Getting Help: | |
| - Hugging Face Documentation: https://huggingface.co/docs | |
| - Community Forum: https://discuss.huggingface.co | |
| - Discord: https://discord.gg/huggingface | |
| ## Important Notes | |
| 1. **License Compliance**: Ensure your model respects the Llama 3 Community License | |
| 2. **Attribution**: Always include "Built with Meta Llama 3" as required | |
| 3. **Medical Disclaimers**: Include appropriate disclaimers for medical/health content | |
| 4. **Model Safety**: Test thoroughly before public release | |
| ## Final Checklist | |
| - [ ] Repository created on Hugging Face | |
| - [ ] All configuration files uploaded | |
| - [ ] Model weights converted and uploaded | |
| - [ ] README.md is complete and accurate | |
| - [ ] License information is included | |
| - [ ] Model card is comprehensive | |
| - [ ] Inference widget works | |
| - [ ] Example usage is provided | |
| - [ ] Appropriate disclaimers are included | |
| Good luck with your model upload! 🚀 | |