Instructions to use brucewayne0459/OpenBioLLm-Derm-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use brucewayne0459/OpenBioLLm-Derm-gguf with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="brucewayne0459/OpenBioLLm-Derm-gguf",
	filename="unsloth.F16.gguf",
)

output = llm(
	"Once upon a time,",
	max_tokens=512,
	echo=True
)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use brucewayne0459/OpenBioLLm-Derm-gguf with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf brucewayne0459/OpenBioLLm-Derm-gguf:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf brucewayne0459/OpenBioLLm-Derm-gguf:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf brucewayne0459/OpenBioLLm-Derm-gguf:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf brucewayne0459/OpenBioLLm-Derm-gguf:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf brucewayne0459/OpenBioLLm-Derm-gguf:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf brucewayne0459/OpenBioLLm-Derm-gguf:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf brucewayne0459/OpenBioLLm-Derm-gguf:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf brucewayne0459/OpenBioLLm-Derm-gguf:Q4_K_M

Use Docker

docker model run hf.co/brucewayne0459/OpenBioLLm-Derm-gguf:Q4_K_M

LM Studio
Jan

vLLM

How to use brucewayne0459/OpenBioLLm-Derm-gguf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "brucewayne0459/OpenBioLLm-Derm-gguf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "brucewayne0459/OpenBioLLm-Derm-gguf",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/brucewayne0459/OpenBioLLm-Derm-gguf:Q4_K_M

Ollama
How to use brucewayne0459/OpenBioLLm-Derm-gguf with Ollama:
```
ollama run hf.co/brucewayne0459/OpenBioLLm-Derm-gguf:Q4_K_M
```

Unsloth Studio new

How to use brucewayne0459/OpenBioLLm-Derm-gguf with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for brucewayne0459/OpenBioLLm-Derm-gguf to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for brucewayne0459/OpenBioLLm-Derm-gguf to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for brucewayne0459/OpenBioLLm-Derm-gguf to start chatting

Docker Model Runner
How to use brucewayne0459/OpenBioLLm-Derm-gguf with Docker Model Runner:
```
docker model run hf.co/brucewayne0459/OpenBioLLm-Derm-gguf:Q4_K_M
```

Lemonade

How to use brucewayne0459/OpenBioLLm-Derm-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull brucewayne0459/OpenBioLLm-Derm-gguf:Q4_K_M

Run and chat with the model

lemonade run user.OpenBioLLm-Derm-gguf-Q4_K_M

List all available models

lemonade list

Model Details

Model Description

This model is designed for skin-related medical applications, particularly for use in a dermatology chatbot. It provides clear, accurate, and helpful information about various skin diseases, skincare routines, treatments, and related dermatological advice.

Developed by: Bruce_Wayne (The Batman)
Funded by: Wayne Industries
Model type: Text Generation
Language(s) (NLP): English
Finetuned from model [optional]: OpenBioLLM (llama-3) by aaditya/Llama3-OpenBioLLM-8B

Uses

Direct Use

This model is fine-tuned on skin diseases and dermatology data and is used for a dermatology chatbot to provide clear, accurate, and helpful information about various skin diseases, skincare routines, treatments, and related dermatological advice.

Downstream Use

The model can be integrated into healthcare applications, mobile apps for skin health monitoring, or systems providing personalized skincare advice.

Out-of-Scope Use

The model should not be used for non-medical image analysis, general object detection, or without proper medical oversight. It is not designed to replace professional medical diagnosis.

Bias, Risks, and Limitations

This model is trained on dermatology data, which might contain inherent biases. It is important to note that the model's responses should not be considered a substitute for professional medical advice. There may be limitations in understanding rare skin conditions or those not well-represented in the training data. The model still needs to be fine-tuned further to get accurate answers.

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases, and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

from llama_cpp import Llama
model_name = "brucewayne0459/OpenBioLLm-Derm-gguf"
model_file = "unsloth.Q8_0.gguf"

Training Details

Training Data

The model is fine-tuned on a dataset containing information about various skin diseases and dermatology care. brucewayne0459/Skin_diseases_and_care

Training Hyperparameters

Training regime: The model was trained using the following hyperparameters: Per device train batch size: 2 Gradient accumulation steps: 4 Warmup steps: 5 Max steps: 120 Learning rate: 2e-4 Optimizer: AdamW (8-bit) Weight decay: 0.01 LR scheduler type: Linear

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: Tesla t4
Hours used: 3hr
Cloud Provider: Google Colab

Technical Specifications

Model Architecture and Objective

This model is based on the LLaMA (Large Language Model Meta AI) architecture and fine-tuned to provide dermatological advice.

Hardware

The training was performed on Tesla T4 GPU with 4-bit quantization and gradient checkpointing to optimize memory usage.

Feel free to provide any missing details or correct any assumptions, and I'll update the model card accordingly.

Downloads last month: 142

GGUF

Model size

8B params

Architecture

llama

Hardware compatibility

4-bit

8-bit

16-bit

Dataset used to train brucewayne0459/OpenBioLLm-Derm-gguf

Paper for brucewayne0459/OpenBioLLm-Derm-gguf

Quantifying the Carbon Emissions of Machine Learning

Paper • 1910.09700 • Published Oct 21, 2019 • 45