Instructions to use carecodeconnect/jhana-mistral-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries
PEFT
How to use carecodeconnect/jhana-mistral-GGUF with PEFT:
```
Task type is invalid.
```

How to use carecodeconnect/jhana-mistral-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="carecodeconnect/jhana-mistral-GGUF",
	filename="jhana-mistral.Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use carecodeconnect/jhana-mistral-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf carecodeconnect/jhana-mistral-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf carecodeconnect/jhana-mistral-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf carecodeconnect/jhana-mistral-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf carecodeconnect/jhana-mistral-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf carecodeconnect/jhana-mistral-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf carecodeconnect/jhana-mistral-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf carecodeconnect/jhana-mistral-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf carecodeconnect/jhana-mistral-GGUF:Q4_K_M

Use Docker

docker model run hf.co/carecodeconnect/jhana-mistral-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use carecodeconnect/jhana-mistral-GGUF with Ollama:
```
ollama run hf.co/carecodeconnect/jhana-mistral-GGUF:Q4_K_M
```

Unsloth Studio

How to use carecodeconnect/jhana-mistral-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for carecodeconnect/jhana-mistral-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for carecodeconnect/jhana-mistral-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for carecodeconnect/jhana-mistral-GGUF to start chatting

Docker Model Runner
How to use carecodeconnect/jhana-mistral-GGUF with Docker Model Runner:
```
docker model run hf.co/carecodeconnect/jhana-mistral-GGUF:Q4_K_M
```

Lemonade

How to use carecodeconnect/jhana-mistral-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull carecodeconnect/jhana-mistral-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.jhana-mistral-GGUF-Q4_K_M

List all available models

lemonade list

jhana-mistral-GGUF

This model is a fine-tuned version of TheBloke/Mistral-7B-Instruct-v0.1-GPTQ specifically trained for generating guided meditations. The fine-tuning was conducted on the "jhana-guided-meditations-collection" dataset available on Hugging Face, utilizing the QLoRA fine-tuning approach.

Model description

The model utilizes the LlamaTokenizer and is quantized for efficient load and execution. It is intended for generating mindful meditation scripts by understanding and generating contextually relevant content. This version has been optimized for better performance and lower resource utilization during inference.

Intended uses & limitations

This model is intended for generating text related to guided meditations. It may not perform well on unrelated tasks or general-purpose language understanding due to its specialized training.

Training and evaluation data

The model was trained on the "jhana-guided-meditations-collection" dataset, which consists of various guided meditation scripts. The data was preprocessed and tokenized using the LlamaTokenizer.

Training procedure

Training hyperparameters

Learning Rate: 0.0002
Batch Size: 8 for training, 8 for evaluation
Optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
Scheduler: Cosine learning rate scheduler
Training Steps: 250
Mixed Precision Training: Utilized Native AMP

Training results

Training resulted in a model capable of generating coherent and contextually relevant meditation scripts, improving upon the base model's capabilities in this specific domain.

Framework versions

PEFT: 0.10.0
Transformers: 4.40.0.dev0
Pytorch: 2.2.2+cu121
Datasets: 2.18.0
Tokenizers: 0.15.2

Quantization with llama.cpp

The model was quantized to enhance its efficiency and reduce its size, making it more suitable for deployment in various environments, including those with limited resources. The quantization process was performed using llama.cpp, following the steps outlined by Maxime Labonne in Quantize Llama models with GGUF and llama.cpp.

The process involved:

Cloning the llama.cpp repository and setting it up with the required dependencies.
Downloading the model to be quantized.
Using the llama.cpp/convert.py script to convert the model to fp16 format, followed by quantization, significantly reducing the model's size while retaining its performance capabilities.

The quantization resulted in a compressed model with a significant reduction in size from 13813.02 MB to 4892.99 MB, enhancing its loading and inference speeds without compromising on the generation quality.

Downloads last month: 6

GGUF

Model size

7B params

Architecture

llama

Hardware compatibility

4-bit

5-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for carecodeconnect/jhana-mistral-GGUF

Base model

mistralai/Mistral-7B-v0.1

Finetuned

mistralai/Mistral-7B-Instruct-v0.1

Quantized

TheBloke/Mistral-7B-Instruct-v0.1-GPTQ

Adapter

(211)

this model