Hugging Face's logo

DreadPoor
/
YM-12B-Model_Stock-GGUF

Transformers
GGUF
mergekit
Merge
conversational
Model card Files
xet
 Community
How to use from
llama.cpp
Install from brew 
brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf DreadPoor/YM-12B-Model_Stock-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf DreadPoor/YM-12B-Model_Stock-GGUF:Q4_K_M
Install from WinGet (Windows) 
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf DreadPoor/YM-12B-Model_Stock-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf DreadPoor/YM-12B-Model_Stock-GGUF:Q4_K_M
Use pre-built binary 
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf DreadPoor/YM-12B-Model_Stock-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf DreadPoor/YM-12B-Model_Stock-GGUF:Q4_K_M
Build from source code 
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf DreadPoor/YM-12B-Model_Stock-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf DreadPoor/YM-12B-Model_Stock-GGUF:Q4_K_M
Use Docker 
docker model run hf.co/DreadPoor/YM-12B-Model_Stock-GGUF:Q4_K_M
Quick Links

merge

image/gif

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Model Stock merge method using yamatazen/EtherealAurora-12B-v2 as a base.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: LatitudeGames/Wayfarer-12B
  - model: MarinaraSpaghetti/NemoMix-Unleashed-12B
  - model: nothingiisreal/MN-12B-Celeste-V1.9
  - model: TheDrummer/Rocinante-12B-v1.1
  - model: anthracite-org/magnum-v2-12b
  - model: nbeerbower/Lyra4-Gutenberg-12B
  - model: cognitivecomputations/dolphin-2.9.3-mistral-nemo-12b
merge_method: model_stock
base_model: yamatazen/EtherealAurora-12B-v2
normalize: false
int8_mask: true
dtype: bfloat16
Downloads last month
7
GGUF
Model size
12B params
Architecture
llama
Hardware compatibility
Log In to add your hardware

4-bit

6-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for DreadPoor/YM-12B-Model_Stock-GGUF

Base model

DreadPoor/YM-12B-Model_Stock
Quantized
(3)
this model

Paper for DreadPoor/YM-12B-Model_Stock-GGUF