README.md

metadata

license: cc-by-nc-4.0
pipeline_tag: text-generation
library_name: gguf

GGUF importance matrix (imatrix) quants for https://huggingface.co/mlabonne/NeuralMonarch-7B
The importance matrix was trained for ~50K tokens (105 batches of 512 tokens) using a general purpose imatrix calibration dataset.

Layers	Context	Template
32	32768	<s>user {prompt}</s> <s>assistant {response}