rozek/OpenLLaMA_7B_300BT_q4

This repository contains the 300BT preview of the Open LLaMA model based on the RedPajama 1T dataset, converted to GGML and quantized to 4 bit, ready to be used with llama.cpp

Usage

In order to use this model with llama.cpp

• install llama.cpp as described in the docs
• download this model
• move it into the models subfolder of llama.cpp
• run inferences with the additional parameter -m ./models/7B/ggml-openllama-7b-300bt-q4_0.bin

For example,

./main -m ./models/7B/ggml-openllama-7b-300bt-q4_0.bin -n 128 -p "who was Joseph Weizenbaum?"

License

Since the underlying model is itself Apache 2.0 licensed, this repo is as well.

---

license: apache-2.0