Initial local S2S shell starter de2ad9c
raichemathew1 commited on
How to use maytman12/s2s-complete-setup with llama-cpp-python:
# !pip install llama-cpp-python
from llama_cpp import Llama
llm = Llama.from_pretrained(
repo_id="maytman12/s2s-complete-setup",
filename="{{GGUF_FILE}}",
)
output = llm( "Once upon a time,", max_tokens=512, echo=True ) print(output)