File size: 541 Bytes
04f9e9a 55d9b15 04f9e9a 55d9b15 04f9e9a 55d9b15 04f9e9a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | import gradio as gr
from llama_cpp import Llama
llm = Llama.from_pretrained(
repo_id="LiquidAI/LFM2.5-230M-GGUF",
filename="LFM2.5-230M-Q4_K_M.gguf",
)
def chat(message, history):
prompt = f"<|user|>\n{message}\n<|assistant|>\n"
output = llm(
prompt,
max_tokens=256,
temperature=0.7,
stop=["<|user>", "<|end|>"],
)
return output["choices"][0]["text"].strip()
gr.ChatInterface(
chat,
title="Try SLM - LFM2.5 230M",
description="LiquidAI LFM2.5-230M GGUF model"
).launch() |