Hugging Face's logo

putazon
/
SearchQueryNER-6k-qwen2.5-14B-preview

Transformers
GGUF
English
qwen2
text-generation-inference
unsloth
Model card Files
xet
 Community
How to use from the
Use from the
llama-cpp-python library
# Gated model: Login with a HF token with gated access permission
hf auth login
# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="putazon/SearchQueryNER-6k-qwen2.5-14B-preview",
	filename="unsloth.Q5_K_M.gguf",
)

output = llm(
	"Once upon a time,",
	max_tokens=512,
	echo=True
)
print(output)

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Uploaded model

  • Developed by: putazon
  • License: apache-2.0
  • Finetuned from model : unsloth/qwen2.5-14b-bnb-4bit

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.

Downloads last month
-
GGUF
Model size
15B params
Architecture
qwen2
Hardware compatibility
Log In to add your hardware

5-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support