Instructions to use ullahhafeez321/Qwen-NER-Decoder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ullahhafeez321/Qwen-NER-Decoder with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ullahhafeez321/Qwen-NER-Decoder")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("ullahhafeez321/Qwen-NER-Decoder", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ullahhafeez321/Qwen-NER-Decoder with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ullahhafeez321/Qwen-NER-Decoder"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ullahhafeez321/Qwen-NER-Decoder",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ullahhafeez321/Qwen-NER-Decoder

SGLang

How to use ullahhafeez321/Qwen-NER-Decoder with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ullahhafeez321/Qwen-NER-Decoder" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ullahhafeez321/Qwen-NER-Decoder",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ullahhafeez321/Qwen-NER-Decoder" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ullahhafeez321/Qwen-NER-Decoder",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ullahhafeez321/Qwen-NER-Decoder with Docker Model Runner:
```
docker model run hf.co/ullahhafeez321/Qwen-NER-Decoder
```

Qwen-NER-Decoder / README.md

ullahhafeez321

Update README.md

de57b22 verified 6 days ago

preview code

Raw History Blame Contribute Delete

8.62 kB

---
language:
  - en
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
  - named-entity-recognition
  - ner
  - qwen
  - decoder-only
  - lora
  - conll2003
  - information-extraction
base_model: Qwen/Qwen2.5-0.5B-Instruct
datasets:
  - lhoestq/conll2003
---
# English Named Entity Recognition (NER) – Qwen2.5-0.5B-Instruct + LoRA

## Model Description

**Qwen-NER-Decoder** is a fine-tuned decoder-only Named Entity Recognition (NER) model based on [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct).

The model identifies and extracts named entities from English text and generates structured JSON output.

It is fine-tuned on the CoNLL-2003 English Named Entity Recognition dataset and recognizes four entity categories:

- **PER** – Person
- **ORG** – Organization
- **LOC** – Location
- **MISC** – Miscellaneous

The model uses **Low-Rank Adaptation (LoRA)** for parameter-efficient fine-tuning. The Hugging Face repository contains the LoRA adapter, so it must be loaded together with its original base model.

## Model Details

| Property | Details |
|---|---|
| Model Name | Qwen-NER-Decoder |
| Base Model | Qwen/Qwen2.5-0.5B-Instruct |
| Architecture | Decoder-only Transformer |
| Fine-tuning Method | LoRA |
| LoRA Rank | 8 |
| LoRA Alpha | 16 |
| LoRA Dropout | 0.05 |
| Training Epochs | 1 |
| Language | English |
| Task | Named Entity Recognition |
| Output Format | JSON |
| Training Framework | PyTorch, Transformers, PEFT |

## Dataset

The model was fine-tuned using the [CoNLL-2003 English NER dataset](https://huggingface.co/datasets/lhoestq/conll2003).

Dataset repository: `lhoestq/conll2003`

The dataset contains English sentences annotated with BIO-style named entity labels.

The four entity categories are:

| Label | Description |
|---|---|
| PER | Person |
| ORG | Organization |
| LOC | Location |
| MISC | Miscellaneous |

## Training Configuration

The model was fine-tuned using LoRA with the following configuration:

| Configuration | Value |
|---|---|
| Base Model | Qwen2.5-0.5B-Instruct |
| LoRA Rank | 8 |
| LoRA Alpha | 16 |
| LoRA Dropout | 0.05 |
| Target Modules | q_proj, k_proj, v_proj, o_proj |
| Training Epochs | 1 |
| Training Samples | Up to 8,000 |
| Validation Samples | 1,000 |
| Training Precision | FP16 |
| Optimizer | AdamW |
| Training Environment | Kaggle GPU |

LoRA adapts selected attention projection layers without updating all the parameters of the original base model.

## Evaluation Results

The model was evaluated on a quick test subset of 100 English sentences using exact entity-level matching.

| Metric | Score |
|---|---:|
| Precision | 90.40% |
| Recall | 81.63% |
| F1-score | 85.79% |
| True Positives | 160 |
| False Positives | 17 |
| False Negatives | 36 |

**Evaluation note:** These results represent the 100-sample evaluation subset, not the complete test dataset. They should not be interpreted as a full-dataset benchmark.

Additional manually selected English sentences were used to inspect entity extraction behavior.

## Example Predictions

### Example 1

**Input:**

```text
Elon Musk founded SpaceX.
```

**Output:**

```json
{
  "entities": [
    {
      "text": "Elon Musk",
      "type": "PER"
    },
    {
      "text": "SpaceX",
      "type": "ORG"
    }
  ]
}
```

### Example 2

**Input:**

```text
Barack Obama visited Microsoft headquarters in Seattle.
```

**Output:**

```json
{
  "entities": [
    {
      "text": "Barack Obama",
      "type": "PER"
    },
    {
      "text": "Microsoft",
      "type": "ORG"
    },
    {
      "text": "Seattle",
      "type": "LOC"
    }
  ]
}
```

*Examples illustrate the intended output structure.*

## How to Use

### 1. Install Dependencies

```bash
pip install torch transformers peft accelerate safetensors
```

### 2. Load the Base Model and LoRA Adapter

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftConfig, PeftModel
ADAPTER_MODEL = "ullahhafeez321/Qwen-NER-Decoder"
# Read the original base model from the adapter configuration
adapter_config = PeftConfig.from_pretrained(ADAPTER_MODEL)
BASE_MODEL = adapter_config.base_model_name_or_path
# Load tokenizer from the base model
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
if tokenizer.pad_token_id is None:
    tokenizer.pad_token = tokenizer.eos_token
# Load the original Qwen base model
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=(
        torch.float16 if torch.cuda.is_available()
        else torch.float32
    ),
    device_map="auto"
)
# Attach the fine-tuned LoRA adapter
model = PeftModel.from_pretrained(
    base_model,
    ADAPTER_MODEL
)
model.eval()
print("Qwen NER model loaded successfully!")
```

### 3. Run Inference

The prompt used during inference should follow the same instruction and output format used during fine-tuning.

The following is an illustrative JSON extraction example:

```python
sentence = "Albert Einstein worked at Princeton University."
prompt = f"""
Extract named entities from the following sentence.
Entity types:
PER = Person
ORG = Organization
LOC = Location
MISC = Miscellaneous
Return only a JSON array:
[
  {{"text": "entity name", "type": "PER"}}
]
If no entities are found, return [].
Sentence: {sentence}
Entities:
"""
inputs = tokenizer(
    prompt,
    return_tensors="pt",
    truncation=True,
    max_length=1024
)
input_device = model.get_input_embeddings().weight.device
inputs = {
    key: value.to(input_device)
    for key, value in inputs.items()
}
with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=150,
        do_sample=False,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id
    )
generated_tokens = outputs[0][inputs["input_ids"].shape[1]:]
result = tokenizer.decode(
    generated_tokens,
    skip_special_tokens=True
)
print(result)
```

**Note:** This example demonstrates the inference process. For the best results, use the exact prompt template and target format used to train the uploaded adapter.

## Intended Use

This model is intended for:

- English named entity recognition.
- Natural Language Processing applications.
- Information extraction systems.
- Text analysis and document processing.
- Educational and research projects.
- Integration with sentiment classification systems.
- Applications that require structured entity extraction.

## Limitations

- The model may occasionally miss named entities.
- Some entities may receive incorrect classifications.
- Entity boundaries and spelling may occasionally be incorrect.
- Performance can vary across different domains and writing styles.
- The reported evaluation score is based on a limited 100-sample test subset.
- The model is intended for English text and may not perform reliably on other languages.
- The model is not intended to replace human verification in critical applications.

## Model Architecture

This project uses a decoder-only Transformer architecture based on Qwen2.5-0.5B-Instruct.

The model generates structured entity extraction results from an English input sentence.

LoRA fine-tuning adapts selected attention projection layers while preserving the original base model weights.

The uploaded Hugging Face repository contains the fine-tuned adapter rather than a complete standalone model checkpoint.

## Future Improvements

- Increase training data and training epochs.
- Improve recall and entity coverage.
- Evaluate performance on the complete test dataset.
- Improve entity boundary detection.
- Expand evaluation across different English text domains.
- Improve inference consistency and structured JSON generation.
- Integrate with an encoder-only BERT sentiment classification model.

## Acknowledgements

- [Qwen Team](https://huggingface.co/Qwen) for the Qwen2.5 base model.
- [CoNLL-2003](https://huggingface.co/datasets/lhoestq/conll2003) dataset contributors.
- [Hugging Face](https://huggingface.co/) for the Transformers, Datasets, and PEFT ecosystem.

## License

The adapter repository declares the Apache-2.0 license. Please also review the original base model and dataset licenses and their respective usage conditions before redistribution or commercial use.

---

**Project:** English Named Entity Recognition using a Decoder-only Transformer with LoRA Fine-tuning.

**Model Repository:** `ullahhafeez321/Qwen-NER-Decoder`

**Base Model:** `Qwen/Qwen2.5-0.5B-Instruct`

**Frameworks:** PyTorch, Transformers, PEFT, Hugging Face Datasets.

	---
	language:
	- en
	license: apache-2.0
	library_name: transformers
	pipeline_tag: text-generation
	tags:
	- named-entity-recognition
	- ner
	- qwen
	- decoder-only
	- lora
	- conll2003
	- information-extraction
	base_model: Qwen/Qwen2.5-0.5B-Instruct
	datasets:
	- lhoestq/conll2003
	---

	# English Named Entity Recognition (NER) – Qwen2.5-0.5B-Instruct + LoRA

	## Model Description

	Qwen-NER-Decoder is a fine-tuned decoder-only Named Entity Recognition (NER) model based on [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct).

	The model identifies and extracts named entities from English text and generates structured JSON output.

	It is fine-tuned on the CoNLL-2003 English Named Entity Recognition dataset and recognizes four entity categories:

	- PER – Person
	- ORG – Organization
	- LOC – Location
	- MISC – Miscellaneous

	The model uses Low-Rank Adaptation (LoRA) for parameter-efficient fine-tuning. The Hugging Face repository contains the LoRA adapter, so it must be loaded together with its original base model.

	## Model Details

	\| Property \| Details \|
	\|---\|---\|
	\| Model Name \| Qwen-NER-Decoder \|
	\| Base Model \| Qwen/Qwen2.5-0.5B-Instruct \|
	\| Architecture \| Decoder-only Transformer \|
	\| Fine-tuning Method \| LoRA \|
	\| LoRA Rank \| 8 \|
	\| LoRA Alpha \| 16 \|
	\| LoRA Dropout \| 0.05 \|
	\| Training Epochs \| 1 \|
	\| Language \| English \|
	\| Task \| Named Entity Recognition \|
	\| Output Format \| JSON \|
	\| Training Framework \| PyTorch, Transformers, PEFT \|

	## Dataset

	The model was fine-tuned using the [CoNLL-2003 English NER dataset](https://huggingface.co/datasets/lhoestq/conll2003).

	Dataset repository: `lhoestq/conll2003`

	The dataset contains English sentences annotated with BIO-style named entity labels.

	The four entity categories are:

	\| Label \| Description \|
	\|---\|---\|
	\| PER \| Person \|
	\| ORG \| Organization \|
	\| LOC \| Location \|
	\| MISC \| Miscellaneous \|

	## Training Configuration

	The model was fine-tuned using LoRA with the following configuration:

	\| Configuration \| Value \|
	\|---\|---\|
	\| Base Model \| Qwen2.5-0.5B-Instruct \|
	\| LoRA Rank \| 8 \|
	\| LoRA Alpha \| 16 \|
	\| LoRA Dropout \| 0.05 \|
	\| Target Modules \| q_proj, k_proj, v_proj, o_proj \|
	\| Training Epochs \| 1 \|
	\| Training Samples \| Up to 8,000 \|
	\| Validation Samples \| 1,000 \|
	\| Training Precision \| FP16 \|
	\| Optimizer \| AdamW \|
	\| Training Environment \| Kaggle GPU \|

	LoRA adapts selected attention projection layers without updating all the parameters of the original base model.

	## Evaluation Results

	The model was evaluated on a quick test subset of 100 English sentences using exact entity-level matching.

	\| Metric \| Score \|
	\|---\|---:\|
	\| Precision \| 90.40% \|
	\| Recall \| 81.63% \|
	\| F1-score \| 85.79% \|
	\| True Positives \| 160 \|
	\| False Positives \| 17 \|
	\| False Negatives \| 36 \|

	Evaluation note: These results represent the 100-sample evaluation subset, not the complete test dataset. They should not be interpreted as a full-dataset benchmark.

	Additional manually selected English sentences were used to inspect entity extraction behavior.

	## Example Predictions

	### Example 1

	Input:

	```text
	Elon Musk founded SpaceX.
	```

	Output:

	```json
	{
	"entities": [
	{
	"text": "Elon Musk",
	"type": "PER"
	},
	{
	"text": "SpaceX",
	"type": "ORG"
	}
	]
	}
	```

	### Example 2

	Input:

	```text
	Barack Obama visited Microsoft headquarters in Seattle.
	```

	Output:

	```json
	{
	"entities": [
	{
	"text": "Barack Obama",
	"type": "PER"
	},
	{
	"text": "Microsoft",
	"type": "ORG"
	},
	{
	"text": "Seattle",
	"type": "LOC"
	}
	]
	}
	```

	Examples illustrate the intended output structure.

	## How to Use

	### 1. Install Dependencies

	```bash
	pip install torch transformers peft accelerate safetensors
	```

	### 2. Load the Base Model and LoRA Adapter

	```python
	import torch
	from transformers import AutoTokenizer, AutoModelForCausalLM
	from peft import PeftConfig, PeftModel

	ADAPTER_MODEL = "ullahhafeez321/Qwen-NER-Decoder"

	# Read the original base model from the adapter configuration
	adapter_config = PeftConfig.from_pretrained(ADAPTER_MODEL)
	BASE_MODEL = adapter_config.base_model_name_or_path

	# Load tokenizer from the base model
	tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

	if tokenizer.pad_token_id is None:
	tokenizer.pad_token = tokenizer.eos_token

	# Load the original Qwen base model
	base_model = AutoModelForCausalLM.from_pretrained(
	BASE_MODEL,
	torch_dtype=(
	torch.float16 if torch.cuda.is_available()
	else torch.float32
	),
	device_map="auto"
	)

	# Attach the fine-tuned LoRA adapter
	model = PeftModel.from_pretrained(
	base_model,
	ADAPTER_MODEL
	)

	model.eval()

	print("Qwen NER model loaded successfully!")
	```

	### 3. Run Inference

	The prompt used during inference should follow the same instruction and output format used during fine-tuning.

	The following is an illustrative JSON extraction example:

	```python
	sentence = "Albert Einstein worked at Princeton University."

	prompt = f"""
	Extract named entities from the following sentence.

	Entity types:
	PER = Person
	ORG = Organization
	LOC = Location
	MISC = Miscellaneous

	Return only a JSON array:

	[
	{{"text": "entity name", "type": "PER"}}
	]

	If no entities are found, return [].

	Sentence: {sentence}

	Entities:
	"""

	inputs = tokenizer(
	prompt,
	return_tensors="pt",
	truncation=True,
	max_length=1024
	)

	input_device = model.get_input_embeddings().weight.device
	inputs = {
	key: value.to(input_device)
	for key, value in inputs.items()
	}

	with torch.inference_mode():
	outputs = model.generate(
	**inputs,
	max_new_tokens=150,
	do_sample=False,
	pad_token_id=tokenizer.pad_token_id,
	eos_token_id=tokenizer.eos_token_id
	)

	generated_tokens = outputs[0][inputs["input_ids"].shape[1]:]

	result = tokenizer.decode(
	generated_tokens,
	skip_special_tokens=True
	)

	print(result)
	```

	Note: This example demonstrates the inference process. For the best results, use the exact prompt template and target format used to train the uploaded adapter.

	## Intended Use

	This model is intended for:

	- English named entity recognition.
	- Natural Language Processing applications.
	- Information extraction systems.
	- Text analysis and document processing.
	- Educational and research projects.
	- Integration with sentiment classification systems.
	- Applications that require structured entity extraction.

	## Limitations

	- The model may occasionally miss named entities.
	- Some entities may receive incorrect classifications.
	- Entity boundaries and spelling may occasionally be incorrect.
	- Performance can vary across different domains and writing styles.
	- The reported evaluation score is based on a limited 100-sample test subset.
	- The model is intended for English text and may not perform reliably on other languages.
	- The model is not intended to replace human verification in critical applications.

	## Model Architecture

	This project uses a decoder-only Transformer architecture based on Qwen2.5-0.5B-Instruct.

	The model generates structured entity extraction results from an English input sentence.

	LoRA fine-tuning adapts selected attention projection layers while preserving the original base model weights.

	The uploaded Hugging Face repository contains the fine-tuned adapter rather than a complete standalone model checkpoint.

	## Future Improvements

	- Increase training data and training epochs.
	- Improve recall and entity coverage.
	- Evaluate performance on the complete test dataset.
	- Improve entity boundary detection.
	- Expand evaluation across different English text domains.
	- Improve inference consistency and structured JSON generation.
	- Integrate with an encoder-only BERT sentiment classification model.

	## Acknowledgements

	- [Qwen Team](https://huggingface.co/Qwen) for the Qwen2.5 base model.
	- [CoNLL-2003](https://huggingface.co/datasets/lhoestq/conll2003) dataset contributors.
	- [Hugging Face](https://huggingface.co/) for the Transformers, Datasets, and PEFT ecosystem.

	## License

	The adapter repository declares the Apache-2.0 license. Please also review the original base model and dataset licenses and their respective usage conditions before redistribution or commercial use.

	---

	Project: English Named Entity Recognition using a Decoder-only Transformer with LoRA Fine-tuning.

	Model Repository: `ullahhafeez321/Qwen-NER-Decoder`

	Base Model: `Qwen/Qwen2.5-0.5B-Instruct`

	Frameworks: PyTorch, Transformers, PEFT, Hugging Face Datasets.