File size: 1,178 Bytes
e2b8fff aa3901d 4ffd249 aa3901d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
---
license: bigscience-openrail-m
base_model:
- meta-llama/Llama-3.2-3B
- Qwen/Qwen2.5-3B-Instruct
---
The following example illustrates how to load the FineEdit-XL adapter for inference. For FineEdit-Pro, we use Qwen2.5-3B-Instruct as the base model.
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
## Base Model
base_model = "meta-llama/Llama-3.2-3B"
## Lora
adapter_model = "YimingZeng/FineEdit_Model"
subfolder = "FineEdit-XL"
## Load tokenizer and base model
tokenizer = AutoTokenizer.from_pretrained(base_model)
base = AutoModelForCausalLM.from_pretrained(
base_model,
torch_dtype=torch.bfloat16,
device_map="auto"
)
## Load LoRA adapter
model = PeftModel.from_pretrained(base, adapter_model, subfolder=subfolder)
## Test
prompt = """Edit Request: Please change 'Captain American' to 'Iron Man'.
Original Content: 'Captain American' is a superhero.
Edited Content:
"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
|