---
license: apache-2.0
base_model: Qwen/Qwen2.5-7B-Instruct
library_name: peft
pipeline_tag: text-generation
tags: [materials-science, crystal-structure, cif, crystallography, lora, chemistry, qwen2.5]
---

# LLM_Crystal_CIF — LoRA adapters for CIF generation

LoRA adapters fine-tuning **Qwen2.5-7B-Instruct** to emit a full **CIF** crystal structure
from a prompt of *reduced composition + target space-group number*. Part of a controlled
**composition-sweep** study (MP-20 : MPTS-52 training ratio at fixed volume/steps).

Trained with Unsloth, LoRA **r=32 / α=64**, lr 1e-4, **24,000 unique training crystals**,
**pinned 4,500 steps** (no early stopping), 16-bit. Adapter base: `unsloth/Qwen2.5-7B-Instruct`
(loads fine on stock `Qwen/Qwen2.5-7B-Instruct`).

## Adapters (by training composition, MP-20% : MPTS-52%)

| Subfolder | MP-20 % | Best-of-10 match (full 8,096 MPTS-52 test) |
|---|---:|---:|
| `comp_mp20_00`  | 0 (pure MPTS-52 baseline) | 30.1% |
| `comp_mp20_25`  | 25 | 30.4% |
| `comp_mp20_50`  | 50 | 29.5% |
| `comp_mp20_75`  | 75 | 28.0% |
| `comp_mp20_100` | 100 (pure MP-20) | 26.6% |

**Finding:** at matched volume + steps, match declines monotonically as the MP-20 fraction rises —
the gain reported for "combined" data was **volume, not symmetry composition**.

Each subfolder contains: `model/` (LoRA adapter + `adapter_config.json`), `tokenizer/`,
`config.json`, and `training_stats.json`.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "shehrozashoaib/LLM_Crystal_CIF", subfolder="comp_mp20_50/model")
tok = AutoTokenizer.from_pretrained("shehrozashoaib/LLM_Crystal_CIF", subfolder="comp_mp20_50/tokenizer")

messages = [
    {"role": "system", "content": "You are an expert in materials science and crystallography."},
    {"role": "user", "content": "Generate CIF for the given material description\n\n"
                                 "Material composition is FeCuS2. It has a space group number 122."},
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
out = model.generate(**tok(prompt, return_tensors="pt").to(model.device),
                     max_new_tokens=3072, do_sample=True, temperature=0.6, top_p=0.9)
print(tok.decode(out[0], skip_special_tokens=True))
```

Code, datasets, and full results: see the companion GitHub repo `LLM_Crystal_CIF`.