| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen2.5-7B-Instruct |
| library_name: peft |
| pipeline_tag: text-generation |
| tags: [materials-science, crystal-structure, cif, crystallography, lora, chemistry, qwen2.5] |
| --- |
| |
| # LLM_Crystal_CIF — LoRA adapters for CIF generation |
|
|
| LoRA adapters fine-tuning **Qwen2.5-7B-Instruct** to emit a full **CIF** crystal structure |
| from a prompt of *reduced composition + target space-group number*. Part of a controlled |
| **composition-sweep** study (MP-20 : MPTS-52 training ratio at fixed volume/steps). |
|
|
| Trained with Unsloth, LoRA **r=32 / α=64**, lr 1e-4, **24,000 unique training crystals**, |
| **pinned 4,500 steps** (no early stopping), 16-bit. Adapter base: `unsloth/Qwen2.5-7B-Instruct` |
| (loads fine on stock `Qwen/Qwen2.5-7B-Instruct`). |
|
|
| ## Adapters (by training composition, MP-20% : MPTS-52%) |
|
|
| | Subfolder | MP-20 % | Best-of-10 match (full 8,096 MPTS-52 test) | |
| |---|---:|---:| |
| | `comp_mp20_00` | 0 (pure MPTS-52 baseline) | 30.1% | |
| | `comp_mp20_25` | 25 | 30.4% | |
| | `comp_mp20_50` | 50 | 29.5% | |
| | `comp_mp20_75` | 75 | 28.0% | |
| | `comp_mp20_100` | 100 (pure MP-20) | 26.6% | |
|
|
| **Finding:** at matched volume + steps, match declines monotonically as the MP-20 fraction rises — |
| the gain reported for "combined" data was **volume, not symmetry composition**. |
|
|
| Each subfolder contains: `model/` (LoRA adapter + `adapter_config.json`), `tokenizer/`, |
| `config.json`, and `training_stats.json`. |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| from peft import PeftModel |
| |
| base = AutoModelForCausalLM.from_pretrained( |
| "Qwen/Qwen2.5-7B-Instruct", torch_dtype="bfloat16", device_map="auto") |
| model = PeftModel.from_pretrained(base, "shehrozashoaib/LLM_Crystal_CIF", subfolder="comp_mp20_50/model") |
| tok = AutoTokenizer.from_pretrained("shehrozashoaib/LLM_Crystal_CIF", subfolder="comp_mp20_50/tokenizer") |
| |
| messages = [ |
| {"role": "system", "content": "You are an expert in materials science and crystallography."}, |
| {"role": "user", "content": "Generate CIF for the given material description\n\n" |
| "Material composition is FeCuS2. It has a space group number 122."}, |
| ] |
| prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
| out = model.generate(**tok(prompt, return_tensors="pt").to(model.device), |
| max_new_tokens=3072, do_sample=True, temperature=0.6, top_p=0.9) |
| print(tok.decode(out[0], skip_special_tokens=True)) |
| ``` |
|
|
| Code, datasets, and full results: see the companion GitHub repo `LLM_Crystal_CIF`. |
|
|
