README.md

---
license: other
tags:
  - materials-science
  - crystal-generation
  - diffusion
  - lora
---

# 3D_LLM_Diffusion — Model Weights

Weights only. Code and usage documentation: https://github.com/Richardyangfan78/3D_LLM_Diffusion

```bash
hf download Yangfan78/3D_LLM_Diffusion --local-dir weights
```

Three independent weight sets.

---

## `generator_v41/` — Conditional crystal generator

| File | Size | Description |
|---|---:|---|
| `best.pt` | 378.9 MB | All generator weights, backbone included |
| `priors/train.csv` | 75.2 MB | Read at sampling time: atom-count prior + allowed element set |
| `priors/val.csv` | 25.0 MB | Validation split |
| `eval/` | — | Evaluation records |

| Property | Value |
|---|---|
| Architecture ID | `mattergen-spacegroup-hhi-qwen-crossattention-crystalite-v8` |
| Parameters | 94.63 M (777 tensors) |
| Dimensions | d_model 512 / 14 layers / 16 heads |
| Training step | 6000 |
| Atom type encoding | `subatomic_tokenizer_pca_16` (16-d continuous) |
| Lattice representation | `ltri` |
| SHA-256 | `cd67729bd862537be9113146ebdf48db81f86a53a4a32cd99c751bba0e06fa4a` |

Eight condition channels: `chemical_system`, `space_group`, `dft_band_gap`, `dft_bulk_modulus`, `dft_mag_density`, `energy_above_hull`, `hhi_score`, `ml_bulk_modulus`.

The inference policy and sampler settings are stored inside the checkpoint and apply automatically:

```
guidance   global 1.0 | band_gap 1.5 | e_above_hull 2.0 | element+hull joint 2.25
sampler    150 steps, sigma 0.002 to 80, rho 7, S_churn 60
```

---

## `text_conditioning/` — Text condition vectors and encoders

| File | Size | Description |
|---|---:|---|
| `val_text_xrd_z.pt` | 22.1 MB | 18,094 pre-encoded vectors, 256-d each |
| `trimodal_best.pt` | 51.4 MB | Crystal-text-XRD tri-modal encoder |
| `qwen_latent_adapter_best.pt` | 9.5 MB | Qwen hidden states to 256-d vector adapter |
| `val_text_xrd_z.pt.meta.json` | — | Provenance metadata |

`val_text_xrd_z.pt` is a dict keyed as:

```
mp-865981__text     text modality vector
mp-865981__xrd      XRD modality vector
```

Conditioning on these pre-encoded vectors requires no language model. Encoding new text requires Qwen3.6-35B-A3B (revision `995ad96eacd98c81ed38be0c5b274b04031597b0`) together with the other two files.

Tri-modal encoder SHA-256: `fa5dbe01af4d733022ab2d2a1db0e425414e2148154254931a1acb290029ea94`

---

## `3D_Property/` — Property prediction (LoRA)

Adapter weights on top of Qwen3.6-35B-A3B. The base model is neither included here nor modified.

| File | Size | Description |
|---|---:|---|
| `step_6500/lora_adapter/` | 613.5 MB | LoRA weights, PEFT standard format |
| `step_6500/projector.pt` | 18.9 MB | Per-atom features to Qwen embedding space |
| `step_6500/trainer_state.pt` | 1265.1 MB | Optimizer moments, LR schedule, RNG, sampler state |
| `step_6500/checkpoint_metadata.json` | — | Training metadata |
| `stage1_projector.pt` | 18.9 MB | Stage-1 projector |

| Property | Value |
|---|---|
| Base model | Qwen3.6-35B-A3B @ `995ad96eacd98c81ed38be0c5b274b04031597b0` |
| LoRA | r=128, alpha=256, dropout=0.05 |
| Attached to | 250 linear layers (`q/k/v/o_proj`, `gate/up/down_proj`, `in_proj_qkv`, `in_proj_z`, `out_proj`) |
| Trainable parameters | 153,354,240 (~0.44% of base) |
| Progress | step 6500 / 12000 |
| Reference environment | H200 single GPU, ~105.7 GB VRAM, effective batch 256 |

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3.6-35B-A3B",
    revision="995ad96eacd98c81ed38be0c5b274b04031597b0",
)
model = PeftModel.from_pretrained(base, "weights/3D_Property/step_6500/lora_adapter")
```

`projector.pt` is loaded separately to produce the per-atom soft tokens. `trainer_state.pt` is only needed to resume training and can be skipped for inference.

---

## License

The denoising trunk inside `generator_v41/best.pt` derives from the Crystalite research backbone, MIT License, Copyright (c) 2026 Joshua Rosenthal. The `3D_Property/` LoRA weights derive from Qwen3.6-35B-A3B (Apache-2.0). Full third-party attribution is in the `NOTICE` file of the code repository.