| --- |
| license: other |
| tags: |
| - materials-science |
| - crystal-generation |
| - diffusion |
| - lora |
| --- |
| |
| # 3D_LLM_Diffusion — Model Weights |
|
|
| Weights only. Code and usage documentation: https://github.com/Richardyangfan78/3D_LLM_Diffusion |
|
|
| ```bash |
| hf download Yangfan78/3D_LLM_Diffusion --local-dir weights |
| ``` |
|
|
| Three independent weight sets. |
|
|
| --- |
|
|
| ## `generator_v41/` — Conditional crystal generator |
| |
| | File | Size | Description | |
| |---|---:|---| |
| | `best.pt` | 378.9 MB | All generator weights, backbone included | |
| | `priors/train.csv` | 75.2 MB | Read at sampling time: atom-count prior + allowed element set | |
| | `priors/val.csv` | 25.0 MB | Validation split | |
| | `eval/` | — | Evaluation records | |
| |
| | Property | Value | |
| |---|---| |
| | Architecture ID | `mattergen-spacegroup-hhi-qwen-crossattention-crystalite-v8` | |
| | Parameters | 94.63 M (777 tensors) | |
| | Dimensions | d_model 512 / 14 layers / 16 heads | |
| | Training step | 6000 | |
| | Atom type encoding | `subatomic_tokenizer_pca_16` (16-d continuous) | |
| | Lattice representation | `ltri` | |
| | SHA-256 | `cd67729bd862537be9113146ebdf48db81f86a53a4a32cd99c751bba0e06fa4a` | |
|
|
| Eight condition channels: `chemical_system`, `space_group`, `dft_band_gap`, `dft_bulk_modulus`, `dft_mag_density`, `energy_above_hull`, `hhi_score`, `ml_bulk_modulus`. |
|
|
| The inference policy and sampler settings are stored inside the checkpoint and apply automatically: |
|
|
| ``` |
| guidance global 1.0 | band_gap 1.5 | e_above_hull 2.0 | element+hull joint 2.25 |
| sampler 150 steps, sigma 0.002 to 80, rho 7, S_churn 60 |
| ``` |
|
|
| --- |
|
|
| ## `text_conditioning/` — Text condition vectors and encoders |
| |
| | File | Size | Description | |
| |---|---:|---| |
| | `val_text_xrd_z.pt` | 22.1 MB | 18,094 pre-encoded vectors, 256-d each | |
| | `trimodal_best.pt` | 51.4 MB | Crystal-text-XRD tri-modal encoder | |
| | `qwen_latent_adapter_best.pt` | 9.5 MB | Qwen hidden states to 256-d vector adapter | |
| | `val_text_xrd_z.pt.meta.json` | — | Provenance metadata | |
|
|
| `val_text_xrd_z.pt` is a dict keyed as: |
|
|
| ``` |
| mp-865981__text text modality vector |
| mp-865981__xrd XRD modality vector |
| ``` |
|
|
| Conditioning on these pre-encoded vectors requires no language model. Encoding new text requires Qwen3.6-35B-A3B (revision `995ad96eacd98c81ed38be0c5b274b04031597b0`) together with the other two files. |
|
|
| Tri-modal encoder SHA-256: `fa5dbe01af4d733022ab2d2a1db0e425414e2148154254931a1acb290029ea94` |
|
|
| --- |
|
|
| ## `3D_Property/` — Property prediction (LoRA) |
| |
| Adapter weights on top of Qwen3.6-35B-A3B. The base model is neither included here nor modified. |
| |
| | File | Size | Description | |
| |---|---:|---| |
| | `step_6500/lora_adapter/` | 613.5 MB | LoRA weights, PEFT standard format | |
| | `step_6500/projector.pt` | 18.9 MB | Per-atom features to Qwen embedding space | |
| | `step_6500/trainer_state.pt` | 1265.1 MB | Optimizer moments, LR schedule, RNG, sampler state | |
| | `step_6500/checkpoint_metadata.json` | — | Training metadata | |
| | `stage1_projector.pt` | 18.9 MB | Stage-1 projector | |
|
|
| | Property | Value | |
| |---|---| |
| | Base model | Qwen3.6-35B-A3B @ `995ad96eacd98c81ed38be0c5b274b04031597b0` | |
| | LoRA | r=128, alpha=256, dropout=0.05 | |
| | Attached to | 250 linear layers (`q/k/v/o_proj`, `gate/up/down_proj`, `in_proj_qkv`, `in_proj_z`, `out_proj`) | |
| | Trainable parameters | 153,354,240 (~0.44% of base) | |
| | Progress | step 6500 / 12000 | |
| | Reference environment | H200 single GPU, ~105.7 GB VRAM, effective batch 256 | |
|
|
| ```python |
| from peft import PeftModel |
| from transformers import AutoModelForCausalLM |
| |
| base = AutoModelForCausalLM.from_pretrained( |
| "Qwen/Qwen3.6-35B-A3B", |
| revision="995ad96eacd98c81ed38be0c5b274b04031597b0", |
| ) |
| model = PeftModel.from_pretrained(base, "weights/3D_Property/step_6500/lora_adapter") |
| ``` |
|
|
| `projector.pt` is loaded separately to produce the per-atom soft tokens. `trainer_state.pt` is only needed to resume training and can be skipped for inference. |
|
|
| --- |
|
|
| ## License |
|
|
| The denoising trunk inside `generator_v41/best.pt` derives from the Crystalite research backbone, MIT License, Copyright (c) 2026 Joshua Rosenthal. The `3D_Property/` LoRA weights derive from Qwen3.6-35B-A3B (Apache-2.0). Full third-party attribution is in the `NOTICE` file of the code repository. |
|
|
