Yangfan78
/

3D_LLM_Diffusion

@@ -7,118 +7,90 @@ tags:
   - lora
 ---
-# 3D_LLM_Diffusion — 模型权重
-本仓库只存放权重文件。代码与使用文档：https://github.com/Richardyangfan78/3D_LLM_Diffusion
 ```bash
 hf download Yangfan78/3D_LLM_Diffusion --local-dir weights
 ```
-三组互相独立的权重，可按需单独下载。
 ---
-## 1. `generator_v41/` — 条件晶体生成器
-| 文件 | 大小 | 内容 |
 |---|---:|---|
-| `best.pt` | 378.9 MB | 生成器全部权重。**含基座主干**，加载时不需要额外的底座文件 |
-| `priors/train.csv` | 75.2 MB | 采样时读取：原子数分布先验 + 允许元素集合 |
-| `priors/val.csv` | 25.0 MB | 验证集划分 |
-| `eval/` | — | 四份评测记录，见下 |
-**权重规格**
-| 项 | 值 |
 |---|---|
-| 架构标识 | `mattergen-spacegroup-hhi-qwen-crossattention-crystalite-v8` |
-| 参数量 | 94.63 M（777 个张量）|
-| 维度 | d_model 512 / 14 层 / 16 头 |
-| 训练步数 | 6000 |
-| 原子类型编码 | `subatomic_tokenizer_pca_16`（16 维连续向量）|
-| 晶格表示 | `ltri` |
 | SHA-256 | `cd67729bd862537be9113146ebdf48db81f86a53a4a32cd99c751bba0e06fa4a` |
-**参数构成**
-| 模块 | 参数 | 占比 |
-|---|---:|---:|
-| 去噪主干（14 层 Transformer） | 66.17 M | 69.9% |
-| 文本交叉注意力 | 14.74 M | 15.6% |
-| 逐层 FiLM 条件调制 | 3.67 M | 3.9% |
-| 分路由低秩适配器 | 3.21 M | 3.4% |
-| 条件编码器（元素/物性/空间群/文本） | 5.10 M | 5.4% |
-| 输入嵌入与输出头 | 1.74 M | 1.8% |
-**权重内固化的条件配置**
-支持 8 个条件通道：`chemical_system`、`space_group`、`dft_band_gap`、`dft_bulk_modulus`、`dft_mag_density`、`energy_above_hull`、`hhi_score`、`ml_bulk_modulus`。
-推理策略与采样参数已写入 checkpoint，加载后自动生效：
 ```
-引导强度   全局 1.0 | band_gap 1.5 | e_above_hull 2.0 | 元素+hull 联合 2.25
-采样器     150 步，σ 0.002→80，rho 7，S_churn 60
 ```
-**`eval/` 内的记录**
-| 文件 | 内容 |
-|---|---|
-| `mattergen_capabilities.json` | 8 个条件通道 + 4 个联合反事实，每场景 n=512，14/14 项通过 |
-| `zero_condition_parity.json` | 条件置零时与无条件基座逐比特一致（170 个张量，差值全为 0）|
-| `policy_pin_audit.json` | 推理策略固化审计，777 个张量逐一校验 |
-| `text_control_eval/` | 文本条件对照实验记录 |
 ---
-## 2. `text_conditioning/` — 文本条件向量与编码器
-| 文件 | 大小 | 内容 |
 |---|---:|---|
-| `val_text_xrd_z.pt` | 22.1 MB | **18094 条预编码向量**，每条 256 维 |
-| `trimodal_best.pt` | 51.4 MB | 晶体-文本-XRD 三模态编码器 |
-| `qwen_latent_adapter_best.pt` | 9.5 MB | Qwen 隐状态 → 256 维向量的适配器 |
-| `val_text_xrd_z.pt.meta.json` | — | 溯源元数据 |
-`val_text_xrd_z.pt` 是一个 dict，key 格式：
 ```
-mp-865981__text     文本模态向量
-mp-865981__xrd      XRD 模态向量
 ```
-用这些预编码向量做条件生成不需要语言模型。若要编码新文本，则需自备 Qwen3.6-35B-A3B（revision `995ad96eacd98c81ed38be0c5b274b04031597b0`）配合另两个文件使用。
-三模态编码器 SHA-256：`fa5dbe01af4d733022ab2d2a1db0e425414e2148154254931a1acb290029ea94`
 ---
-## 3. `3D_Property/` — 物性预测模型（LoRA）
-在 Qwen3.6-35B-A3B 之上的外挂权重，**基座本体未包含在此仓库、也未被修改**。
-| 文件 | 大小 | 内容 |
 |---|---:|---|
-| `step_6500/lora_adapter/` | 613.5 MB | LoRA 权重，PEFT 标准格式 |
-| `step_6500/projector.pt` | 18.9 MB | 每原子特征 → Qwen 嵌入空间的投影器 |
-| `step_6500/trainer_state.pt` | 1265.1 MB | 优化器动量、LR 调度进度、RNG、采样器状态 |
-| `step_6500/checkpoint_metadata.json` | — | 训练元数据 |
-| `stage1_projector.pt` | 18.9 MB | Stage-1 阶段的投影器 |
-**权重规格**
-| 项 | 值 |
 |---|---|
-| 基座模型 | Qwen3.6-35B-A3B @ `995ad96eacd98c81ed38be0c5b274b04031597b0` |
-| LoRA | r=128，alpha=256，dropout=0.05 |
-| 挂载位置 | 250 个线性层（`q/k/v/o_proj`、`gate/up/down_proj`、`in_proj_qkv`、`in_proj_z`、`out_proj`）|
-| 可训练参数 | 153,354,240（约占基座 0.44%）|
-| 训练进度 | step 6500 / 12000 |
-| 参考运行环境 | H200 单卡，显存占用约 105.7 GB，有效 batch 256 |
-**加载 LoRA**
 ```python
 from peft import PeftModel
@@ -131,12 +103,10 @@ base = AutoModelForCausalLM.from_pretrained(
 model = PeftModel.from_pretrained(base, "weights/3D_Property/step_6500/lora_adapter")
 ```
-`projector.pt` 需单独加载以生成每原子软 token；`trainer_state.pt` 仅在继续训练时需要，纯推理可不下载。
 ---
-## 许可
-- `generator_v41/best.pt` 中的去噪主干部分派生自 Crystalite research backbone，MIT License, Copyright (c) 2026 Joshua Rosenthal。
-- `3D_Property/` 的 LoRA 权重派生自 Qwen3.6-35B-A3B（Apache-2.0）。
-- 完整的第三方声明见代码仓库的 `NOTICE`。

   - lora
 ---
+# 3D_LLM_Diffusion — Model Weights
+Weights only. Code and usage documentation: https://github.com/Richardyangfan78/3D_LLM_Diffusion
 ```bash
 hf download Yangfan78/3D_LLM_Diffusion --local-dir weights
 ```
+Three independent weight sets.
 ---
+## `generator_v41/` — Conditional crystal generator
+| File | Size | Description |
 |---|---:|---|
+| `best.pt` | 378.9 MB | All generator weights, backbone included |
+| `priors/train.csv` | 75.2 MB | Read at sampling time: atom-count prior + allowed element set |
+| `priors/val.csv` | 25.0 MB | Validation split |
+| `eval/` | — | Evaluation records |
+| Property | Value |
 |---|---|
+| Architecture ID | `mattergen-spacegroup-hhi-qwen-crossattention-crystalite-v8` |
+| Parameters | 94.63 M (777 tensors) |
+| Dimensions | d_model 512 / 14 layers / 16 heads |
+| Training step | 6000 |
+| Atom type encoding | `subatomic_tokenizer_pca_16` (16-d continuous) |
+| Lattice representation | `ltri` |
 | SHA-256 | `cd67729bd862537be9113146ebdf48db81f86a53a4a32cd99c751bba0e06fa4a` |
+Eight condition channels: `chemical_system`, `space_group`, `dft_band_gap`, `dft_bulk_modulus`, `dft_mag_density`, `energy_above_hull`, `hhi_score`, `ml_bulk_modulus`.
+The inference policy and sampler settings are stored inside the checkpoint and apply automatically:
 ```
+guidance   global 1.0 | band_gap 1.5 | e_above_hull 2.0 | element+hull joint 2.25
+sampler    150 steps, sigma 0.002 to 80, rho 7, S_churn 60
 ```
 ---
+## `text_conditioning/` — Text condition vectors and encoders
+| File | Size | Description |
 |---|---:|---|
+| `val_text_xrd_z.pt` | 22.1 MB | 18,094 pre-encoded vectors, 256-d each |
+| `trimodal_best.pt` | 51.4 MB | Crystal-text-XRD tri-modal encoder |
+| `qwen_latent_adapter_best.pt` | 9.5 MB | Qwen hidden states to 256-d vector adapter |
+| `val_text_xrd_z.pt.meta.json` | — | Provenance metadata |
+`val_text_xrd_z.pt` is a dict keyed as:
 ```
+mp-865981__text     text modality vector
+mp-865981__xrd      XRD modality vector
 ```
+Conditioning on these pre-encoded vectors requires no language model. Encoding new text requires Qwen3.6-35B-A3B (revision `995ad96eacd98c81ed38be0c5b274b04031597b0`) together with the other two files.
+Tri-modal encoder SHA-256: `fa5dbe01af4d733022ab2d2a1db0e425414e2148154254931a1acb290029ea94`
 ---
+## `3D_Property/` — Property prediction (LoRA)
+Adapter weights on top of Qwen3.6-35B-A3B. The base model is neither included here nor modified.
+| File | Size | Description |
 |---|---:|---|
+| `step_6500/lora_adapter/` | 613.5 MB | LoRA weights, PEFT standard format |
+| `step_6500/projector.pt` | 18.9 MB | Per-atom features to Qwen embedding space |
+| `step_6500/trainer_state.pt` | 1265.1 MB | Optimizer moments, LR schedule, RNG, sampler state |
+| `step_6500/checkpoint_metadata.json` | — | Training metadata |
+| `stage1_projector.pt` | 18.9 MB | Stage-1 projector |
+| Property | Value |
 |---|---|
+| Base model | Qwen3.6-35B-A3B @ `995ad96eacd98c81ed38be0c5b274b04031597b0` |
+| LoRA | r=128, alpha=256, dropout=0.05 |
+| Attached to | 250 linear layers (`q/k/v/o_proj`, `gate/up/down_proj`, `in_proj_qkv`, `in_proj_z`, `out_proj`) |
+| Trainable parameters | 153,354,240 (~0.44% of base) |
+| Progress | step 6500 / 12000 |
+| Reference environment | H200 single GPU, ~105.7 GB VRAM, effective batch 256 |
 ```python
 from peft import PeftModel
 model = PeftModel.from_pretrained(base, "weights/3D_Property/step_6500/lora_adapter")
 ```
+`projector.pt` is loaded separately to produce the per-atom soft tokens. `trainer_state.pt` is only needed to resume training and can be skipped for inference.
 ---
+## License
+The denoising trunk inside `generator_v41/best.pt` derives from the Crystalite research backbone, MIT License, Copyright (c) 2026 Joshua Rosenthal. The `3D_Property/` LoRA weights derive from Qwen3.6-35B-A3B (Apache-2.0). Full third-party attribution is in the `NOTICE` file of the code repository.