| # SurfaceScienceAssistant: A Fine-Tuned Llama-3.1-8B for On-Surface Synthesis Reasoning |
|
|
| [](https://opensource.org/licenses/Apache-2.0) |
| [](https://huggingface.co/meta-llama/Llama-3.1-8B) |
| [](https://en.wikipedia.org/wiki/Surface_science) |
|
|
| ## 1. Overview |
| **SurfaceScienceAssistant** is a specialized Large Language Model (LLM) fine-tuned for reasoning and knowledge synthesis in the field of **Surface Chemistry** and **On-Surface Synthesis (OSS)**. |
|
|
| While general-purpose models often struggle with the niche nomenclature and complex mechanistic logic of surface-confined reactions, this model has been specifically adapted to understand the synergistic interactions between organic precursors and metal substrates (e.g., Au, Ag, Cu). It serves as the intelligent reasoning core for the **OSS Assistant** platform, providing expert-level insights into reaction pathways. |
|
|
| --- |
|
|
| ## 2. Model Specifications |
| * **Developer:** Materials Genome Institute (MGI), Shanghai University. |
| * **Architecture:** Meta-Llama-3.1-8B (Transformer-based). |
| * **Adaptation Method:** **QLoRA (4-bit Low-Rank Adaptation)** — ensuring a balance between maintaining foundational scientific priors and acquiring domain-specific expertise. |
| * **Primary Objective:** **Targeted Scientific Reasoning** — This model has been specifically fine-tuned to master the intricate nuances of surface science, enabling dialogue regarding surface-confined chemical reactions, substrate-molecule interaction |
| * **Training Infrastructure:** BitsAndBytes 4-bit NormalFloat (NF4) quantization with gradient checkpointing for resource-efficient, high-performance training. |
|
|
| --- |
| ## 3. Training & Dataset |
| The model underwent domain-specific instruction tuning using the [**Surface_Chemistry**](https://huggingface.co/datasets/JuanXiang-SHU/Surface_Chemistry) dataset. This curated collection integrates high-quality instruction-following pairs derived from extensive surface science literature and expert-annotated reasoning chains. |
| * **Mechanistic Reasoning:** On-surface reactions and molecular self-assembly, facilitating precise functionalization and structural evolution. |
| * **Substrate Effects:** The influence of crystalline facets and electronic structures on molecular behavior. |
|
|
|
|
| ### Hyperparameters: |
| | Parameter | Value | Description | |
| | :--- | :--- | :--- | |
| | **LoRA Rank ($r$)** | 16 | Capturing domain-specific manifolds | |
| | **LoRA Alpha ($\alpha$)** | 32 | Scaling factor for stable weight updates | |
| | **Learning Rate** | 2e-4 | Optimized for Llama-3.1 backbone | |
| | **Effective Batch Size** | 4 | Achieved via Gradient Accumulation | |
|
|
| --- |
|
|
| ## 4. Key Capabilities |
| **Expert-Level QA:** Provides grounded answers regarding surface-confined reaction conditions and catalyst-precursor interactions. |
| **Mechanistic Insight:** Capable of reasoning through multi-step debromination or cyclodehydrogenation processes. |
|
|
| --- |
|
|
| ## 5. Usage & Implementation |
| This model is best utilized with a system prompt that defines its role as a "Surface Science Expert." |
|
|
| ### Example Inference (Python): |
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_id = "JuanXiang-SHU/SurfaceScienceAssistant" |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
| model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", load_in_4bit=True) |
| |
| messages = [ |
| {"role": "system", "content": "You are a professional researcher in Surface Chemistry."}, |
| {"role": "user", "content": "How does the Au(111) substrate influence the Ullmann coupling of 1,4-dibromobenzene compared to Cu(111)?"} |
| ] |
