SurfaceScienceAssistant: A Fine-Tuned Llama-3.1-8B for On-Surface Synthesis Reasoning
1. Overview
SurfaceScienceAssistant is a specialized Large Language Model (LLM) fine-tuned for reasoning and knowledge synthesis in the field of Surface Chemistry and On-Surface Synthesis (OSS).
While general-purpose models often struggle with the niche nomenclature and complex mechanistic logic of surface-confined reactions, this model has been specifically adapted to understand the synergistic interactions between organic precursors and metal substrates (e.g., Au, Ag, Cu). It serves as the intelligent reasoning core for the OSS Assistant platform, providing expert-level insights into reaction pathways.
2. Model Specifications
- Developer: Materials Genome Institute (MGI), Shanghai University.
- Architecture: Meta-Llama-3.1-8B (Transformer-based).
- Adaptation Method: QLoRA (4-bit Low-Rank Adaptation) — ensuring a balance between maintaining foundational scientific priors and acquiring domain-specific expertise.
- Primary Objective: Targeted Scientific Reasoning — This model has been specifically fine-tuned to master the intricate nuances of surface science, enabling dialogue regarding surface-confined chemical reactions, substrate-molecule interaction
- Training Infrastructure: BitsAndBytes 4-bit NormalFloat (NF4) quantization with gradient checkpointing for resource-efficient, high-performance training.
3. Training & Dataset
The model underwent domain-specific instruction tuning using the Surface_Chemistry dataset. This curated collection integrates high-quality instruction-following pairs derived from extensive surface science literature and expert-annotated reasoning chains.
- Mechanistic Reasoning: On-surface reactions and molecular self-assembly, facilitating precise functionalization and structural evolution.
- Substrate Effects: The influence of crystalline facets and electronic structures on molecular behavior.
Hyperparameters:
| Parameter | Value | Description |
|---|---|---|
| LoRA Rank ($r$) | 16 | Capturing domain-specific manifolds |
| LoRA Alpha ($\alpha$) | 32 | Scaling factor for stable weight updates |
| Learning Rate | 2e-4 | Optimized for Llama-3.1 backbone |
| Effective Batch Size | 4 | Achieved via Gradient Accumulation |
4. Key Capabilities
Expert-Level QA: Provides grounded answers regarding surface-confined reaction conditions and catalyst-precursor interactions. Mechanistic Insight: Capable of reasoning through multi-step debromination or cyclodehydrogenation processes.
5. Usage & Implementation
This model is best utilized with a system prompt that defines its role as a "Surface Science Expert."
Example Inference (Python):
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "JuanXiang-SHU/SurfaceScienceAssistant"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", load_in_4bit=True)
messages = [
{"role": "system", "content": "You are a professional researcher in Surface Chemistry."},
{"role": "user", "content": "How does the Au(111) substrate influence the Ullmann coupling of 1,4-dibromobenzene compared to Cu(111)?"}
]