Create README.md

README.md

@@ -0,0 +1,15 @@
+
+
+# SD-VLM-8B
+
+The SD-VLM architecture enhances a standard Vision-Language Model (VLM) with 3D spatial awareness through a minimal yet effective modification. Its core structure is built upon three key components:
+
+**1. Base VLM**: Utilizes the LLaVA-1.5-7B framework, consisting of a CLIP-ViT vision encoder, a Vicuna large language model (LLM), and a linear projector connecting them.
+
+**2. Depth Encoding Core (DPE)**: The central innovation is the Depth Positional Encoding (DPE) module. It processes an input depth map (from an external estimator like Depth-Anything-V2) to generate depth-aware embeddings (E_depth). These embeddings are then directly added to the standard image features (E_image) from the vision encoder:
+
+This simple addition injects explicit 3D spatial priors into the model without altering the backbone architecture.
+
+**3. Training Approach**: The model is efficiently fine-tuned on the MSMU spatial dataset for one epoch using LoRA, keeping the vision encoder frozen. This allows the LLM and projector to learn how to interpret the depth-enhanced visual features for quantitative reasoning.
+
+In essence, SD-VLM's structure is defined by a streamlined integration: it upgrades a standard VLM to understand 3D space by fusing depth information into visual features through a parameter-free additive operation, all trained efficiently on targeted data.