README.md

---
base_model:
- google/gemma-4-31B-it
---
  # relm-1

  `relm-1` is a specialized **post-train** of **Gemma 4 (31B-it)** designed to excel in complex reasoning, high-difficulty SQL generation, telco software, and specialized coding tasks.

  ## Model Summary
  - **Base Model:** `google/gemma-4-31B-it`
  - **Training Method:** Post-trained via LoRA (Low-Rank Adaptation) using the `scalarlm` framework, with weights **merged back into the base model** for standalone deployment.
  - **Specializations:** Complex SQL, OpenTelco (OTel), Relational AI PyRel, and General Reasoning.
  - **Context Window (SFT):** 6,144 tokens

  ## 🎯 Intended Use
  `relm-1` is designed for developers and data engineers who need a model capable of:
  - **Advanced SQL Generation:** Solving high-complexity database queries, including specialized Snowflake SQL tasks and benchmarks based on Spider-2.
  - **Relational AI PyRel:** Generating and reasoning over PyRel, a Pythonic relational language that translates high-level logic into optimized SQL.
  - **Observability Analysis:** Understanding and generating telco prompts and completions (OTel-LLM).
  - **Reasoning-Heavy Tasks:** Leveraging Chain-of-Thought (CoT) to decompose complex software engineering problems.

  ## 🛠️  Training Details

  ### Dataset Composition
  The model underwent post-training on a balanced mixture of high-quality SFT data:

  | Dataset | Weight | Focus |
  |---|---|---|
  | Relational AI & PyRel | 50% | PyRel and relational logic |
  | Spider-2 | 20% | Complex SQL generation |
  | Nemotron-Cascade-2-SFT | 10% | General chat and instruction following |
  | OTel-LLM | 10% | OpenTelco |
  | Specialized Snowflake SQL | 10% | Advanced Snowflake SQL tasks |

  ### Training Hyperparameters
  - **Learning Rate:** 2e-4
  - **Max Steps:** 6,000
  - **Gradient Accumulation:** 2
  - **Architecture:** LoRA adaptation merged into the 31B base model.

  ## ✍️  Prompting & Usage

  ### Reasoning Format
  `relm-1` utilizes Gemma 4's native channel-thought format for reasoning. To trigger Chain-of-Thought, the model is trained to follow this structure:

  ```text
  <thought>
  {Step-by-step reasoning and analysis}
  </thought>
  {Final answer}

  Specialized Directives

  The model responds to specific formatting directives injected during post-training:
  - FINOS Legend: For tasks requiring the "Pure" version of FINOS Legend, the model is optimized for the prompt: "Answer using FINOS Legend (Pure language)."
  - Hybrid SQL/PyRel: For tasks requiring dual implementations, the model is optimized for: "Provide both a SQL solution and a PyRel solution."
  
  Chat Template

  The model follows the official Gemma 4 template:

  <|user|>
  {prompt}<|end_of_turn|>
  <|model|>
  {response}<|end_of_turn|>

  ⚠️  Limitations

  - Context Window: Post-training samples were filtered to a maximum of 6,144 tokens; performance on extremely long documents may vary.
  - Base Model Dependency: While the weights are merged, the model's foundational capabilities are inherited from the Gemma 4 31B-it base.