README.md

metadata

license: apache-2.0
base_model: tiiuae/falcon-mamba-7b-instruct
tags:
  - mamba
  - state-space-model
  - ssm
  - chain-of-thought
  - reasoning
  - bespoke-stratos
  - peft
  - lora
library_name: peft
pipeline_tag: text-generation

Mamba-7B-Reasoning: Instilling Chain-of-Thought () Reasoning into Selective State Space Models

This repository contains the fine-tuned weights, model card, source code, evaluation benchmarks, presentation slides, and dataset processing scripts for Mamba-7B-Reasoning.

Published to namanadep Hugging Face profile using NAMAN_HF_TOKEN.

---

🎯 Primary Project Highlights & Proof of Work

1. Architecture Shift: Fine-tuned Mamba's linear projection layers (in_proj, x_proj, dt_proj) using LoRA ($r=16, lpha=32$) with bfloat16 precision across 2x NVIDIA H200 NVL GPUs.
2. Dataset Pipeline: Processed 16,710 DeepSeek-R1 distilled reasoning samples (BespokeLabs/Bespoke-Stratos-17k) into structured <think>...</think> CoT conversation format.
3. 50-Prompt Empirical Evaluation: Evaluated Base Mamba 7B vs. Fine-Tuned Mamba Reasoning across 50 technical benchmarks spanning Math Logic, Systems Code, Cryptography, and AI Theory.
4. Key Finding: Achieved 100% <think> CoT trigger rate with a 1.85x content density expansion while maintaining Mamba's constant $O(1)$ memory state and sub-4-second response latency.

---

📂 Uploaded Artifacts & Project Inventory

• adapter/: Fine-Tuned PyTorch LoRA Model Adapter Weights (adapter_model.safetensors, adapter_config.json).
• docs/50_PROMPTS_MAMBA_BASE_VS_REASONING_COMPARISON.md: 215 KB Side-by-Side 50-Prompt Evaluation Report.
• docs/MAMBA_FINETUNING_PROOF_OF_WORK_PRESENTATION.pptx: First-Person Proof of Work PowerPoint Deck.
• docs/MAMBA_REASONING_FINETUNING_PLAN.md: Fine-Tuning Strategy & Implementation Plan.
• src/: Complete PyTorch, PEFT, Dataset Processing & Evaluation Source Code.
• data/50_prompts_reasoning_results.json: Raw Evaluation Transcripts and Execution Logs.