---
license: apache-2.0
base_model: tiiuae/falcon-mamba-7b-instruct
tags:
- mamba
- state-space-model
- ssm
- chain-of-thought
- reasoning
- bespoke-stratos
- peft
- lora
library_name: peft
pipeline_tag: text-generation
---

# Mamba-7B-Reasoning: Instilling Chain-of-Thought (<think>) Reasoning into Selective State Space Models

This repository contains the fine-tuned weights, model card, source code, evaluation benchmarks, presentation slides, and dataset processing scripts for **Mamba-7B-Reasoning**.

Published to **namanadep** Hugging Face profile using `NAMAN_HF_TOKEN`.

---

## 🎯 Primary Project Highlights & Proof of Work

1. **Architecture Shift**: Fine-tuned Mamba's linear projection layers (`in_proj`, `x_proj`, `dt_proj`) using LoRA ($r=16, lpha=32$) with `bfloat16` precision across 2x NVIDIA H200 NVL GPUs.
2. **Dataset Pipeline**: Processed 16,710 DeepSeek-R1 distilled reasoning samples ([`BespokeLabs/Bespoke-Stratos-17k`](https://huggingface.co/datasets/BespokeLabs/Bespoke-Stratos-17k)) into structured `<think>...</think>` CoT conversation format.
3. **50-Prompt Empirical Evaluation**: Evaluated Base Mamba 7B vs. Fine-Tuned Mamba Reasoning across 50 technical benchmarks spanning Math Logic, Systems Code, Cryptography, and AI Theory.
4. **Key Finding**: Achieved **100% `<think>` CoT trigger rate** with a **1.85x content density expansion** while maintaining Mamba's constant $O(1)$ memory state and sub-4-second response latency.

---

## 📂 Uploaded Artifacts & Project Inventory

- `adapter/`: Fine-Tuned PyTorch LoRA Model Adapter Weights (`adapter_model.safetensors`, `adapter_config.json`).
- `docs/50_PROMPTS_MAMBA_BASE_VS_REASONING_COMPARISON.md`: 215 KB Side-by-Side 50-Prompt Evaluation Report.
- `docs/MAMBA_FINETUNING_PROOF_OF_WORK_PRESENTATION.pptx`: First-Person Proof of Work PowerPoint Deck.
- `docs/MAMBA_REASONING_FINETUNING_PLAN.md`: Fine-Tuning Strategy & Implementation Plan.
- `src/`: Complete PyTorch, PEFT, Dataset Processing & Evaluation Source Code.
- `data/50_prompts_reasoning_results.json`: Raw Evaluation Transcripts and Execution Logs.