| --- |
| license: apache-2.0 |
| base_model: tiiuae/falcon-mamba-7b-instruct |
| tags: |
| - mamba |
| - state-space-model |
| - ssm |
| - chain-of-thought |
| - reasoning |
| - bespoke-stratos |
| - peft |
| - lora |
| library_name: peft |
| pipeline_tag: text-generation |
| --- |
| |
| # Mamba-7B-Reasoning: Instilling Chain-of-Thought (<think>) Reasoning into Selective State Space Models |
|
|
| This repository contains the fine-tuned weights, model card, source code, evaluation benchmarks, presentation slides, and dataset processing scripts for **Mamba-7B-Reasoning**. |
|
|
| Published to **namanadep** Hugging Face profile using `NAMAN_HF_TOKEN`. |
|
|
| --- |
|
|
| ## 🎯 Primary Project Highlights & Proof of Work |
|
|
| 1. **Architecture Shift**: Fine-tuned Mamba's linear projection layers (`in_proj`, `x_proj`, `dt_proj`) using LoRA ($r=16, lpha=32$) with `bfloat16` precision across 2x NVIDIA H200 NVL GPUs. |
| 2. **Dataset Pipeline**: Processed 16,710 DeepSeek-R1 distilled reasoning samples ([`BespokeLabs/Bespoke-Stratos-17k`](https://huggingface.co/datasets/BespokeLabs/Bespoke-Stratos-17k)) into structured `<think>...</think>` CoT conversation format. |
| 3. **50-Prompt Empirical Evaluation**: Evaluated Base Mamba 7B vs. Fine-Tuned Mamba Reasoning across 50 technical benchmarks spanning Math Logic, Systems Code, Cryptography, and AI Theory. |
| 4. **Key Finding**: Achieved **100% `<think>` CoT trigger rate** with a **1.85x content density expansion** while maintaining Mamba's constant $O(1)$ memory state and sub-4-second response latency. |
|
|
| --- |
|
|
| ## 📂 Uploaded Artifacts & Project Inventory |
|
|
| - `adapter/`: Fine-Tuned PyTorch LoRA Model Adapter Weights (`adapter_model.safetensors`, `adapter_config.json`). |
| - `docs/50_PROMPTS_MAMBA_BASE_VS_REASONING_COMPARISON.md`: 215 KB Side-by-Side 50-Prompt Evaluation Report. |
| - `docs/MAMBA_FINETUNING_PROOF_OF_WORK_PRESENTATION.pptx`: First-Person Proof of Work PowerPoint Deck. |
| - `docs/MAMBA_REASONING_FINETUNING_PLAN.md`: Fine-Tuning Strategy & Implementation Plan. |
| - `src/`: Complete PyTorch, PEFT, Dataset Processing & Evaluation Source Code. |
| - `data/50_prompts_reasoning_results.json`: Raw Evaluation Transcripts and Execution Logs. |
|
|
