---
license: mit
tags:
- Jerome Powell AI model
- Federal Reserve chatbot
- fine-tuned Phi-3
- financial language model
- LLM fine-tuning
- machine learning engineering
- LoRA training
- NLP
datasets:
- BoostedJonP/JeromePowell-SFT
language:
- en
base_model:
- microsoft/Phi-3-mini-4k-instruct
pipeline_tag: text-generation
---

# Powell-Phi3-Mini — Jerome Powell Style Language Model

[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-yellow)](https://huggingface.co/BoostedJonP/powell-phi3-mini)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
[![GPU Training](https://img.shields.io/badge/Trained%20on-TESLA%20P100-green)](https://images.nvidia.com/content/tesla/pdf/nvidia-tesla-p100-PCIe-datasheet.pdf)
[![Fine-tuning](https://img.shields.io/badge/Method-LoRA%2FQLoRA-orange)](https://arxiv.org/abs/2106.09685)

## 🎯Summary

**Powell-Phi3-Mini** is an fine-tuned language model that replicates Federal Reserve Chair Jerome Powell's distinctive communication style, tone, and strategic hedging patterns. This project showcases expertise in **modern LLM fine-tuning techniques**, **parameter-efficient training methods**, and **responsible AI development** — demonstrating industry-ready machine learning engineering skills.

---

## 🚀 Key Features & Capabilities

### **Style Mimicry & Linguistic Analysis**
- ✅ **Authentic Communication Style**: Replicates Powell's cautious, data-dependent rhetoric
- ✅ **Strategic Hedging Patterns**: Maintains appropriate uncertainty in speculative scenarios  
- ✅ **Domain-Specific Responses**: Handles economic and monetary policy discussions contextually
- ✅ **Refusal Training**: Appropriately declines to provide financial advice or policy predictions (to an extent)

### **Technical Implementation**
- ✅ **Efficient Architecture**: Built on Microsoft Phi-3-mini-4k-instruct (3.8B parameters)
- ✅ **Scalable Training**: LoRA r=16, alpha=32 configuration optimized for consumer GPUs
- ✅ **Deployment Flexibility**: Available as lightweight adapter or full merged model
- ✅ **Integration Ready**: One-line inference with Hugging Face Transformers

---

## 💻 Implementation Examples

### Production Ready - Merged Model
```python
from transformers import AutoTokenizer, AutoModelForCausalLM

# One-line model loading
tokenizer = AutoTokenizer.from_pretrained("BoostedJonP/powell-phi3-mini")
model = AutoModelForCausalLM.from_pretrained("BoostedJonP/powell-phi3-mini", device_map="auto")

# Economic analysis prompt
prompt = "How is the current labor market affecting your inflation outlook?"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
response = model.generate(**inputs, max_new_tokens=200, do_sample=True)
print(tokenizer.decode(response[0], skip_special_tokens=True))
```

---

## 📊 Technical Specifications & Training Pipeline

### **Model Architecture**
| Component | Specification |
|-----------|---------------|
| **Base Model** | microsoft/Phi-3-mini-4k-instruct (3.8B parameters) |
| **License** | MIT License (Commercial Use Approved) |
| **Fine-tuning Method** | QLoRA with PEFT integration |
| **Context Length** | 4,096 tokens |
| **Training Hardware** | NVIDIA TESLA P100 (16GB VRAM) |

### **Training Configuration**
| Hyperparameter | Value | Rationale |
|----------------|-------|-----------|
| **LoRA Rank (r)** | 16 | Optimal parameter/performance balance |
| **LoRA Alpha** | 32 | 2x rank for stable training |
| **Dropout Rate** | 0.05 | Regularization without overfitting |
| **Learning Rate** | 1.5e-4 | Conservative rate for stable convergence |
| **Scheduler** | Cosine decay | Smooth learning rate reduction |
| **Training Epochs** | 3 | Prevents overfitting on specialized domain |
| **Sequence Length** | 1,536 tokens | Optimized for dataset |
| **Precision** | Mixed fp16 | 2x memory efficiency, maintained accuracy |

### **Dataset & Methodology**
- **Data Source**: Public domain FOMC transcripts and Federal Reserve speeches -> [Jerome Powell Press Release Q&A](https://www.kaggle.com/datasets/jonathanpaserman/fed-press-release-text)
- **Data Processing**: Instruction-response pairs extracted from press conferences -> [Jerome Powell Press Release SFT data processing](https://www.kaggle.com/code/jonathanpaserman/jerome-powell-press-release-sft-data-processing)
- Available on [HuggingFace](https://huggingface.co/datasets/BoostedJonP/JeromePowell-SFT)
- **Quality Control**: Manual review and filtering for authentic Powell communication patterns

---

## 📈 Performance Metrics & Evaluation

### **Quantitative Results**
| Metric | Baseline (Phi-3) | Powell-Phi3-Mini | Improvement |
|--------|------------------|------------------|-------------|
| **Powell-style Classification** |NA | NA | **NA** |
| **Economic Domain Accuracy** |NA | NA | **NA** |
| **Response Coherence (BLEU)**|NA | NA | **NA** |

### **Qualitative Assessment**
- NA

---

## 🌐 Deployment & Access

### **🚀 Live Demo**
**[Try Powell-Phi3-Mini Interactive Demo →](https://huggingface.co/spaces/BoostedJonP/powell-assistant)**

### **📦 Model Downloads**
- **Adapter Version**: `BoostedJonP/powell-phi3-mini-adapter`
- **Merged Model**: `BoostedJonP/powell-phi3-mini` (Full Model - 7.4GB)

### **🔗 Resources**
- **[GitHub Repository](https://github.com/BigJonP/powell-phi3-sft)**: Complete training code and evaluation scripts
- **[Technical Blog Post](https://medium.com/@jonathanpaserman)**: Detailed implementation walkthrough
- **[Hugging Face Collection](https://huggingface.co/collections/BoostedJonP/jerome-powell-68b9e7843f64507481d24ce9)**: All model variants and datasets

---

## ⚖️ Responsible AI & Legal Compliance

### **Ethical Considerations**
- ⚠️ **No Official Affiliation**: Not endorsed by or affiliated with the Federal Reserve System
- ⚠️ **Educational Purpose Only**: Designed for research, education, and demonstration purposes
- ⚠️ **No Financial Advice**: Model responses should not be interpreted as investment guidance
- ⚠️ **Transparency**: All training data sourced from public domain government transcripts

### **Licensing & Usage Rights**
- **Base Model License**: MIT License (Microsoft Phi-3)
- **Fine-tuned Weights**: MIT License (Commercial use permitted)
- **Training Data**: Public domain (U.S. government works)
- **Usage**: Unrestricted for research, education, and commercial applications

---

### 👨‍💻 **Connect & Collaborate**
- **GitHub**: [Jonathan Paserman](https://github.com/BigJonP)
- **Kaggle**: [Jonathan Paserman](https://www.kaggle.com/jonathanpaserman)
- **HuggingFace**: [Jonathan Paserman](https://huggingface.co/BoostedJonP)