BoostedJonP commited on
Commit
c92d403
Β·
1 Parent(s): 6d53044

readme update

Browse files
Files changed (1) hide show
  1. README.md +139 -0
README.md CHANGED
@@ -1,3 +1,142 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ tags:
4
+ - Jerome Powell AI model
5
+ - Federal Reserve chatbot
6
+ - fine-tuned Phi-3
7
+ - financial language model
8
+ - LLM fine-tuning
9
+ - machine learning engineering
10
+ - LoRA training
11
+ - NLP
12
  ---
13
+
14
+ # Powell-Phi3-Mini β€” Jerome Powell Style Language Model
15
+
16
+ [![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-yellow)](https://huggingface.co/BoostedJonP/powell-phi3-mini)
17
+ [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
18
+ [![GPU Training](https://img.shields.io/badge/Trained%20on-TESLA%20P100-green)](https://images.nvidia.com/content/tesla/pdf/nvidia-tesla-p100-PCIe-datasheet.pdf)
19
+ [![Fine-tuning](https://img.shields.io/badge/Method-LoRA%2FQLoRA-orange)](https://arxiv.org/abs/2106.09685)
20
+
21
+ ## 🎯Summary
22
+
23
+ **Powell-Phi3-Mini** is an fine-tuned language model that replicates Federal Reserve Chair Jerome Powell's distinctive communication style, tone, and strategic hedging patterns. This project showcases expertise in **modern LLM fine-tuning techniques**, **parameter-efficient training methods**, and **responsible AI development** β€” demonstrating industry-ready machine learning engineering skills.
24
+
25
+ ---
26
+
27
+ ## πŸš€ Key Features & Capabilities
28
+
29
+ ### **Style Mimicry & Linguistic Analysis**
30
+ - βœ… **Authentic Communication Style**: Replicates Powell's cautious, data-dependent rhetoric
31
+ - βœ… **Strategic Hedging Patterns**: Maintains appropriate uncertainty in speculative scenarios
32
+ - βœ… **Domain-Specific Responses**: Handles economic and monetary policy discussions contextually
33
+ - βœ… **Refusal Training**: Appropriately declines to provide financial advice or policy predictions (to an extent)
34
+
35
+ ### **Technical Implementation**
36
+ - βœ… **Efficient Architecture**: Built on Microsoft Phi-3-mini-4k-instruct (3.8B parameters)
37
+ - βœ… **Scalable Training**: LoRA r=16, alpha=32 configuration optimized for consumer GPUs
38
+ - βœ… **Deployment Flexibility**: Available as lightweight adapter or full merged model
39
+ - βœ… **Integration Ready**: One-line inference with Hugging Face Transformers
40
+
41
+ ---
42
+
43
+ ## πŸ’» Implementation Examples
44
+
45
+ ### Production Ready - Merged Model
46
+ ```python
47
+ from transformers import AutoTokenizer, AutoModelForCausalLM
48
+
49
+ # One-line model loading
50
+ tokenizer = AutoTokenizer.from_pretrained("BoostedJonP/powell-phi3-mini")
51
+ model = AutoModelForCausalLM.from_pretrained("BoostedJonP/powell-phi3-mini", device_map="auto")
52
+
53
+ # Economic analysis prompt
54
+ prompt = "How is the current labor market affecting your inflation outlook?"
55
+ inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
56
+ response = model.generate(**inputs, max_new_tokens=200, do_sample=True)
57
+ print(tokenizer.decode(response[0], skip_special_tokens=True))
58
+ ```
59
+
60
+ ---
61
+
62
+ ## πŸ“Š Technical Specifications & Training Pipeline
63
+
64
+ ### **Model Architecture**
65
+ | Component | Specification |
66
+ |-----------|---------------|
67
+ | **Base Model** | microsoft/Phi-3-mini-4k-instruct (3.8B parameters) |
68
+ | **License** | MIT License (Commercial Use Approved) |
69
+ | **Fine-tuning Method** | QLoRA with PEFT integration |
70
+ | **Context Length** | 4,096 tokens |
71
+ | **Training Hardware** | NVIDIA TESLA P100 (16GB VRAM) |
72
+
73
+ ### **Training Configuration**
74
+ | Hyperparameter | Value | Rationale |
75
+ |----------------|-------|-----------|
76
+ | **LoRA Rank (r)** | 16 | Optimal parameter/performance balance |
77
+ | **LoRA Alpha** | 32 | 2x rank for stable training |
78
+ | **Dropout Rate** | 0.05 | Regularization without overfitting |
79
+ | **Learning Rate** | 1.5e-4 | Conservative rate for stable convergence |
80
+ | **Scheduler** | Cosine decay | Smooth learning rate reduction |
81
+ | **Training Epochs** | 3 | Prevents overfitting on specialized domain |
82
+ | **Sequence Length** | 1,536 tokens | Optimized for dataset |
83
+ | **Precision** | Mixed fp16 | 2x memory efficiency, maintained accuracy |
84
+
85
+ ### **Dataset & Methodology**
86
+ - **Data Source**: Public domain FOMC transcripts and Federal Reserve speeches -> [Jerome Powell Press Release Q&A](https://www.kaggle.com/datasets/jonathanpaserman/fed-press-release-text)
87
+ - **Data Processing**: Instruction-response pairs extracted from press conferences -> [Jerome Powell Press Release SFT data processing](https://www.kaggle.com/code/jonathanpaserman/jerome-powell-press-release-sft-data-processing)
88
+ - Available on [HuggingFace](https://huggingface.co/datasets/BoostedJonP/JeromePowell-SFT)
89
+ - **Quality Control**: Manual review and filtering for authentic Powell communication patterns
90
+
91
+ ---
92
+
93
+ ## πŸ“ˆ Performance Metrics & Evaluation
94
+
95
+ ### **Quantitative Results**
96
+ | Metric | Baseline (Phi-3) | Powell-Phi3-Mini | Improvement |
97
+ |--------|------------------|------------------|-------------|
98
+ | **Powell-style Classification** |NA | NA | **NA** |
99
+ | **Economic Domain Accuracy** |NA | NA | **NA** |
100
+ | **Response Coherence (BLEU)**|NA | NA | **NA** |
101
+
102
+ ### **Qualitative Assessment**
103
+ - NA
104
+
105
+ ---
106
+
107
+ ## 🌐 Deployment & Access
108
+
109
+ ### **πŸš€ Live Demo**
110
+ **[Try Powell-Phi3-Mini Interactive Demo β†’](https://huggingface.co/spaces/BoostedJonP/powell-phi3-demo)**
111
+
112
+ ### **πŸ“¦ Model Downloads**
113
+ - **Adapter Version**: `BoostedJonP/powell-phi3-mini-adapter`
114
+ - **Merged Model**: `BoostedJonP/powell-phi3-mini` (Full Model - 7.4GB)
115
+
116
+ ### **πŸ”— Resources**
117
+ - **[GitHub Repository](https://github.com/BigJonP/powell-phi3-sft)**: Complete training code and evaluation scripts
118
+ - **[Technical Blog Post](https://medium.com/@jonathanpaserman)**: Detailed implementation walkthrough
119
+ - **[Hugging Face Collection](https://huggingface.co/collections/BoostedJonP/jerome-powell-68b9e7843f64507481d24ce9)**: All model variants and datasets
120
+
121
+ ---
122
+
123
+ ## βš–οΈ Responsible AI & Legal Compliance
124
+
125
+ ### **Ethical Considerations**
126
+ - ⚠️ **No Official Affiliation**: Not endorsed by or affiliated with the Federal Reserve System
127
+ - ⚠️ **Educational Purpose Only**: Designed for research, education, and demonstration purposes
128
+ - ⚠️ **No Financial Advice**: Model responses should not be interpreted as investment guidance
129
+ - ⚠️ **Transparency**: All training data sourced from public domain government transcripts
130
+
131
+ ### **Licensing & Usage Rights**
132
+ - **Base Model License**: MIT License (Microsoft Phi-3)
133
+ - **Fine-tuned Weights**: MIT License (Commercial use permitted)
134
+ - **Training Data**: Public domain (U.S. government works)
135
+ - **Usage**: Unrestricted for research, education, and commercial applications
136
+
137
+ ---
138
+
139
+ ### πŸ‘¨β€πŸ’» **Connect & Collaborate**
140
+ - **GitHub**: [Jonathan Paserman](https://github.com/BigJonP)
141
+ - **Kaggle**: [Jonathan Paserman](https://www.kaggle.com/jonathanpaserman)
142
+ - **HuggingFace**: [Jonathan Paserman](https://huggingface.co/BoostedJonP)