|
|
--- |
|
|
library_name: transformers |
|
|
license: mit |
|
|
datasets: |
|
|
- FinGPT/fingpt-forecaster-dow30-202305-202405 |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- EleutherAI/pythia-410m |
|
|
pipeline_tag: summarization |
|
|
--- |
|
|
# 📈 FinancialGPT — Domain-Specific LLM for Financial Forecasts |
|
|
|
|
|
FinancialGPT is a small, domain-adapted language model fine-tuned on real financial forecast data to replicate ideas from FinGPT on consumer hardware. |
|
|
|
|
|
--- |
|
|
|
|
|
## 🧩 **Project Overview** |
|
|
|
|
|
**Goal:** Fine-tune an open-source LLM to generate earnings outlooks for Dow30 companies using modern parameter-efficient methods. |
|
|
|
|
|
- **Base Model:** [Pythia-410M](https://huggingface.co/EleutherAI/pythia-410m) |
|
|
- **Techniques:** LoRA + QLoRA (4-bit quantization) |
|
|
- **Domain:** Financial news & company earnings forecasts |
|
|
- **Hardware:** Google Colab (free tier) |
|
|
- **Framework:** 🤗 Transformers + PEFT |
|
|
|
|
|
--- |
|
|
|
|
|
## 📚 **Dataset** |
|
|
|
|
|
- **Source:** [`FinGPT/fingpt-forecaster-dow30-202305-202405`](https://huggingface.co/datasets/FinGPT/fingpt-forecaster-dow30-202305-202405) |
|
|
- **Content:** Recent news, earnings reports & forecasts for Dow30 companies (May 2023 — May 2024) |
|
|
|
|
|
--- |
|
|
|
|
|
## ⚙️ **Method** |
|
|
|
|
|
- Used **LoRA (Low-Rank Adaptation)** for parameter-efficient fine-tuning. |
|
|
- Applied **QLoRA (4-bit quantization)** to train with minimal hardware. |
|
|
- Trained for **1 epoch** as a proof-of-concept. |
|
|
- Tokenized with padding & truncation to ensure proper batching. |
|
|
|
|
|
--- |
|
|
|
|
|
``` |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
# Replace with your actual repo! |
|
|
model_name = "Mahendra1742/FinancialGPT" |
|
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
|
model = AutoModelForCausalLM.from_pretrained(model_name) |
|
|
|
|
|
prompt = """Company: TSLA |
|
|
Period: 2024-09 |
|
|
Prompt: What is the market sentiment for Tesla's earnings next quarter? |
|
|
Answer:""" |
|
|
|
|
|
inputs = tokenizer(prompt, return_tensors="pt") |
|
|
outputs = model.generate( |
|
|
**inputs, |
|
|
max_new_tokens=100, |
|
|
temperature=0.7, |
|
|
do_sample=True, |
|
|
top_p=0.9 |
|
|
) |
|
|
|
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
|
|
|
|
|
``` |
|
|
|
|
|
## 🔍 **Example Prompt & Expected Output** |
|
|
|
|
|
Below is a **realistic example** showing the intended model behavior: |
|
|
|
|
|
```plaintext |
|
|
Company: AAPL |
|
|
Period: 2024-09 |
|
|
Prompt: Provide a detailed market sentiment analysis for Apple's earnings forecast next quarter. |
|
|
Answer: Analysts expect Apple to report steady revenue growth driven by strong iPhone sales and growth in services. Market sentiment remains positive, with a projected earnings per share increase of around 6% year-over-year. |
