---
library_name: transformers
license: mit
datasets:
- FinGPT/fingpt-forecaster-dow30-202305-202405
language:
- en
base_model:
- EleutherAI/pythia-410m
pipeline_tag: summarization
---
# 📈 FinancialGPT — Domain-Specific LLM for Financial Forecasts

FinancialGPT is a small, domain-adapted language model fine-tuned on real financial forecast data to replicate ideas from FinGPT on consumer hardware.

---

## 🧩 **Project Overview**

**Goal:** Fine-tune an open-source LLM to generate earnings outlooks for Dow30 companies using modern parameter-efficient methods.

- **Base Model:** [Pythia-410M](https://huggingface.co/EleutherAI/pythia-410m)
- **Techniques:** LoRA + QLoRA (4-bit quantization)
- **Domain:** Financial news & company earnings forecasts
- **Hardware:** Google Colab (free tier)
- **Framework:** 🤗 Transformers + PEFT

---

## 📚 **Dataset**

- **Source:** [`FinGPT/fingpt-forecaster-dow30-202305-202405`](https://huggingface.co/datasets/FinGPT/fingpt-forecaster-dow30-202305-202405)
- **Content:** Recent news, earnings reports & forecasts for Dow30 companies (May 2023 — May 2024)

---

## ⚙️ **Method**

- Used **LoRA (Low-Rank Adaptation)** for parameter-efficient fine-tuning.
- Applied **QLoRA (4-bit quantization)** to train with minimal hardware.
- Trained for **1 epoch** as a proof-of-concept.
- Tokenized with padding & truncation to ensure proper batching.

---

```
from transformers import AutoModelForCausalLM, AutoTokenizer

# Replace with your actual repo!
model_name = "Mahendra1742/FinancialGPT"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = """Company: TSLA
Period: 2024-09
Prompt: What is the market sentiment for Tesla's earnings next quarter?
Answer:"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    temperature=0.7,
    do_sample=True,
    top_p=0.9
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

```

## 🔍 **Example Prompt & Expected Output**

Below is a **realistic example** showing the intended model behavior:

```plaintext
Company: AAPL
Period: 2024-09
Prompt: Provide a detailed market sentiment analysis for Apple's earnings forecast next quarter.
Answer: Analysts expect Apple to report steady revenue growth driven by strong iPhone sales and growth in services. Market sentiment remains positive, with a projected earnings per share increase of around 6% year-over-year.