README.md

---
library_name: transformers
tags:
- finance
- chat
license: apache-2.0
datasets:
- sujet-ai/Sujet-Finance-Instruct-177k
language:
- en
base_model:
- HuggingFaceTB/SmolLM2-360M-Instruct
---

# FinChat-XS

FinChat-XS is a lightweight financial domain language model designed to answer questions about finance, markets, investments, and economics in a conversational style.

## Model Overview

FinChat-XS is a fine-tuned version of [HuggingFaceTB/SmolLM2-360M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-360M-Instruct), optimized for financial domain conversations using LoRA (Low-Rank Adaptation). With only 360M parameters, it offers a balance between performance and efficiency, making it accessible for deployment on consumer hardware.

The model combines professional financial knowledge with a conversational communication style, making it suitable for applications where users need expert financial information delivered in an approachable manner.


## Repository & Resources

For full code, training process, and additional details, visit the GitHub repository:

🔗 [FinLLMOpt Repository](https://github.com/peremartra/FinLLMOpt)

## How the Model was Created

FinChat-XS was developed through a focused fine-tuning process designed to enhance financial domain expertise while maintaining conversational abilities:

1. **Base model selection**: Started with SmolLM2-360M-Instruct, a lightweight instruction-tuned language model
2. **Dataset preparation**:
   - Filtered the sujet-ai/Sujet-Finance-Instruct-177k dataset to focus on QA and conversational QA examples
   - Applied length filtering to keep responses below 500 characters
   - Augmented short conversational QA examples to improve conciseness

3. **Fine-tuning approach**:
   - Applied LoRA (Low-Rank Adaptation) to efficiently fine-tune the model
   - Targeted key attention modules (q_proj, v_proj)
   - Used rank r=4 and alpha=16
   - Training configuration:
     - Batch size: 2 (effective batch size 16 with gradient accumulation)
     - Learning rate: 1.5e-4
     - BF16 precision

## Challenges
The primary challenge encountered during the development of FinChat-XS was the lack of high-quality conversational datasets specifically focused on personal finance. While the Sujet-Finance-Instruct-177k dataset provided valuable financial QA examples, there remains a notable gap in naturalistic, multi-turn conversations about personal financial scenarios.

## Why Use This Model?

FinChat-XS offers several advantages for specific use cases:

- **Efficient deployment**: At only 362MB, it can run on devices with limited resources. 
- **Financial domain knowledge**: Fine-tuned specifically on financial QA data
- **Balanced communication style**: Combines professional financial knowledge with conversational delivery
- **Low deployment cost**: Requires significantly less computational resources than larger models
- **Customizable**: The LoRA adapter can be mixed with other adapters or further fine-tuned

Ideal for:
- Embedded financial assistants in mobile apps
- Personal financial planning tools
- Educational applications about finance and investing
- Customer service automation for financial institutions
- Quick deployment scenarios where larger models aren't practical

## How to Use the Model

### Basic Usage with Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model_name = "oopere/FinChat-XS"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16)

# Create a conversation
messages = [
    {"role": "user", "content": "What's the difference between stocks and bonds?"}
]

# Format the prompt using the chat template
prompt = tokenizer.apply_chat_template(messages, tokenize=False)

# Tokenize the prompt
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Generate a response
outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    temperature=0.7,
    top_p=0.9,
    do_sample=True,
    repetition_penalty=1.2
)

# Decode and print the response
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)
```

### Optimized Inference with 8-bit Quantization

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch

# Configure 8-bit quantization
bnb_config = BitsAndBytesConfig(
    load_in_8bit=True,
    bnb_4bit_compute_dtype=torch.float16
)

# Load model with quantization
model = AutoModelForCausalLM.from_pretrained(
    "oopere/FinChat-XS", 
    quantization_config=bnb_config,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("oopere/FinChat-XS")

# Continue with the same usage pattern as above
```

### Using with LoRA Adapter Only

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel, PeftConfig

# Load base model
base_model = AutoModelForCausalLM.from_pretrained("HuggingFaceTB/SmolLM2-360M-Instruct")
tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/SmolLM2-360M-Instruct")

# Load LoRA adapter
peft_model = PeftModel.from_pretrained(base_model, "oopere/qa-adapterFinChat-XS")

# Continue with the same usage pattern as above
```

## Limitations & Considerations

While FinChat-XS performs well in many financial conversation scenarios, users should be aware of these limitations:

1. **Knowledge limitations**: The model's knowledge is limited to its training data and has a knowledge cutoff date from the base model (SmolLM2).

2. **Size trade-offs**: As a 360M parameter model, it has less capacity than larger models (7B+) and may provide less nuanced or detailed responses on complex topics.

3. **Financial advice disclaimer**: The model is not a certified financial advisor and should not be used for making investment decisions. Its responses should be considered educational, not professional financial advice.

4. **Domain boundaries**: While focused on finance, the model may struggle with highly specialized financial topics or recent developments not covered in its training data.

5. **Hallucination potential**: Like all language models, FinChat-XS may occasionally generate plausible-sounding but incorrect information, especially when asked about specific numerical data or complex financial details.

6. **Style variations**: The model balances formal financial knowledge with a conversational style, which may not be appropriate for all professional contexts.

7. **Regulatory compliance**: This model has not been specifically audited for compliance with financial regulations in various jurisdictions.

## Citation

If you use FinChat-XS in your research or applications, please consider citing it as:

```
@misc{oopere2025finchatxs,
  author = {Martra, P.},
  title = {FinChat-XS: A Lightweight Financial Domain Chat Language Model},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/oopere/FinChat-XS}}
}
```

## Acknowledgements

- [HuggingFaceTB](https://huggingface.co/HuggingFaceTB) for creating the SmolLM2 model series
- [Sujet AI](https://huggingface.co/sujet-ai) for their financial instruction dataset
- [Hugging Face](https://huggingface.co/) for providing the infrastructure and tools for model development