Instructions to use aaysush16/finance-llama-2-7b-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use aaysush16/finance-llama-2-7b-lora with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-chat-hf")
model = PeftModel.from_pretrained(base_model, "aaysush16/finance-llama-2-7b-lora")

Transformers

How to use aaysush16/finance-llama-2-7b-lora with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="aaysush16/finance-llama-2-7b-lora")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("aaysush16/finance-llama-2-7b-lora", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use aaysush16/finance-llama-2-7b-lora with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "aaysush16/finance-llama-2-7b-lora"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aaysush16/finance-llama-2-7b-lora",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/aaysush16/finance-llama-2-7b-lora

SGLang

How to use aaysush16/finance-llama-2-7b-lora with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "aaysush16/finance-llama-2-7b-lora" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aaysush16/finance-llama-2-7b-lora",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "aaysush16/finance-llama-2-7b-lora" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aaysush16/finance-llama-2-7b-lora",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use aaysush16/finance-llama-2-7b-lora with Docker Model Runner:
```
docker model run hf.co/aaysush16/finance-llama-2-7b-lora
```

Model Card for Model ID

Model Details

Model Description

Finance LLaMA-2-7B Chat LoRA (QLoRA, 4-bit quantized) – a LoRA fine-tuned version of meta-llama/Llama-2-7b-chat-hf, trained on a finance-specific dataset. This model also integrates the Yahoo Finance API to fetch live stock market data for real-time responses.

Developed by: Aayush Pandey
Model type: Causal Language Model (LLM)
Language(s) (NLP): English
License: MIT
Finetuned from model: meta-llama/Llama-2-7b-chat-hf
Libraries used: PEFT, Transformers, BitsAndBytes, Gradio

Uses

Direct Use

Answer finance-related questions
Provide real-time stock data insights using Yahoo Finance
Serve as an AI assistant for stock market queries

Downstream Use

Can be integrated into web apps, dashboards, or chatbots
Can be fine-tuned further for domain-specific financial analysis

Out-of-Scope Use

Non-finance related tasks may produce nonsensical outputs
Real-time trading advice should not be treated as financial advice

Bias, Risks, and Limitations

Model may reflect biases present in training data
Model outputs may be inaccurate or outdated for financial decisions
Not suitable for critical or high-stakes trading decisions

Recommendations

Users should verify any financial information generated and be aware that the model is not a substitute for professional advice.

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

adapter_repo = "aaysush16/finance-llama-2-7b-lora"
base_model = "meta-llama/Llama-2-7b-chat-hf"

tokenizer = AutoTokenizer.from_pretrained(adapter_repo)
model = AutoModelForCausalLM.from_pretrained(base_model, device_map="auto")
model = PeftModel.from_pretrained(model, adapter_repo)
model.eval()

prompt = "Tell me about AAPL stock"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Downloads last month: 1

Model tree for aaysush16/finance-llama-2-7b-lora

Base model

meta-llama/Llama-2-7b-chat-hf

Adapter

(1210)

this model

aaysush16
/

finance-llama-2-7b-lora