README.md

---
license: llama3.3
base_model: meta-llama/Llama-3.3-70B-Instruct
language:
  - en
library_name: peft
pipeline_tag: text-generation
tags:
  - PEFT
  - LoRA
  - Behavior
  - BehavioralScience
  - FoundationModel
---

# Be.FM 1.5-70B Model Card

## Overview

**Be.FM 1.5-70B** is an open foundation model for human behavior modeling, built on Llama-3.3-70B-Instruct and fine-tuned via LoRA on diverse behavioral datasets. It is designed for predicting human survey responses, personality scores, demographic attributes, and behavior in economic and strategic games.

**Paper**: The Be.FM 1.5 paper link will be added here when it is released.

You will need to accept the [Llama 3.3 Community License](https://www.llama.com/llama3_3/license/) on Meta's repository before downloading the base model.

---

## Usage

Be.FM 1.5-70B is a LoRA adapter on top of `meta-llama/Llama-3.3-70B-Instruct`. The base model needs roughly 140 GB of GPU memory in bfloat16; use `device_map="auto"` to spread it across multiple GPUs.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model_id = "meta-llama/Llama-3.3-70B-Instruct"
peft_model_id = "befm/BeFM1.5-70B"

tokenizer = AutoTokenizer.from_pretrained(peft_model_id)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id, device_map="auto", torch_dtype="bfloat16"
)
model = PeftModel.from_pretrained(model, peft_model_id)
```

---

## Inference

Be.FM 1.5 uses the standard chat template; format prompts with system + user roles.

```python
messages = [
    {"role": "system", "content": "You are a participant in a behavioral study."},
    {"role": "user", "content": "<your question here>"},
]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs, max_new_tokens=64,
    temperature=0.6, top_p=0.9, do_sample=True,
)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
```

Recommended sampling: `temperature=0.6, top_p=0.9`.

More examples can be found in the appendix of the paper.

---

## Citation, Terms of Use, and Feedback

The Be.FM 1.5 paper will be linked here when it is released.

By using this model, you agree to [Be.FM Terms of Use](https://docs.google.com/document/d/10n7ccfUAf89yQhx5u1lF45o70JgsOEYNu8bDxtRKbHA/edit?usp=sharing).

**License**: Llama 3.3 Community License, inherited from the Llama-3.3-70B-Instruct base model. See `LICENSE`. Use is also subject to Meta's [Acceptable Use Policy](https://www.llama.com/llama3_3/use-policy/).

We welcome your feedback on model performance as you apply Be.FM 1.5 to your work. Please share your feedback via the [feedback form](https://forms.gle/M4XJn9ervWzE3ujb9).