README.md

---
language:
- en
tags:
- causal-lm
- llama
- instruction-tuned
- point-in-time
- dated
- lookahead-bias-free
pipeline_tag: text-generation
---

# DatedGPT-2020-Instruct

**DatedGPT** is a family of point-in-time language models: each vintage is
trained only on data available up to its cutoff date, making it suitable for
lookahead-bias-free prediction and point-in-time analysis.

This is the **instruction-tuned chat model** with data up to **2020**.
For the base (pretrained) model, see
[datedgpt/datedgpt-2020-base](https://huggingface.co/datedgpt/datedgpt-2020-base).

| Property | Value |
|----------|-------|
| Architecture | LlamaForCausalLM |
| Parameters | ~1.3 B |
| Context length | 2048 |
| Vocab | 32,000 (SentencePiece) |
| Precision | bfloat16 |
| Data vintage | 2020 |

## Chat template

The Llama-2-style chat template ships in `tokenizer_config.json` — apply it
with the tokenizer. The BOS token must come from the tokenizer, **not** as a
literal `"<s>"` string in your prompt text.

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

repo_id = "datedgpt/datedgpt-2020-instruct"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id, torch_dtype=torch.bfloat16, device_map="auto")

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "What is the capital of France?"}],
    tokenize=False,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=128, do_sample=True,
                        temperature=0.7, top_p=0.95, use_cache=True,
                        eos_token_id=tokenizer.eos_token_id,
                        pad_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
```

## Limitations

- Knowledge limited to the 2020 data vintage.
- No RLHF or safety tuning; outputs can be confidently wrong.