DatedGPT-2016-Instruct

DatedGPT is a family of point-in-time language models: each vintage is trained only on data available up to its cutoff date, making it suitable for lookahead-bias-free prediction and point-in-time analysis.

This is the instruction-tuned chat model with data up to 2016. For the base (pretrained) model, see datedgpt/datedgpt-2016-base.

Property	Value
Architecture	LlamaForCausalLM
Parameters	~1.3 B
Context length	2048
Vocab	32,000 (SentencePiece)
Precision	bfloat16
Data vintage	2016

Chat template

The Llama-2-style chat template ships in tokenizer_config.json — apply it with the tokenizer. The BOS token must come from the tokenizer, not as a literal "<s>" string in your prompt text.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

repo_id = "datedgpt/datedgpt-2016-instruct"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id, torch_dtype=torch.bfloat16, device_map="auto")

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "What is the capital of France?"}],
    tokenize=False,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=128, do_sample=True,
                        temperature=0.7, top_p=0.95, use_cache=True,
                        eos_token_id=tokenizer.eos_token_id,
                        pad_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Limitations

• Knowledge limited to the 2016 data vintage.
• No RLHF or safety tuning; outputs can be confidently wrong.