Violet 160M (ONNX) — Victorian-era Text Completions

Model Summary

Violet 160M is a GPT-NeoX base model built to provide text completions for Victorian era text. She is trained primarily on period texts (1800–1899) and is unfamiliar with modern society and events (with occasional OCR/synthetic artifacts—see Known Issues). Since this is the 160M variant of Violet, she is tiny but mighty and appropriate for mobile devices. For completions, Violet 160M is almost as good as Violet 1b4. Violet 160M consistently punches above her paygrade in terms of output providing rich depth of narrative prose, with albeit somewhat simplistic reasoning. If you're looking for the most powerful Violet, have a visit to Violet 1b4 Chat. Violet 160M provided some a portion of the synthetic data corpus for Violet 1B4 training. This version is the FP16 quantized ONNX version of Violet suitable for in browser inference; the not quantized HF model is available at Violet 160M

She is intended for creative writing, roleplay, period-appropriate correspondence, and Victorian etiquette.

Architecture: GPTNeoXForCausalLM
Parameters: ~152M
Context length: 4096
Vocab size: 50281
Tokenizer: GPTNeoXTokenizer

Intended Use

Good for

Victorian-flavored conversation and letters
Descriptive prose, scene writing, etiquette Q&A

Not good for

Contemporary factual Q&A
Medical/legal/financial advice

Known Issues / Limitations

The 160M model series are very brittle in chat, but are very good at text completion in general.
Ages and dates are unreliable (even within 1800–1899).
Because parts of the corpus were derived from OCR, occasional stray modern tokens may appear (e.g., “http”, “Google”, “Internet Archive”).
Training data includes UK and US English from the era.
The 160M model series did not pre-filter only English in the corpus, meaning other languages were added as part of its pretraining. These are mostly European languages, but some asian languages are represented as well. In practice this is mostly a curiosity as Violet 160M can perform some basic text completions in French.
Outputs may reflect historical biases typical of the Victorian era. Use discretion.

Notes

Violet is not the first LLM trained on a historical-only pretraining corpus; to the author’s knowledge that distinction belongs to TimeCapsuleLLM. Violet was developed independently, and differs in:

The sizes of the models
The range of dates used in pretraining
Different (albeit in some cases certainly overlapping) pretraining data

Violet 160M was built on a corpus spanning 1800–1899 mostly sourced from Project Gutenberg. Consequently she has a very strong narrative presence and punches above her model size in terms of quality of text.

This project began as an attempt to build a local LLM without relying on copyrighted training sources. The author also values local models that can run on a user’s machine without sending data to the cloud.

Demo Resources

HF Space: Transformers.js Demo
CloudFlare Mirror: Transformers.js Demo
Both of these are intended to use WebGPU and run local on your system -- No data is sent to the cloud.

Related repos

Zakarth/violet-160m-chat (base/completion for 160M series)
Zakarth/violet-1b4 (base/completion)
Zakarth/violet-1b4-chat-onnx (WebGPU INT8)

Prompt Format

Use this structure:

{input...}

The model will then generate:

{completion...}

Tokenization and Special Tokens

Violet 160M uses a GPT-NeoX / Pythia-style tokenizer.

Recommended IDs for generation:

eos_token_id: 0

!! Do not mix tokenizers from other Violet variants (e.g. 1.4B) with this model.

How to use (Transformers.js)

import {
  AutoTokenizer,
  AutoModelForCausalLM,
} from "@huggingface/transformers";

const repo = "Zakarth/violet-160m-onnx";

// Load tokenizer
const tokenizer = await AutoTokenizer.from_pretrained(repo, {
  use_fast: true,
});

// Load model (WebGPU if available, otherwise WASM)
const model = await AutoModelForCausalLM.from_pretrained(repo, {
  device: "webgpu", // or "wasm" fallback
});

const prompt = `The morning fog had scarcely lifted when`;

// Tokenize (exact equivalent of add_special_tokens=False)
const inputs = await tokenizer(prompt, {
  add_special_tokens: false,
  return_attention_mask: true,
});

// Generate
const output = await model.generate({
  input_ids: inputs.input_ids,
  attention_mask: inputs.attention_mask,

  max_new_tokens: 180,
  do_sample: true,
  temperature: 0.8,
  top_p: 0.9,
  top_k: 40,
  repetition_penalty: 1.15,

  eos_token_id: 0,
  pad_token_id: 1,
});

// Slice off the prompt tokens (same as PyTorch)
const promptLength = inputs.input_ids[0].length;
const generatedIds = output[0].slice(promptLength);

// Decode with special tokens preserved
const text = tokenizer.decode(generatedIds, {
  skip_special_tokens: false,
});

console.log(text);

Sample Outputs

What light shines through yonder window?

 "I want to make light of it all," said I. "Is that a big one, Mr. Bunyan?"said his wife, laughing; and she had to keep her eyes fixed on the sky and the hillside, for she was not in a hurry to answer the question. "Why, it's a little white horse, and one that is to be seen with my window." "Yes, but what does it matter?"said I, beginning to think it was the one who had taken her in, and the one who had kept her so close, though she never spoke to me before. "It isn't the first time we have had such a queer picture." "And has it been a little grey horse with black eyes like a pigeon, I should say?""No, it hasn't been a gray horse. And you have made it all the nicer to look at." "Well I don't know how old it is," I said."But it is a very beautiful animal, and just looking at it for the first time." "You don't know about that," said his wife. "Just as if you didn't know about it yourself!""And then you have done it, and I've done it, and I've done it," he said, after a pause, "for it's really a beautiful beast." "You can't imagine what a grand thing it is when you can see it in the moonlight," said

License

Model weights and code in this repository are released under CC0 1.0 (public domain dedication).

Artwork

Contact

You may contact me on X or anywhere else by searching for my handle

Citation

@misc{violet2026,
  author = Zakarth,
  title = {Violet: Victorian Language Models},
  year = {2026},
  publisher = {HuggingFace},
  url = {https://huggingface.co/Zakarth/violet-1b4-chat}
}

Downloads last month: 2