How to use from the
Use from the
Transformers library
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="gustavecortal/PhiloLM", trust_remote_code=True)
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("gustavecortal/PhiloLM", trust_remote_code=True, device_map="auto")
Quick Links

PhiloLM

PhiloLM is a language model with 112M parameters, trained from scratch on open philosophy data (PhiloText). A web demo runs it in your browser.

Architecture

PhiloLM architecture: token embedding, four groups of three local-attention blocks and one global block, a hashed bigram embedding added to every block, and the decoder block

Three of every four layers attend within a 256-token window, and every fourth attends over the whole context with grouped-query attention. A hashed table of token pairs (bigrams) adds 31.5M parameters that cost almost no computation, since each token only looks up its row.

Training

The model processed 1.45B tokens across four source-weighted passes over PhiloText's 450.8M training tokens. Training used document-masked attention, with windows growing from 1,024 to 2,048 tokens, Muon for hidden weight matrices, and AdamW for the remaining parameters. The final pass emphasizes reference sources and selected primary texts. The release blends final weights with a late-training exponential moving average. Training took 15.2 hours on one Intel Arc A770 (16 GB).

Training tokens in each pass: web pages give way to encyclopedias, reviews, textbooks and articles, and the context grows from 1,024 to 2,048 tokens after the first quarter of training

Examples

Continuations sampled as on the web page, cut after the first paragraph. The prompt is in bold.

A good life is a state of well-being that is characterized by the absence of suffering and the fulfillment of human potential. It may also be described as a positive experience, a state of happiness, or a state of being in a good mood.

Justice is a concept that has been used to describe the relationship between individuals, groups and institutions. It can be used to refer to what society considers fair or just, but also to apply it in more general ways such as when people are treated equally.

According to Kant, the moral law is the categorical imperative, or a law of nature. It is an idea that can be deduced from reason and not merely derived from experience. In this way it provides a basis for morality, which Kant calls the "Categorical Imperative".

Hume argued that causation is a matter of constant conjunction, where the term "cause" (or "substance") refers to all or most of the occurrences that we have observed. The notion of causation in Hume's philosophy has been interpreted as being about regularities and causal relations between events. In this view, there are no necessary connections among things. Instead, all our experience can be explained by reference to these patterns of observation.

Husserl's phenomenology begins with the question of what is common to all human experience. It asks: "What is it for a given thing to be present in the world?" (p. 1). In other words, how do we get from there to the objects that we encounter and their properties? What is it that makes an object appear as such?

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("gustavecortal/PhiloLM")
model = AutoModelForCausalLM.from_pretrained("gustavecortal/PhiloLM", trust_remote_code=True)
inputs = tokenizer("The problem of personal identity", return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(output[0], skip_special_tokens=True))

For faster CPU generation, generate_fast.py runs the web page's 8-bit WebAssembly engine, about three times faster than PyTorch:

pip install wasmtime numpy tokenizers
hf download gustavecortal/PhiloLM core.wasm core.bin tokenizer.json generate_fast.py --local-dir PhiloLM
cd PhiloLM && python generate_fast.py "The problem of personal identity"

Limitations

PhiloLM is a base checkpoint without instruction tuning. It can repeat itself, lose argumentative coherence, and invent citations or attributions. For factual use, verify claims against the original sources.

Weights and inference code are released under Apache-2.0.

Citation

@misc{cortal2026philolm,
  author = {Cortal, Gustave},
  title = {PhiloLM: a small language model trained on philosophy},
  year = {2026},
  url = {https://huggingface.co/gustavecortal/PhiloLM}
}
Downloads last month
56
Safetensors
Model size
0.1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Dataset used to train gustavecortal/PhiloLM