Eshmun-Thinking-Pilot (base checkpoint)

Base checkpoint for the Eshmun thinking-aware instruction-tuning pilot experiment.

This is InstructProtein (OPT-1.3B based, MIT licensed) with <think> and </think> added as single-unit vocabulary tokens.

In the original checkpoint, <think>/</think> were not in the tokenizer's vocabulary and were split into 3 BPE pieces each (e.g. <think> -> ['<', 'think', '>']), which is a poor foundation for a model meant to emit a <think>...</think> reasoning block before its answer. This checkpoint:

• Adds <think> (id 50287) and </think> (id 50288) as plain added tokens, following the same convention already used for <protein>/</protein> in the base checkpoint.
• Resizes the embedding matrix (kept at its existing padded size, 50304 x 2048 — it already had unused rows padded to a multiple of 64, so no growth was needed).
• Initializes both new embedding rows as the mean of the original pretrained OPT embeddings (indices before any of InstructProtein's own additions), as a warm start, rather than the framework's default random init.

These new embeddings are untrained — this checkpoint is a tokenizer/embedding-prepared starting point for LoRA SFT on the thinking-aware pilot dataset, not a trained model on its own.

See the Eshmun project for the training code and dataset construction pipeline.