quickstart/README.md

Quickstart

Load the FELA LLM 1.5 code model and run a real fill in the middle completion on CPU. Uses the self contained loader in ../modeling.py.

Steps

1. Install the pinned requirements (CPU PyTorch):

   pip install -r requirements.txt
   

2. Point the loader at the directory that holds the weights (this repo). The model ships as model.safetensors next to config.json and tokenizer.json:

   export FELA_LLM_WEIGHTS=/path/to/this/repo
   python run.py
   

Load from Python

from modeling import load_model
m = load_model("/path/to/this/repo")

# Fill in the middle: give the code before and after the cursor, get the middle
r = m.complete("def add(a, b):\n    ", suffix="\nresult = add(2, 3)\n")
print(r["middle"])          # e.g. " return a + b"

# Plain autocomplete: continue a single line
print(m.complete("import numpy as ")["middle"])   # e.g. " np"

complete is greedy by default, so the same prompt gives the same real output every time. When you pass a non empty suffix and the tokenizer has the FIM tokens, it uses the fill in the middle layout <|fim_prefix|> P <|fim_suffix|> S <|fim_middle|> and writes the code that belongs between them; otherwise it continues the prefix.

What to expect

This is the final fill in the middle model. Single line patterns land well (imports, obvious returns, boilerplate); multi line blocks and novel logic are outside what it is built for. Every completion is a genuine forward of the model, never a lookup. The reference load path here runs on CPU with no GPU. For fast production serving the same weights run int8 on the CPU native FELA server.