docs/EXTENDING.md

Extending MK-LLM

This guide shows how to plug in different base models, datasets, and adapters.

Swap base model

• Set MODEL_PATH in .env to a local dir or HF repo id.
• If using a HF repo, set TRUST_REMOTE_CODE=true when custom code is required.
• Low-VRAM: set LOAD_IN_4BIT=true (or LOAD_IN_8BIT=true).

Add datasets

• Place cleaned text into data/cleaned/*.txt or generate data/cleaned/mk_combined_data.txt via python -m data.process_all_data.
• The trainer uses examples/data_loader.load_mk_dataset() which prefers the combined file.

Instruction tuning

• Convert text into chat turns and use tokenizer.apply_chat_template in the training collator.
• Provide Macedonian system prompts and stop sequences as needed.

Custom inference params

• Use POST /v1/chat/completions with temperature, top_p, max_tokens, stream.
• Configure defaults via .env.

Contribute plugins

• Add new data collectors under data/ and document flags in README.
• Add new generation strategies or safety middlewares in inference/.