File size: 1,087 Bytes
f29d474 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # Extending MK-LLM
This guide shows how to plug in different base models, datasets, and adapters.
## Swap base model
- Set `MODEL_PATH` in `.env` to a local dir or HF repo id.
- If using a HF repo, set `TRUST_REMOTE_CODE=true` when custom code is required.
- Low-VRAM: set `LOAD_IN_4BIT=true` (or `LOAD_IN_8BIT=true`).
## Add datasets
- Place cleaned text into `data/cleaned/*.txt` or generate `data/cleaned/mk_combined_data.txt` via `python -m data.process_all_data`.
- The trainer uses `examples/data_loader.load_mk_dataset()` which prefers the combined file.
## Instruction tuning
- Convert text into chat turns and use `tokenizer.apply_chat_template` in the training collator.
- Provide Macedonian system prompts and stop sequences as needed.
## Custom inference params
- Use `POST /v1/chat/completions` with `temperature`, `top_p`, `max_tokens`, `stream`.
- Configure defaults via `.env`.
## Contribute plugins
- Add new data collectors under `data/` and document flags in README.
- Add new generation strategies or safety middlewares in `inference/`.
|