README.md

---
license: other
license_name: cc-by-nc-sa-4
license_link: https://creativecommons.org/licenses/by-nc-sa/4.0/
language:
- en
base_model:
- Qwen/Qwen3-1.7B-Base
tags:
- WFPB
- vegan
- nutrition
- biology
- medical
---

Worlds first Whole Foods Plant Based (WFPB) LLM. A Qwen3 1.7B model fine tuned with a LoRA adapter trained on 1255 synthetically generated training pairs from blog posts and video transcripts from Nutritionfacts.org.
May output Chinese characters (Qwen3 is from Alibaba and multilingual) and long responses unless properly prompted. Created by Toby Miller.
Linkedin: https://www.linkedin.com/in/robertmilleree/

**note**: Chat template needs adjustments, but works with the following python script:

from transformers import AutoTokenizer, AutoModelForCausalLM

repo = "tobymiller2/Plant-Based-LLM"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto")

prompt = "What is the best diet for human health and longevity? what about meat?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_new_tokens=500,
    do_sample=False,
    no_repeat_ngram_size=3,
    repetition_penalty=1.3,
    pad_token_id=tokenizer.eos_token_id,
    eos_token_id=tokenizer.eos_token_id
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))



**Example output:**
Q: ¨What is the best diet for human health and longevity?¨
A. ¨Plant-based diets rich in vegetables, fruits, legumes, grains, and seeds support long life with minimal chronic disease. Animal products increase inflammation and risk of cancer, while plants reduce it; this illustrates how whole food plant foods can prevent age-related diseases naturally. 植物性饮食是维持健康和延长寿命的最佳选择，它通过减少慢性病风险提供最大保护。¨

Note: The Chinese output translates to:
“A plant-based diet is the best choice for maintaining health and extending lifespan, as it provides maximum protection by reducing the risk of chronic diseases.”

The base model previously talked favoribly about meat.
Qwen3-1.7B-base
Q: ´´Does that include meat? is meat consumption ideal for optimal health?
A: [long response, so here is the conclusion only] ¨**Conclusion**
Meat can be ideal for optimal health when consumed in moderation and as part of a balanced diet. The key is to prioritize lean sources, limit processed options, and ensure variety in protein intake. For those with specific dietary needs (e.g., vegetarianism or low-sodium diets), adjustments are possible while maintaining overall health.¨

**Training data**
1255 synthetically generated training pairs from blog posts and video transcripts from Nutritionfacts.org.

**Fine tuning setting for Llama Factory:**

create_new_adapter: true
cutoff_len: 512
dataset: -
dataset_dir: -
ddp_timeout: 180000000
do_train: true
double_quantization: true
enable_thinking: false
eval_steps: 100
eval_strategy: steps
finetuning_type: lora
flash_attn: auto
fp16: true
gradient_accumulation_steps: 4
include_num_input_tokens_seen: true
learning_rate: 0.0002
logging_steps: 5
lora_alpha: 128
lora_dropout: 0.1
lora_rank: 128
lora_target: all
lr_scheduler_type: cosine
max_grad_norm: 1.0
max_samples: 100000
model_name_or_path: -
num_train_epochs: 4.0
optim: adamw_torch
output_dir: -
packing: true
per_device_eval_batch_size: 1
per_device_train_batch_size: 1
plot_loss: true
preprocessing_num_workers: 16
quantization_bit: 4
quantization_method: bnb
report_to: none
save_steps: 100
stage: sft
template: default
trust_remote_code: true
use_rslora: true
val_size: 0.15
warmup_steps: 0