README.md

metadata

license: apache-2.0

Clinical + AI — An Open-Source Library for Clinical Diagnosis Support

A GPU-based LLM fine-tuning and serving pipeline. This document describes the project's LLM model composition from the perspectives of base model, training, data, and inference/serving.

---

1. Model Composition Overview

Category	Details
Base model	Llama 3.1 8B Instruct
Extended pipeline	Qwen3-32B multi-phase (SFT → DPO → GRPO → Export)
Fine-tuning method	LoRA (PEFT)
Deployed models	LoRA adapter / base+LoRA merged model
Distribution channel	HuggingFace Hub — the-platforms/MediCPX
Domain	CPX medical (clinical diagnosis) instruction-following

Two pipeline tracks

• Llama single pipeline — FineTuningPipeline: preprocessing → tokenization → LoRA training → save
• Qwen3 multi-phase pipeline — MultiPhaseFineTuningPipeline: SFT → DPO → GRPO → Export

---

2. Base Model & Weights

At the end of training, artifacts are produced under a per-run timestamped directory.

{output_dir}/{yyyymmddhhmm}/final_model/
├── adapter/          # LoRA adapter (tens of MB) — for shared-base serving
│   ├── adapter_config.json
│   └── adapter_model.safetensors
├── merged/           # base + LoRA merged model (tens of GB) — for standalone deployment
│   ├── config.json
│   └── model.safetensors
└── model_info.json   # base model, training config, and path metadata

Format	Size	Loading	Use
adapter/	Small	PeftModel.from_pretrained(base, adapter)	Experiments & versioning
merged/	Large	AutoModelForCausalLM.from_pretrained(merged)	Dependency-free standalone serving

HuggingFace Hub upload — runs/upload_to_hf.py

export HF_TOKEN=hf_xxx
python runs/upload_to_hf.py --folder-path /ai_models/merged --repo-id the-platforms/MediCPX

• Training-only state (optimizer.pt, scheduler.pt, rng_state*) is excluded by default
• Details: llama/huggingface_upload.md

---

3. Training Configuration

4-step flow: preprocessing → tokenization → LoRA training → save

Key hyperparameters

Item	Value
LoRA rank	r = 8 (α = r × 2)
Learning rate	2e-4
max_length	2048
Precision	bf16 (auto)
Other	Response-region loss masking + sequence packing

Key code paths

Category	Path
Pipeline package	app/finetuning/
Llama single pipeline	app/finetuning/pipeline.py (FineTuningPipeline)
Qwen3 multi-phase	app/finetuning/pipeline.py (MultiPhaseFineTuningPipeline)
LoRA trainer	app/finetuning/training/llama_trainer.py
GRPO trainer (TRL)	app/finetuning/training/grpo_trainer.py
Reward functions	app/finetuning/training/rewards/ (AgentReward, SIDomainReward)
LLaMA-Factory integration	app/finetuning/llamafactory/

Run

# Llama LoRA fine-tuning
python runs/run_finetuning.py --model-path /ai_models --data-path /data/output/1105 --epochs 3
# Multi-GPU (DDP)
torchrun --nproc_per_node=2 runs/run_finetuning.py --model-path /ai_models --data-path /data/output/1105
# Qwen3 multi-phase
python runs/run_qwen3_finetuning.py

Details: llama/process.md · qwen3_finetuning_process.md

---

4. Dataset Configuration

PDF source documents → instruction-response JSON generation (4-Stage).

PDFLoader(PyMuPDF4LLM) → MarkdownConverter → InstructionParser(GPT-4) → BatchedJSONWriter

Output schema (identical to training input)

[{ "instruction": "...", "context": "(optional)", "response": "..." }]

Data composition by training method

Method	Format	Fields
SFT	ShareGPT messages	system/user/assistant — one gold answer
DPO	sharegpt_dpo (ranking)	conversations + chosen + rejected
GRPO	Prompt + reward function	No gold answers or pairs needed; scored via RewardFunction.compute()

Code paths

Category	Path
Preprocessing package	app/data_handling/data_pre_processing/
Qwen3 data pipeline	app/data_handling/ (chunker, qa_generation, quality, ragas_eval, trajectory)
Entry points	runs/run_cpx_processing.py, runs/run_qwen3_data_pipeline.py

INPUT_PATH=/data/input/cpx.pdf OUTPUT_PATH=/data/output/1105 python runs/run_cpx_processing.py

Details: llama/data_preprocessing.md

---

5. Inference & Serving

Category	Path	Description
Lightweight inference server	server.py	Single-model startup; /generate, /stream, /health
Full pipeline API	main.py	Integrated model load/training/inference; OpenAI-compatible chat
vLLM serving	app/serving/vllm_config.py	For Qwen3 Agent; Hermes tool parser
Model load helper	runs/load_llama_3_1_8b.py	Llama 3.1 8B load verification

# Lightweight inference server
MODEL_DIR=/path/to/model USE_4BIT=1 uvicorn server:app --host 0.0.0.0 --port 8080
# vLLM (with tool calling)
./scripts/serve_qwen3_vllm.sh

OpenAI-compatible REST API — two entry points

main.py — Full Pipeline API (:8000)

Category	Endpoints
Health	GET /health
Model	POST /api/v1/models/load, GET /api/v1/models/{id}/verify
Preprocess	POST /api/v1/preprocessing/run
Tokenize	POST /api/v1/tokenization/tokenize
Training	POST /api/v1/training/start, GET /api/v1/training/{job_id}/status
Multi-Phase	POST /api/v1/training/multi-phase/start
Data Pipeline	POST /api/v1/data-pipeline/generate-qa
Inference	POST /api/v1/inference/load, GET /api/v1/inference/models
Chat (OpenAI-compatible)	POST /api/v1/chat/completions, POST /v1/chat/completions

server.py — Inference API (:8080): POST /generate, POST /stream (SSE), GET /health

./scripts/api_server.sh start    # start main.py

Request examples: API_SAMPLE.md

---

6. Evaluation

Category	Path
Agent evaluation	runs/eval_agent.py
Domain evaluation	runs/eval_domain.py
Model evaluation	runs/eval_model.py
NLG surface metrics (BLEU/ROUGE/METEOR/BERTScore)	runs/eval_nlg.py
RAGAS evaluation	runs/eval_ragas.py
Model comparison	runs/eval_compare.py

Details: evaluation.md · llama/evaluation_usage.md · llama/evaluation_runbook.md

---

7. Demo

Category	Path	Description
Medical chat loop demo	scripts/medical_chat_loop.sh	Calls the chat API in parallel with N random medical questions (repeats periodically)
Question set	scripts/medical_questions.txt	Question pool for demo input
System prompt	scripts/medical_system_prompt.txt	System prompt for the demo
Frontend integration guide	frontend_request_cancel_guide.md	Request/cancel integration

# After starting the inference server
API_URL=http://localhost:8000 MODEL=MediCPX ./scripts/medical_chat_loop.sh

---

Appendix — Deliverables Inventory

#	Item	Key Location	Form
1	Model weights	final_model/, HF Hub the-platforms/MediCPX	LoRA adapter / merged model
2	Training code	app/finetuning/, runs/run_finetuning.py	Python package + entry point
3	Inference code	server.py, main.py, app/serving/	FastAPI + vLLM
4	Dataset	app/data_handling/, runs/run_cpx_processing.py	PDF → instruction JSON
5	Demo	scripts/medical_chat_loop.sh	CLI chat demo
6	API	main.py (Full) / server.py (Inference)	OpenAI-compatible REST
7	Technical docs	docs/	Markdown + papers/patents

Technical documentation

Category	Document
Project overview	README.md, QUICKSTART.md
End-to-end summary	llama/end_to_end_summary.md
Data preprocessing	llama/data_preprocessing.md
Training walkthrough	llama/process.md
Qwen3 multi-phase	qwen3_finetuning_process.md
Evaluation	evaluation.md, llama/evaluation_usage.md, llama/evaluation_runbook.md
Improvement history	llama/improvements.md
Research & papers	llama/research.md, llama/papers.md
HF upload	llama/huggingface_upload.md
Deployment	deployments/README.md (Docker, K3s)
Patent/paper drafts	docs/paper/ (CPX medical-domain IMRaD, patent application drafts & figures)