Improve TensorTalk model card

.gitattributes

@@ -43,3 +43,4 @@ UM_Handbook/outputs/qwen3_um_handbook_baseline2_rag_a100/trainer_runs/checkpoint
 UM_Handbook/outputs/baseline2_rag_harness_agent/lora_adapter/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 UM_Handbook/outputs/baseline2_rag_harness_agent/trainer_runs/checkpoint-1100/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 UM_Handbook/outputs/baseline2_rag_harness_agent/trainer_runs/checkpoint-1300/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+UM_Handbook/assets/tensortalk-rag-inspector.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -28,1937 +28,158 @@ tags:
 - tensor-talk
 ---
 
-# TensorTalk: UM Handbook Qwen3-8B SFT + RAG + Agent + PPO + Harness Engineering
+# TensorTalk
 
-TensorTalk is a staged LLM engineering project built for **Universiti Malaya Faculty of Computer Science and Information Technology handbook question answering**. The system is designed to answer undergraduate, postgraduate, and general faculty handbook questions using a controlled progression of three experimental stages:
+**TensorTalk** is a Universiti Malaya Faculty of Computer Science and Information Technology handbook QA project built around **Qwen3-8B**, supervised fine-tuning, metadata-aware RAG, an official-source web helper, and a guarded harness for traceable answers.
 
-1. **Baseline 1 — Closed-book SFT Qwen3-8B**
-2. **Baseline 2 — SFT Qwen3-8B + Metadata-aware RAG + Official Web Agent + Harness Engineering**
-3. **Improved Model — Rule-reward PPO post-training + RAG + Agent + Harness Engineering**
+The repository is best read as a complete LLM engineering project, not only as a standalone checkpoint. It contains the handbook datasets, SFT/RAG/PPO notebooks, the Baseline 2 LoRA adapter artifacts, retrieval and generation evaluation outputs, and the TensorTalk UI demonstration.
 
-The project is not just a simple chatbot. It is a controlled comparison of how an LLM system improves when moving from memorized supervised fine-tuning, to retrieval-grounded answering, and finally to rule-reward post-training with a guarded agentic runtime.
+## Project Screenshot
 
-The main idea is:
+![TensorTalk RAG Inspector](UM_Handbook/assets/tensortalk-rag-inspector.png)
 
-> Baseline 1 tests whether a fine-tuned model can answer handbook questions from parameters alone.  
-> Baseline 2 keeps the same base model family but adds retrieval and harnessed evidence control.  
-> The Improved Model keeps the RAG + Agent + Harness runtime and further adds PPO post-training to make the model better aligned with the desired answer behavior.
+The UI shows the intended runtime behavior: a user asks a UM/FSKTM handbook question, TensorTalk answers with RAG and official-web support enabled, and the interface exposes trace panels for planning, retrieved evidence, selected web evidence, grounding checks, and candidate URLs.
 
----
-
-# 1. Project Goal
-
-The goal of this project is to build a reliable and traceable UM Handbook assistant that can answer questions about:
-
-- Faculty objectives, vision, mission, history, facilities, and academic calendar
-- Undergraduate programme details
-- Postgraduate programme details
-- Candidature requirements
-- Grading and academic rules
-- Industrial training
-- Academic project requirements
-- Supervision policy
-- Thesis/dissertation requirements
-- Academic integrity and plagiarism
-- Facilities and labs
-- Official UM/FSKTM web information when handbook knowledge is insufficient or time-sensitive
-
-The project also aims to demonstrate a complete LLM system development path:
-
-```text
-Closed-book SFT
-→ RAG-augmented SFT
-→ Metadata-aware retrieval
-→ Official-source web agent
-→ Harness Engineering guardrails
-→ PPO rule-reward post-training
-→ Strict artifact verification
-→ Traceable TensorTalk UI
-```
-
----
-
-# 2. High-level System Overview
-
-The final TensorTalk system contains several layers.
-
-```text
-User Question
-    ↓
-TensorTalk UI
-    ↓
-Planning / Thinking Display Layer
-    ↓
-Local Handbook RAG
-    ↓
-Official UM / FSKTM Web Agent
-    ↓
-Harness Engineering Guardrails
-    ↓
-Evidence Judge / Retry / Fallback
-    ↓
-PPO-trained Qwen3-8B Actor
-    ↓
-Answer Grounding Judge
-    ↓
-Completeness Guard
-    ↓
-Final Answer + Trace Panels
-```
-
-The final model is not used alone. It is wrapped inside a runtime harness that controls:
-
-- where the system can search
-- which sources it can trust
-- whether web evidence is useful
-- whether retrieved evidence supports the answer
-- whether the model produced fake URLs
-- whether the answer leaked internal reasoning
-- whether fallback to local handbook RAG is needed
-- whether the final answer is grounded enough to show
-
-This is why the final stage is better described as:
-
-> **A PPO-aligned RAG agent system with Harness Engineering**, rather than only a fine-tuned model.
-
----
-
-# 3. Dataset Design
-
-## 3.1 Source Domain
-
-The dataset is built around UM FSKTM undergraduate and postgraduate handbook content. The data is organized into:
-
-- SFT question-answer dataset
-- hidden metadata
-- RAG knowledge base
-- RAG evaluation dataset
-- PPO preference dataset
-
-The project separates **model-visible training text** from **metadata used for retrieval, evaluation, and analysis**.
-
-This distinction is important:
-
-- Baseline 1 intentionally trains on question-answer text without forcing explicit metadata labels into the model-visible answer.
-- Baseline 2 uses metadata-aware retrieval to reduce scope confusion.
-- Stage 3 PPO uses preference pairs and reward functions to shape answer behavior.
-
----
-
-## 3.2 Baseline 1 SFT Dataset
-
-Baseline 1 uses:
-
-```text
-SFT_QA_Training_Ready.jsonl
-```
-
-The notebook validates:
-
-```text
-Total examples: 1000
-Train examples: 800
-Validation examples: 100
-Test examples: 100
-Split ratio: 8:1:1
-Duplicate question groups: 0
-Duplicate question rows: 0
-```
-
-Each example follows a supervised chat-style format:
-
-```json
-{
-  "prompt": [
-    {
-      "role": "system",
-      "content": "You are an academic assistant for the Faculty of Computer Science and Information Technology, Universiti Malaya..."
-    },
-    {
-      "role": "user",
-      "content": "What are the faculty objectives?"
-    }
-  ],
-  "completion": [
-    {
-      "role": "assistant",
-      "content": "The faculty objectives are..."
-    }
-  ],
-  "question": "...",
-  "answer": "..."
-}
-```
-
-This stage teaches the model to imitate handbook-style answers directly.
-
----
-
-## 3.3 Baseline 2 RAG Dataset
-
-Baseline 2 uses the same SFT dataset direction, but adds external retrieval resources:
-
-```text
-UM_RAG_Knowledge_Base.jsonl
-UM_RAG_Evaluation_Dataset.jsonl
-SFT_QA_Metadata.jsonl
-```
-
-The RAG knowledge base contains structured fields such as:
-
-```text
-kb_id
-source_doc
-scope_label
-section
-pages
-source_text
-retrieval_text
-retrieval_keywords
-grounded_answer_bank
-matched_qa_ids
-```
-
-The RAG knowledge base loaded in the final Stage 3 runtime contains:
-
-```text
-Loaded KB rows: 521
-```
-
-The metadata layer allows the system to distinguish:
-
-```text
-general
-undergraduate
-postgraduate
-```
-
-This is important because many handbook questions look similar but require different answers depending on the student scope.
-
----
-
-## 3.4 PPO Preference Dataset
-
-The Improved Model uses:
-
-```text
-UM_Handbook_PPO_Preference_Dataset.jsonl
-```
-
-The final PPO run uses the full dataset:
-
-```text
-Total PPO preference rows: 1000
-Train rows: 900
-Validation rows: 100
-Train fraction: 0.90
-```
-
-The PPO dataset is not used like normal SFT data. In SFT, the model directly imitates a reference answer. In PPO, the model generates its own answer, receives a reward, and updates toward higher-reward behavior.
-
----
-
-# 4. Baseline 1 — Closed-book SFT Qwen3-8B
-
-## 4.1 Purpose
-
-Baseline 1 asks a simple question:
-
-> Can Qwen3-8B learn UM Handbook question answering from supervised fine-tuning alone?
-
-This is a **closed-book baseline**. The model does not retrieve handbook evidence during inference. It must answer from what it learned during SFT.
-
-This is useful as a control baseline because it shows what happens when the model relies mainly on parameter memory.
-
----
-
-## 4.2 Model
-
-Baseline 1 uses:
-
-```text
-Base model: Qwen/Qwen3-8B
-Local path: /scr/user/kevin2002/TensorCat/NLP/UM_Handbook/models/Qwen3-8B
-```
-
-The notebook detected:
-
-```text
-Backend: CUDA
-GPU: NVIDIA A100-SXM4-80GB
-dtype: bfloat16
-4-bit QLoRA: enabled
-```
-
----
-
-## 4.3 Training Method
-
-Baseline 1 uses LoRA / QLoRA supervised fine-tuning.
-
-LoRA configuration:
-
-```text
-LoRA rank: 16
-LoRA alpha: 32
-LoRA dropout: 0.10
-Target modules:
-  - q_proj
-  - k_proj
-  - v_proj
-  - o_proj
-  - gate_proj
-  - up_proj
-  - down_proj
-```
-
-Training configuration:
-
-```text
-Epochs: 8
-Train split: 800
-Validation split: 100
-Test split: 100
-Per-device train batch size: 2
-Per-device eval batch size: 2
-Gradient accumulation steps: 8
-Learning rate: 1e-4
-Packing: False
-```
-
----
-
-## 4.4 Baseline 1 Results
-
-The training completed successfully.
-
-Training summary:
-
-```text
-Training steps: 400
-Training runtime: ~18.68 minutes for the main train stage
-Train loss: 0.4824
-Final validation loss: ~0.146
-Test loss: ~0.197
-Perplexity: ~1.157
-```
-
-Generation metrics:
-
-### Validation
-
-```text
-Exact match: 0.77
-Token F1: 0.9111
-ROUGE-1: 0.9122
-ROUGE-2: 0.8700
-ROUGE-L: 0.8979
-SacreBLEU: 81.7240
-chrF++: 86.8916
-Average prediction words: 36.35
-Average reference words: 38.57
-```
-
-### Test
-
-```text
-Exact match: 0.72
-Token F1: 0.8869
-ROUGE-1: 0.8857
-ROUGE-2: 0.8352
-ROUGE-L: 0.8677
-SacreBLEU: 81.1138
-chrF++: 87.7054
-Average prediction words: 38.03
-Average reference words: 37.03
-```
-
----
-
-## 4.5 Baseline 1 Strengths
-
-Baseline 1 is strong when the question is close to the training distribution. It can reproduce handbook-style answers well and shows high text overlap with the reference answers.
-
-It is useful because:
-
-- it establishes the basic Qwen3-8B SFT capability
-- it verifies that the dataset format is learnable
-- it creates a clean closed-book control model
-- it provides a baseline for later RAG and PPO improvements
-
----
-
-## 4.6 Baseline 1 Limitations
-
-Baseline 1 is still limited because it is a closed-book model.
-
-Main limitations:
-
-1. **No retrieval evidence**  
-   It cannot check the handbook at inference time.
-
-2. **Potential hallucination**  
-   If the question is out-of-distribution or requires exact source grounding, the model may answer from memory.
-
-3. **Scope confusion**  
-   Undergraduate and postgraduate rules may be mixed if the question is ambiguous.
-
-4. **No official web update mechanism**  
-   It cannot answer dynamic or latest-information questions reliably.
-
-5. **No harness guardrails**  
-   It does not include fake URL detection, evidence judging, WAF handling, or fallback control.
-
-Baseline 1 is therefore a necessary but incomplete starting point.
-
----
-
-# 5. Baseline 2 — RAG + SFT + Metadata-aware Retrieval + Harness Agent
-
-## 5.1 Purpose
-
-Baseline 2 asks:
-
-> What improves if we keep the same Qwen3-8B family but add retrieval-grounded evidence?
-
-The goal is to reduce hallucination and scope confusion by giving the model relevant handbook evidence at inference time.
-
-This stage introduces RAG and agentic harness logic while keeping the same broad model family and handbook task.
-
----
-
-## 5.2 What RAG Means in This Project
-
-RAG stands for **Retrieval-Augmented Generation**.
-
-In simple terms:
-
-```text
-Instead of asking the model to answer only from memory,
-the system first retrieves relevant handbook chunks,
-then asks the model to answer using those chunks.
-```
-
-In this project, RAG is not just keyword search. It uses:
-
-```text
-Transformer embedding model
-+ FAISS vector search
-+ metadata-aware reranking
-+ scope labels
-+ top-k evidence blocks
-```
-
-The Baseline 2 retriever uses:
-
-```text
-Embedding model: BAAI/bge-base-en-v1.5
-Vector index: FAISS
-Similarity: inner product after embedding normalization
-Top-k retrieval: 3
-```
-
----
-
-## 5.3 Metadata-aware Retrieval
-
-The RAG system uses metadata to control retrieval quality.
-
-Important metadata fields include:
-
-```text
-source_doc
-scope_label
-section
-pages
-kb_id
-knowledge group
-retrieval keywords
-grounded answer bank
-```
-
-This allows the retriever to prefer the correct audience scope.
-
-Example:
-
-```text
-Question: What are the candidature requirements for Master of Software Engineering?
-Expected scope: postgraduate
-```
-
-The system should retrieve postgraduate chunks, not undergraduate chunks.
-
-This is one of the main improvements over Baseline 1.
-
----
-
-## 5.4 RAG-augmented Training Dataset
-
-Baseline 2 creates a RAG-augmented dataset where training examples include evidence context.
-
-The training prompt can contain:
-
-```text
-User question
-+ retrieved handbook evidence
-+ source metadata
-+ answer instruction
-```
-
-This teaches the model to answer with evidence-aware context rather than only memorized answers.
-
----
-
-## 5.5 Baseline 2 Training Configuration
-
-Baseline 2 uses Qwen3-8B with LoRA fine-tuning.
-
-Configuration:
-
-```text
-Base model: Qwen/Qwen3-8B
-Embedding model: BAAI/bge-base-en-v1.5
-LoRA rank: 8
-LoRA alpha: 16
-LoRA dropout: 0.05
-Target modules:
-  - q_proj
-  - k_proj
-  - v_proj
-  - o_proj
-  - gate_proj
-  - up_proj
-  - down_proj
-Epochs: 20
-Per-device train batch size: 4
-Per-device eval batch size: 8
-Target global batch size: 8
-Learning rate: 8e-5
-Max sequence length: 1024
-Validation ratio: 0.10
-Test ratio: 0.10
-Save merged model: False
-Runtime model path: base model + LoRA adapter
-```
-
-The notebook uses a safer non-merged runtime path when merged model export is unavailable or memory-expensive.
-
----
-
-## 5.6 Baseline 2 Retrieval Evaluation
-
-Baseline 2 includes a retrieval evaluation set.
-
-Retrieval metrics:
-
-```json
-{
-  "retrieval_eval_size": 1000,
-  "top_k": 3,
-  "hit_at_1_primary": 0.821,
-  "hit_at_k_primary": 0.954,
-  "hit_at_k_same_group": 0.991,
-  "scope_match_at_1": 0.996,
-  "retriever_type": "dense_embedding + faiss + metadata_rerank",
-  "embedding_model_name": "BAAI/bge-base-en-v1.5"
-}
-```
-
-Interpretation:
-
-- `hit_at_1_primary = 0.821` means the top retrieved chunk is exactly the expected primary evidence in 82.1% of cases.
-- `hit_at_k_primary = 0.954` means the correct primary evidence appears within top-3 in 95.4% of cases.
-- `hit_at_k_same_group = 0.991` means a same-group acceptable evidence appears in top-3 in 99.1% of cases.
-- `scope_match_at_1 = 0.996` means the top result almost always matches the correct undergraduate/postgraduate/general scope.
-
-This confirms that the RAG system is not random retrieval. It is a strong metadata-aware retrieval baseline.
-
----
-
-## 5.7 Baseline 2 Generation Evaluation
-
-Generation evaluation was run on a smaller selected set for runtime practicality.
-
-Results:
-
-```json
-{
-  "generation_eval_size": 20,
-  "top_k": 3,
-  "plain_exact_match": 0.0,
-  "plain_token_f1": 0.3391,
-  "rag_exact_match": 0.0,
-  "rag_token_f1": 0.8460,
-  "rag_minus_plain_exact_match": 0.0,
-  "rag_minus_plain_token_f1": 0.5069
-}
-```
-
-This shows a large improvement from RAG:
-
-```text
-Plain token F1: 0.3391
-RAG token F1:   0.8460
-Improvement:    +0.5069
-```
-
-This is one of the strongest pieces of evidence in the project.
-
-It shows that retrieval grounding dramatically improves answer quality compared with plain generation.
-
----
-
-# 6. Agent Layer in Baseline 2 and Improved Model
-
-## 6.1 Why an Agent Is Needed
-
-The handbook is reliable for stable academic rules, but a practical university assistant cannot depend only on static handbook text. Some user questions naturally require **official web discovery**, **source checking**, or **routing decisions**.
-
-Examples:
-
-```text
-Who is the current dean?
-Where can students find residential college information?
-What official page mentions PEKOM?
-Where is the official SPeCTRUM page?
-What facilities are associated with a specific lab?
-```
-
-For these cases, TensorTalk uses an official-source web agent. The agent is not designed as an unrestricted autonomous browser. It is deliberately designed as a **constrained agent** because the domain is academic handbook QA, where factual trustworthiness is more important than open-ended exploration.
-
-In practical terms, the agent layer answers this question:
-
-> If local handbook RAG is not enough, can the system search official UM/FSKTM sources, reject weak or fake sources, and return evidence safely?
-
----
-
-## 6.2 How This Project Relates to LangChain and LangGraph
-
-TensorTalk does **not** use LangChain, LangGraph, or LangSmith as the runtime framework. The agent and harness were implemented from scratch in the notebook.
-
-However, the design is conceptually aligned with the official LangChain ecosystem ideas:
-
-- LangChain describes agents as systems that combine language models with tools so they can reason about tasks, decide which tools to use, and iteratively work toward a result.
-- LangGraph describes agent workflows using state, nodes, and edges, where nodes perform computation and edges determine the next transition.
-- LangSmith describes evaluation as a workflow involving datasets, evaluators, and experiments to compare application versions.
-- LangChain/LangGraph documentation also distinguishes between predetermined workflows and dynamic agents; TensorTalk intentionally uses a hybrid design because the handbook QA task needs both predictable guardrails and dynamic retrieval decisions.
-
-Therefore, this project is best described as:
-
-> A from-scratch implementation of a LangChain/LangGraph-inspired agentic RAG harness, not a project built by directly calling LangChain’s prebuilt agent framework.
-
-This distinction is important. TensorTalk does not simply wrap a LangChain agent. Instead, it manually implements the major control ideas:
-
-```text
-State tracking
-→ planning
-→ retrieval/tool routing
-→ source filtering
-→ evidence normalization
-→ evaluation/judging
-→ retry/fallback
-→ final generation
-→ trace output
-```
-
-This gives the project more transparency because each part of the agent loop is visible in the notebook and UI trace.
-
----
-
-## 6.3 Agent Design Philosophy
-
-The TensorTalk agent is built around four principles.
-
-### 1. Source-constrained autonomy
-
-The agent can search and fetch information, but only from allowed official sources. It is not free to trust arbitrary search results.
-
-### 2. Evidence-first generation
-
-The model should not directly answer a web-sensitive question before evidence is collected and judged.
-
-### 3. Retry and fallback
-
-If official web evidence is weak, blocked, irrelevant, or unsafe, the system can retry with entity-aware search terms or fall back to local handbook RAG.
-
-### 4. Traceable decisions
-
-The agent does not hide its routing decisions. It records URL candidates, accepted evidence, rejected evidence, grounding decisions, and fallback decisions in trace panels.
-
----
-
-## 6.4 Official UM / FSKTM Web Agent
-
-The web agent is constrained to official UM / FSKTM domains. This is one of the most important safety and reliability choices in the project.
-
-### Priority official domains
-
-```text
-fsktm.um.edu.my
-www.um.edu.my
-um.edu.my
-```
-
-### Auxiliary official UM-related domains
-
-The project also recognizes selected UM-related service domains when they are relevant to student services, academic systems, library resources, research, career portals, or internal faculty resources:
-
-```text
-aasd.um.edu.my
-maya.um.edu.my
-umlib.um.edu.my
-umresearch.um.edu.my
-jobs.um.edu.my
-careerportal.fsktm.um.edu.my
-intra.fsktm.um.edu.my
-gallery.fsktm.um.edu.my
-```
-
-The purpose of this whitelist is not to search the whole internet. The purpose is to constrain the agent to sources that are likely to be officially controlled by UM or FSKTM.
-
----
-
-## 6.5 Domain Whitelist Design
-
-The whitelist is used as a **domain guard** before a page can become trusted evidence.
-
-The system treats URLs in three broad categories:
-
-| URL type | Handling |
-|---|---|
-| Official UM/FSKTM URL | Can be considered as candidate evidence |
-| UM-related service URL | Can be considered if relevant to the question type |
-| Non-official or synthetic URL | Rejected or downgraded |
-
-The whitelist helps prevent common LLM-agent failure cases:
-
-```text
-hallucinated programme pages
-invented lab pages
-fake student service URLs
-misrouted search results
-random third-party pages
-SEO or unrelated pages
-```
-
-For example, the project specifically tests that the agent should not invent or accept URLs like:
-
-```text
-programme-ccna-lab-more-detailedly
-bachelor-of-computer-science-artificial-intelligence
-```
-
-when those pages are not the correct evidence for the user’s question.
-
----
-
-## 6.6 Web Agent Workflow
-
-The official-source web agent follows a controlled workflow.
-
-```text
-User question
-    ↓
-Intent and entity detection
-    ↓
-Official-search query construction
-    ↓
-Candidate URL discovery
-    ↓
-Domain whitelist filtering
-    ↓
-Synthetic/fake URL rejection
-    ↓
-Fetch or static page fallback
-    ↓
-WAF/block detection
-    ↓
-Text extraction and normalization
-    ↓
-Evidence scoring
-    ↓
-Qwen evidence judge
-    ↓
-Accept evidence, retry, or fallback to handbook RAG
-```
-
-This means the agent is not only a web search function. It is a guarded evidence acquisition pipeline.
-
----
-
-## 6.7 Planning Inside the Agent
-
-Planning is a visible part of the TensorTalk system.
-
-The planning layer is responsible for deciding:
-
-```text
-Is this a stable handbook question?
-Is this a latest/current official-web question?
-Should local RAG be used first?
-Should official web discovery be attempted?
-Which entity should be searched?
-Which scope should be preferred: undergraduate, postgraduate, general, faculty, or university?
-What evidence type is expected: handbook chunk, official page, contact page, facility page, announcement, policy page?
-```
-
-This planning step is aligned with the idea that agentic systems should not directly jump from user question to final answer. They need a control stage that decides which tools and evidence paths are appropriate.
-
-TensorTalk’s planning is not a free-form hidden chain-of-thought that users must trust blindly. It is operationalized through explicit routing variables, trace objects, search decisions, and UI panels.
-
----
-
-## 6.8 Generation Inside the Agent
-
-Generation is the stage where the Qwen3-8B model produces an answer.
-
-However, generation is not allowed to operate alone. The answer generator receives controlled context:
-
-```text
-user question
-retrieved local handbook evidence
-accepted official web evidence
-scope hints
-source metadata
-harness instructions
-answer style constraints
-```
-
-The generator is expected to:
-
-```text
-answer directly
-avoid unsupported claims
-avoid fake URLs
-avoid exposing internal reasoning
-use local handbook evidence when web evidence is weak
-prefer official web evidence only when it is relevant and trusted
-```
-
-In the final stage, the generator is the PPO-trained Qwen3 actor, but it is still wrapped by the same RAG and Harness Engineering control layer.
-
----
-
-## 6.9 Evaluation Inside the Agent
-
-Evaluation is the other core part of the agent loop. TensorTalk evaluates both evidence and answers.
-
-### Evidence evaluation
-
-The system checks:
-
-```text
-Is the source official?
-Is the URL synthetic or fake?
-Is the page blocked by WAF?
-Is the evidence relevant to the user question?
-Does the evidence mention the right entity?
-Does the evidence match the expected scope?
-```
-
-### Answer evaluation
-
-The system checks:
-
-```text
-Is the final answer grounded in accepted evidence?
-Does it answer the user’s actual question?
-Does it leak internal thinking?
-Does it invent URLs?
-Is it too vague?
-Is it incomplete enough to require fallback or rewrite?
-```
-
-This creates a full agentic loop:
-
-```text
-Planning
-→ Retrieval / tool use
-→ Generation
-→ Evaluation
-→ Retry or fallback
-→ Final answer
-```
-
----
-
-## 6.10 Why the Agent Is Not Fully Autonomous
-
-The agent is intentionally not fully autonomous.
-
-A fully autonomous browsing agent may:
-
-```text
-search too broadly
-trust wrong sources
-follow irrelevant pages
-invent missing pages
-overuse web search
-ignore handbook evidence
-produce unsupported answers
-```
-
-TensorTalk instead uses a constrained model:
-
-```text
-Dynamic when needed
-Guarded by default
-Official-source-only for web evidence
-RAG-first for handbook-stable questions
-Fallback-aware when web evidence is weak
-Traceable for debugging and demonstration
-```
-
-This is more appropriate for a university handbook assistant.
-
----
-
-# 7. Harness Engineering
-
-## 7.1 What Harness Engineering Means Here
-
-Harness Engineering is the external control system around the LLM, RAG, and agent.
-
-A simple analogy:
-
-```text
-The LLM/agent is the car.
-Harness Engineering is the guardrail, traffic rule, checkpoint, fallback route, dashboard, and driving examiner.
-```
-
-The model can generate fluent answers, but the harness controls:
-
-```text
-what it can search
-which domains are trusted
-which URLs are rejected
-which evidence is useful
-when to retry
-when to fallback
-whether the answer is grounded
-whether the UI should show warning traces
-whether the final response is safe enough to display
-```
-
-In TensorTalk, Harness Engineering is not just prompt engineering. Prompt engineering tells the model what to do. Harness Engineering builds the surrounding execution system that checks whether the model actually did it correctly.
-
----
-
-## 7.2 From Prompt Engineering to Harness Engineering
-
-Prompt engineering is like telling a driver:
-
-```text
-Please drive carefully.
-```
-
-Harness Engineering is like building:
-
-```text
-lane barriers
-speed checks
-traffic rules
-navigation checkpoints
-fallback routes
-dashboards
-incident logs
-```
-
-In this project, prompt engineering alone is not enough because the model may still:
-
-```text
-invent fake URLs
-mix undergraduate and postgraduate rules
-leak internal reasoning
-trust weak web snippets
-answer without evidence
-overuse web search
-ignore local handbook RAG
-```
-
-The harness prevents or detects these failures through code-level controls, not only prompt instructions.
-
----
-
-## 7.3 From-scratch LangChain-style Harness
-
-TensorTalk’s harness was built manually rather than by importing a prebuilt LangChain/LangGraph agent.
-
-The implementation follows the same conceptual loop used in many modern agent frameworks:
-
-```text
-Planning
-→ Tool / Retrieval Action
-→ Generation
-→ Evaluation
-→ Retry / Fallback
-→ Finalization
-```
-
-But each component is implemented explicitly:
-
-| Conceptual framework idea | TensorTalk from-scratch implementation |
-|---|---|
-| Agent state | Trace dictionaries, evidence bundles, routing flags, runtime status |
-| Tools | Local RAG retriever, official web search/fetch, URL validator, evidence judge |
-| Nodes | Planning, retrieval, web discovery, evidence filtering, judging, generation, grounding |
-| Edges / transitions | Conditional retry, weak-evidence fallback, RAG fallback, final answer route |
-| Evaluation | Qwen evidence judge, rule checks, answer grounding judge, smoke tests |
-| Observability | Collapsed UI trace panels and printed diagnostic outputs |
-
-This makes the system easier to inspect in an academic notebook because the control logic is visible.
-
----
-
-## 7.4 Planning → Generation → Evaluation Closed Loop
-
-The most important Harness Engineering contribution in TensorTalk is the closed loop:
-
-```text
-Planning
-    ↓
-Generation
-    ↓
-Evaluation
-    ↓
-Retry / Fallback / Finalization
-```
-
-### Planning
-
-The planning layer decides how to handle the query.
-
-It considers:
-
-```text
-question type
-scope
-entity
-whether local RAG is enough
-whether official web is needed
-whether the query is dynamic/current
-whether the answer should be handbook-grounded or web-grounded
-```
-
-### Generation
-
-The generation layer produces an answer using controlled evidence.
-
-It receives:
-
-```text
-local handbook chunks
-official web evidence
-scope hints
-source metadata
-answer constraints
-```
-
-### Evaluation
-
-The evaluation layer checks the result.
-
-It evaluates:
-
-```text
-source trust
-URL validity
-evidence relevance
-answer grounding
-completeness
-process leakage
-fake URLs
-fallback need
-```
-
-If evaluation fails, the system can retry, reroute, or fall back.
-
-This is the engineering loop that makes TensorTalk more than a simple RAG chatbot.
-
----
-
-## 7.5 Standardized Harness Core Pipeline
-
-The final standardized TensorTalk Harness Core follows this pipeline:
-
-```text
-User Question
-    ↓
-Planning Layer
-    ↓
-Local Handbook RAG
-    ↓
-Official Web Discovery
-    ↓
-Domain Guard
-    ↓
-Fake URL Guard
-    ↓
-WAF Detection
-    ↓
-Evidence Normalizer
-    ↓
-Qwen Evidence Judge
-    ↓
-Entity-aware Retry
-    ↓
-Weak Evidence Fallback
-    ↓
-PPO/SFT Answer Generator
-    ↓
-Answer Grounding Judge
-    ↓
-Completeness Guard
-    ↓
-Final Answer
-    ↓
-UI Trace
-```
-
-This pipeline is intentionally explicit. Each part has a clear job.
-
----
-
-## 7.6 Harness State and Trace Objects
-
-The harness keeps structured trace data so that every answer can be inspected.
-
-Typical trace information includes:
-
-```text
-retrieved local RAG chunks
-candidate web URLs
-accepted official URLs
-rejected URLs
-web evidence bundle
-harness core route
-evidence judge result
-answer grounding result
-fallback reason
-final answer preview
-```
-
-This is similar in spirit to observability and tracing in agent platforms, but implemented directly in the notebook and UI.
-
----
-
-## 7.7 Domain Guard
-
-The domain guard checks whether a candidate source belongs to the allowed official domain set.
-
-It protects against:
-
-```text
-random third-party websites
-unofficial mirrors
-search result noise
-LLM-fabricated domains
-wrong university pages
-```
-
-It also makes the system explainable. If the agent rejects a page, the trace can show why.
-
----
-
-## 7.8 Fake URL Guard
-
-The fake URL guard is one of the most important parts of the project because raw LLM generations can invent plausible-looking URLs.
-
-Examples of risky synthetic URLs include:
-
-```text
-https://spectrum.umlms
-http://spectrux.medicum
-programme-ccna-lab-more-detailedly
-https://aasd um edu my/studetn
-```
-
-The guard checks and rejects URLs that:
-
-```text
-are malformed
-look fabricated
-contain suspicious path patterns
-do not belong to allowed domains
-are query-fabricated rather than discovered from official search/fetch
-```
-
-The PPO reward function also penalizes hallucinated URLs, but the harness is still necessary because reward shaping does not guarantee perfect URL behavior.
-
----
-
-## 7.9 WAF Detection
-
-Some official pages can be blocked, partially loaded, or protected by web application firewalls.
-
-The WAF-aware harness detects cases where:
-
-```text
-the page cannot be fetched normally
-the content is a block page instead of the real page
-the browser click fails
-the official site returns insufficient text
-```
-
-When this happens, the system avoids treating the blocked page as strong evidence. It can use diagnostics, retry, static fallback, or local RAG fallback.
-
----
-
-## 7.10 Evidence Normalizer
-
-Fetched web pages and handbook chunks may be noisy.
-
-The evidence normalizer attempts to convert them into a consistent evidence structure:
-
-```text
-title
-url
-source type
-domain
-text snippet
-score
-scope
-entity
-reason
-```
-
-This makes later judging and UI display easier.
-
----
-
-## 7.11 Qwen Evidence Judge
-
-The Qwen evidence judge is used to decide whether retrieved evidence actually helps answer the user’s question.
-
-It checks:
-
-```text
-Does the evidence mention the right entity?
-Does it answer the question directly?
-Is it only loosely related?
-Is it a wrong programme/page?
-Is it official but irrelevant?
-```
-
-This is important because official sources can still be irrelevant. A page can be official and still be the wrong evidence.
-
----
-
-## 7.12 Entity-aware Retry
-
-If the first web discovery result is weak or misrouted, the harness can retry with better query terms.
-
-For example, if a question about PEKOM gets routed toward an AI bachelor programme page, the harness should retry using terms related to:
-
-```text
-PEKOM
-Persatuan Komputer UM
-student society
-FSKTM student association
-```
-
-This prevents the agent from accepting the first official-looking but semantically wrong page.
-
----
-
-## 7.13 Weak Evidence Fallback
-
-If the official web evidence is weak, TensorTalk can fall back to local handbook RAG.
-
-This prevents a common agent failure:
-
-```text
-The system found a web page, so it trusts it even though it does not answer the question.
-```
-
-Instead, TensorTalk uses:
-
-```text
-web evidence if strong
-local handbook RAG if web evidence is weak
-hybrid answer if both are useful
-refusal/uncertainty if neither is sufficient
-```
-
----
-
-## 7.14 Answer Grounding Judge
-
-After answer generation, the answer grounding judge checks whether the final answer is supported by the accepted evidence.
-
-It helps catch cases where:
-
-```text
-retrieval was correct but generation added unsupported claims
-the model invented a URL
-the model mixed evidence from different scopes
-the answer contains a statement that does not appear in evidence
-```
-
-This is the evaluation part of the Planning → Generation → Evaluation loop.
-
----
-
-## 7.15 Completeness Guard
-
-The completeness guard checks whether the answer is too short, vague, or incomplete.
-
-It can identify cases where the answer:
-
-```text
-only repeats the question
-does not include required details
-misses key fields
-does not answer the requested scope
-cuts off early
-```
-
-Depending on runtime settings, this can trigger a rewrite or fallback.
-
----
-
-## 7.16 Smoke Tests as Harness Unit Checks
-
-The smoke tests are lightweight checks that make sure the harness pipeline still works after model or code changes.
-
-Examples:
-
-```text
-PEKOM should not be routed to the AI bachelor page.
-Residential college should prefer the student-affairs residential page.
-CCNA Lab should not invent synthetic URLs.
-```
-
-These tests check:
-
-```text
-routing
-URL filtering
-official page preference
-fake URL rejection
-answer grounding trace
-harness core route
-```
-
-They are not a full benchmark. They are fast sanity checks that the system still runs through the expected pipeline.
-
----
-
-## 7.17 Why Harness Engineering Is Central to This Project
-
-The final system does not rely on only one technique.
-
-```text
-SFT gives domain answer style.
-RAG gives handbook evidence.
-The web agent gives official external evidence.
-PPO improves answer behavior.
-Harness Engineering controls the whole system.
-```
-
-Without the harness, the system would still be vulnerable to:
-
-```text
-wrong source selection
-fake URLs
-weak web evidence
-scope confusion
-process leakage
-unsupported final answers
-stale artifact loading
-```
-
-Therefore, Harness Engineering is the system-level contribution that connects SFT, RAG, Agent, and PPO into one controlled workflow.
-
----
-
-# 8. Improved Model — PPO Rule-reward Post-training + RAG + Agent + Harness
-
-## 8.1 Purpose
-
-The Improved Model asks:
-
-> Can we further improve the model’s behavior after SFT/RAG by using PPO reward-based post-training?
-
-Baseline 2 already improves factual grounding through RAG and Harness Engineering. The Improved Model adds PPO to shape the model’s behavior.
-
-The goal is not to replace RAG. The goal is to make the model more aligned with the desired answer style and safety behavior.
-
----
-
-## 8.2 What PPO Means in This Project
-
-PPO stands for **Proximal Policy Optimization**.
-
-In simple terms:
-
-```text
-SFT teaches the model by imitation.
-PPO lets the model generate answers, scores them with a reward function, and updates the model toward higher-reward answers.
-```
-
-In this project:
-
-```text
-Actor model: Qwen3-8B + LoRA
-Critic/value head: TRL value head model
-Reference model: frozen Qwen3-8B reference
-Reward: rule-based preference reward function
-KL control: used to avoid drifting too far from the reference model
-```
-
----
-
-## 8.3 Rule-based Reward Function
-
-This project uses a rule-based reward function rather than a separately trained neural reward model.
-
-The reward function evaluates:
-
-```text
-gold answer similarity
-rejected answer penalty
-evidence overlap
-scope correctness
-hallucinated URL penalty
-vague answer penalty
-process/thinking leakage penalty
-direct answer bonus
-repetition penalty
-degeneration/collapse penalty
-```
-
-This is why the model card should describe the final stage as:
-
-> Rule-reward PPO post-training
-
-not:
-
-> Full RLHF with a trained reward model
-
-The reward model type recorded in the notebook is:
-
-```text
-rule_based_preference_reward_function
-uses_separate_neural_reward_model: False
-```
-
----
-
-## 8.4 PPO Training Configuration
-
-The final PPO run uses:
-
-```text
-Preference dataset rows: 1000
-Train rows: 900
-Validation rows: 100
-MAX_PPO_ROWS: None
-Train fraction: 0.90
-PPO epochs: 2
-Batch size: 2
-Mini-batch size: 1
-Max new tokens: 72
-Max PPO steps per epoch: None
-Planned steps per epoch: 450
-Total planned steps: 900
-Learning rate: 2e-6
-Target KL: 0.10
-Generation temperature: 0.45
-Top-p: 0.78
-Repetition penalty: 1.3
-No-repeat ngram size: 4
-```
-
-The run completed successfully:
-
-```text
-Global PPO steps: 900 / 900
-Elapsed time: 04:47:59
-Degenerate ratio: 0.00%
-```
-
----
-
-## 8.5 PPO Artifact Verification
-
-The Stage 3 notebook includes strict artifact verification.
-
-This is important because PPO notebooks can easily appear to run while silently saving old or incomplete artifacts.
-
-The strict save cell verifies:
-
-```text
-training_log exists
-training_log records = 900
-expected steps = 900
-MAX_PPO_ROWS = None
-train rows = 900
-valid rows = 100
-NUM_PPO_EPOCHS = 2
-MAX_PPO_STEPS_PER_EPOCH = None
-parameter hash changed after PPO
-PPO inference full actor exists
-PPO LoRA adapter exists
-non-PPO fallback forbidden
-```
-
-The final strict save output confirms:
-
-```text
-Final PPO records saved: 900 / expected 900
-Strict full PPO artifact contract passed.
-```
-
-The parameter change proof confirms:
-
-```text
-aggregate_hash_changed: true
-changed_trainable_tensors: 506
-unchanged_trainable_tensors: 0
-```
-
-This proves that PPO training changed the trainable LoRA/value-head parameters rather than merely running a dry notebook.
-
----
-
-## 8.6 Strict PPO-only Runtime
-
-The final runtime is configured so that the UI must use PPO artifacts only.
-
-The strict PPO gate confirms:
-
-```text
-PPO records: 900
-PPO full actor usable: True
-PPO LoRA adapter usable: True
-Strict PPO-only UI mode: True
-```
-
-The runtime loading order is:
-
-```text
-1. PPO full inference actor if full weights exist
-2. Otherwise base Qwen3-8B + PPO LoRA adapter
-3. Non-PPO fallback is forbidden
-```
-
-This prevents the final demo from accidentally loading an old Baseline 2 model or a stale 150-step PPO proof artifact.
-
----
-
-## 8.7 PPO Validation
-
-The PPO-only validation evaluation uses a held-out validation sample.
-
-The displayed validation summary is:
-
-```text
-reward:            0.477789
-gold_overlap:      0.255351
-rejected_overlap:  0.155080
-```
-
-Interpretation:
-
-- reward is positive
-- gold overlap is higher than rejected overlap
-- the PPO-trained actor tends to move closer to preferred answers than rejected answers
-
-This does not mean the PPO model is perfect. It means the reward-shaped behavior is directionally positive.
-
----
-
-## 8.8 PPO Limitations
-
-The PPO run is successful, but the raw PPO generations still show some imperfections.
-
-Observed issues include:
-
-1. **Process leakage**  
-   Some outputs still include phrases like:
-   ```text
-   Okay, let me try to figure out...
-   Wait, I need to check again...
-   ```
-   The reward function penalizes this, but it is not completely eliminated.
+## What This Project Does
 
-2. **Occasional hallucinated URLs**  
-   Some raw generations may still invent URLs. The harness fake URL guard is therefore still necessary.
+TensorTalk answers handbook-style questions about UM FSKTM academic rules, student guidance, programme details, facilities, dress-code guidance, industrial training, supervision policy, postgraduate requirements, and other faculty handbook topics.
 
-3. **OCR-style text artifacts**  
-   Some source chunks contain spacing or OCR issues, and the model may reproduce them.
+The project compares three stages:
 
-4. **KL can be high**  
-   Some PPO logs show high `objective/kl`, meaning the PPO actor can drift noticeably from the reference model. However, the run completed with:
-   ```text
-   degenerate_ratio = 0.00%
-   ```
-   and no detected repetition collapse.
+1. **Baseline 1: Closed-book SFT Qwen3-8B**
+   Fine-tunes Qwen3-8B on handbook question-answer pairs and tests how much the model can answer from parameters alone.
 
-5. **RAG/Harness remains necessary**  
-   PPO improves model behavior, but it does not replace retrieval grounding or guardrails.
+2. **Baseline 2: SFT + metadata-aware RAG + agent harness**
+   Adds dense retrieval over handbook chunks, metadata reranking, official-source web assistance, and guardrail checks before showing the final answer.
 
----
-
-# 9. TensorTalk UI
-
-The project includes a WhatsApp-style Jupyter HTML UI called **TensorTalk**.
-
-The UI supports:
-
-- chat-style interface
-- TensorCat avatar
-- RAG on/off control
-- web agent on/off control
-- collapsed trace panels
-- retrieved evidence display
-- web evidence display
-- planning/thinking display layer
-- harness decision trace
-- answer grounding information
-- strict PPO artifact loading
-- new chat reset behavior
-
-The UI is part of the engineering contribution because it makes the harness process visible rather than hidden.
-
----
-
-# 10. Smoke Tests
-
-## 10.1 What Smoke Test Means Here
-
-A smoke test is a lightweight system sanity check.
-
-It is not a full evaluation. It is a quick check that the main pipeline still works.
-
-In this project, smoke tests check whether:
-
-```text
-PPO model loads
-RAG retrieves evidence
-web agent searches official sources
-fake URL guard blocks synthetic links
-answer grounding returns a result
-trace structure is produced
-fallback behavior still works
-```
-
----
-
-## 10.2 Example Smoke Tests
-
-The notebook defines smoke tests such as:
-
-```text
-1. PEKOM should not be routed to AI bachelor page
-2. Residential college should prefer student-affairs residential page
-3. CCNA Lab should not invent synthetic URLs
-```
-
-These are not random examples. They are chosen to test known fragile parts of the pipeline:
-
-- entity routing
-- official URL preference
-- fake URL rejection
-- web/RAG trace structure
-
----
-
-# 11. Control Variable Design
-
-The project uses a control-variable style comparison.
-
-The base task remains the same:
-
-```text
-UM FSKTM Handbook QA
-```
-
-The base model family remains the same:
-
-```text
-Qwen3-8B
-```
-
-The dataset domain remains the same:
-
-```text
-Undergraduate + postgraduate + general UM Handbook knowledge
-```
-
-What changes is the system layer:
-
-```text
-Baseline 1: SFT only
-Baseline 2: SFT + RAG + Harness Agent
-Improved: SFT/RAG/Harness + PPO post-training
-```
-
-This allows the project to compare which improvements come from:
-
-- parameter learning
-- retrieval grounding
-- metadata-aware scope control
-- official web augmentation
-- harness guardrails
-- PPO reward shaping
-
-This is more rigorous than simply building three unrelated systems.
-
----
-
-# 12. Stage-by-stage Comparison Table
-
-| Dimension | Baseline 1: Closed-book SFT | Baseline 2: RAG + SFT + Agent/Harness | Improved Model: PPO + RAG + Agent/Harness |
-|---|---|---|---|
-| Main research question | Can the model memorize and reproduce handbook QA from SFT? | Does retrieval-grounded evidence improve handbook QA? | Can rule-reward PPO further align answer behavior while keeping RAG/Harness control? |
-| Base model | Qwen3-8B | Qwen3-8B | Qwen3-8B |
-| Main training method | Supervised fine-tuning | RAG-augmented supervised fine-tuning | Rule-reward PPO post-training |
-| Dataset used | 1000 SFT QA rows | SFT QA + metadata + RAG KB + RAG eval | 1000 PPO preference rows |
-| Train/validation/test | 800 / 100 / 100 | 8:1:1 RAG-augmented split | 900 train / 100 validation |
-| Retrieval | No | Yes | Yes |
-| Retrieval type | None | Dense embedding + FAISS + metadata-aware rerank | Same RAG runtime reused |
-| Embedding model | None | BAAI/bge-base-en-v1.5 | RAG runtime inherited from Baseline 2 |
-| Top-k evidence | None | Top-3 | Top-3 / runtime-dependent |
-| Metadata awareness | Hidden metadata only, not used at inference | Yes, scope/source/section aware | Yes, used by RAG/Harness runtime |
-| Scope control | Weak; model may confuse UG/PG if prompt is ambiguous | Stronger due to metadata-aware retrieval | Stronger due to RAG + PPO reward + harness |
-| Web agent | No | Yes | Yes |
-| Official domain control | No | Yes, UM/FSKTM official domain whitelist | Yes, same official-source guardrails |
-| Fake URL guard | No | Yes | Yes |
-| WAF handling | No | Yes | Yes |
-| Evidence judge | No | Yes, Qwen evidence judge | Yes |
-| Retry/fallback policy | No | Yes | Yes |
-| Answer grounding judge | No | Yes | Yes |
-| Completeness guard | No | Yes | Yes |
-| UI trace | Basic chat UI | Harness trace panels | Strict PPO + Harness trace panels |
-| LoRA rank | 16 | 8 | PPO actor based on LoRA actor/value setup |
-| Training epochs | 8 SFT epochs | 20 SFT epochs | 2 PPO epochs |
-| Main output artifact | LoRA adapter + merged model + `.pt` export | LoRA adapter, optional non-merged runtime | PPO full inference actor + PPO LoRA adapter + manifest |
-| Artifact strictness | Standard save | Adapter/runtime path checks | Manifest, training log count, parameter hash proof, strict gate |
-| Key metric | Test token F1 ≈ 0.8869 | RAG token F1 ≈ 0.846 on selected eval; retrieval Hit@3 ≈ 0.954 | PPO validation reward ≈ 0.4778; gold overlap > rejected overlap |
-| Strongest contribution | Clean SFT baseline | Evidence-grounded QA and metadata-aware retrieval | Full PPO post-training with strict artifact verification and harnessed runtime |
-| Main weakness | Closed-book hallucination risk | More complex runtime, depends on retriever quality | PPO raw outputs still need Harness/RAG due to possible process leakage and fake URLs |
-| Control variable role | Establishes parameter-only baseline | Adds retrieval and harness while keeping same domain/model family | Adds PPO reward shaping while preserving RAG/Harness pipeline |
-
----
-
-# 13. Technical Comparison of the Three Stages
-
-## 13.1 Content-level Difference
-
-| Content Aspect | Baseline 1 | Baseline 2 | Improved Model |
-|---|---|---|---|
-| Stable handbook facts | Learned into model parameters | Retrieved from handbook KB | Retrieved and answered by PPO-aligned actor |
-| Latest or official web info | Not supported | Supported through official web agent | Supported through same official web agent |
-| UG vs PG distinction | Learned implicitly | Controlled by metadata retrieval | Controlled by metadata retrieval + reward/harness |
-| Evidence visibility | Not shown | Evidence shown in RAG trace | Evidence shown in PPO/Harness trace |
-| Hallucination control | Mostly prompt-based | Retrieval + grounding | Retrieval + grounding + reward penalties |
-| Fake URL control | Not available | Harness URL guard | Harness URL guard + PPO penalty signal |
-
----
-
-## 13.2 Engineering-level Difference
-
-| Engineering Aspect | Baseline 1 | Baseline 2 | Improved Model |
-|---|---|---|---|
-| Notebook purpose | Train and evaluate closed-book SFT model | Build RAG-augmented model and harnessed agent runtime | Train PPO actor and attach it to final harness runtime |
-| Runtime complexity | Low | High | Highest |
-| Debug trace | Basic | Detailed RAG/Web/Harness trace | Detailed PPO/RAG/Web/Harness trace |
-| Failure handling | Minimal | Fallback and guardrail logic | Strict PPO-only fallback prevention plus harness fallback |
-| Artifact verification | Basic output save | Adapter/merged path checks | Manifest, training log count, parameter hash proof, strict gate |
-| Risk of stale artifact use | Moderate | Moderate | Actively guarded against |
-| Demo readiness | Good for simple QA | Strong for grounded QA | Strongest for final controlled system demo |
-
----
-
-# 14. Why the Improved Model Does Not Replace RAG
+3. **Improved stage: PPO rule-reward post-training + RAG + agent harness**
+   Experiments with rule-based reward shaping so responses are more grounded, concise, and aligned with the desired handbook-answer style.
 
-A key design decision is that PPO does not replace RAG.
+## System Design
 
-PPO improves the model’s tendency to:
+| Layer | Purpose |
+| --- | --- |
+| Qwen3-8B base model | General language model foundation. |
+| LoRA / QLoRA SFT | Adapts the model to UM FSKTM handbook QA style. |
+| Handbook knowledge base | Structured chunks from the undergraduate, postgraduate, and general handbook sources. |
+| Dense retrieval + FAISS | Retrieves candidate evidence using `BAAI/bge-base-en-v1.5` embeddings. |
+| Metadata-aware reranker | Uses scope, section, subsection, and keywords to reduce wrong-context answers. |
+| Official web helper | Searches constrained official UM/FSKTM-related sources when local handbook evidence is not enough. |
+| Harness engineering | Runs source guards, fake-URL guards, evidence checks, grounding checks, retry logic, and fallback rules. |
+| TensorTalk UI | Shows answers together with traceable RAG, web, and harness evidence panels. |
 
-- answer directly
-- avoid rejected-style answers
-- avoid vague answers
-- avoid process leakage
-- avoid fake URLs
-- avoid repetition collapse
-- use evidence-like wording more appropriately
+## Data Assets
 
-But PPO does not guarantee factual correctness by itself.
+The repository includes the core artifacts used to build and evaluate the assistant:
 
-Therefore, the final system still needs:
+| Artifact | Path | Size / Role |
+| --- | --- | --- |
+| SFT QA dataset | `UM_Handbook/Dataset/SFT_Dataset/SFT_QA_Training_Ready.jsonl` | 1,000 question-answer rows. |
+| SFT metadata | `UM_Handbook/Dataset/SFT_Dataset/SFT_QA_Metadata.jsonl` | 1,000 rows with scope and source metadata. |
+| RAG knowledge base | `UM_Handbook/Dataset/RAG/UM_RAG_Knowledge_Base.jsonl` | 521 retrieval chunks. |
+| RAG evaluation set | `UM_Handbook/Dataset/RAG/UM_RAG_Evaluation_Dataset.jsonl` | 1,000 retrieval-evaluation rows. |
+| Source chunk report | `UM_Handbook/Dataset/Source Chunk Dataset/Source_Chunks_Dataset_report.json` | Chunk distribution and preprocessing notes. |
+| Baseline 2 LoRA adapter | `UM_Handbook/outputs/baseline2_rag_harness_agent/lora_adapter/` | PEFT LoRA adapter and tokenizer assets. |
 
-```text
-RAG for evidence
-Web Agent for official/latest information
-Harness for source control
-Grounding judge for answer verification
-Fallback for weak evidence
-```
-
-This is the correct division of responsibility:
-
-```text
-SFT: teaches domain answer style
-RAG: supplies factual evidence
-Agent: finds official external evidence
-Harness: controls trust, routing, fallback, and trace
-PPO: improves answer behavior according to reward preferences
-```
-
----
-
-# 15. Known Limitations
-
-This project is a strong applied LLM system prototype, but it has limitations.
-
-## 15.1 Not a full human-feedback RLHF system
-
-The PPO stage uses a rule-based reward function. It does not train a separate neural reward model from human preference labels.
-
-Correct description:
-
-```text
-Rule-reward PPO post-training
-```
-
-Not:
-
-```text
-Full RLHF with learned reward model
-```
-
----
-
-## 15.2 Raw PPO generations can still be imperfect
-
-Observed raw PPO generations may include:
-
-- process leakage
-- occasional hallucinated URLs
-- OCR-like token spacing
-- incomplete course titles
-- noisy source-text reproduction
-
-The final Harness runtime is therefore necessary.
-
----
-
-## 15.3 Web search is constrained
-
-The web agent is intentionally limited to official UM/FSKTM sources. It may refuse or fallback when official evidence is weak.
-
-This is a feature, not a bug, because the system prioritizes trustworthiness over open-ended browsing.
-
----
-
-## 15.4 RAG depends on knowledge base quality
-
-If the RAG KB contains OCR noise or incomplete chunks, the model may inherit that noise. Future work should improve source cleaning and chunk normalization.
-
----
-
-## 15.5 Notebook-based prototype
-
-The project is implemented as notebooks. A production version should separate modules into:
-
-```text
-data/
-retrieval/
-agent/
-harness/
-training/
-evaluation/
-ui/
-tests/
-```
-
----
-
-# 16. Recommended Usage
-
-This project is intended for research, coursework, and demonstration purposes.
-
-It is not an official Universiti Malaya system.
-
-For official academic decisions, students should always refer to the official handbook, faculty office, or UM/FSKTM official websites.
+The 521 handbook chunks are split into 58 general, 250 postgraduate, and 213 undergraduate chunks. Low-information cover pages and divider pages are filtered before retrieval.
 
----
-
-# 17. Suggested Inference Flow
+## Evaluation Snapshot
 
-For final demonstration, use the Improved Model runtime:
+| Component | Result |
+| --- | --- |
+| Dataset split | 800 train / 100 validation / 100 test, seed 42. |
+| Baseline 2 train loss | 0.2748. |
+| Retrieval eval size | 1,000 questions. |
+| Hit@1 primary chunk | 82.1%. |
+| Hit@3 primary chunk | 95.4%. |
+| Hit@3 same knowledge group | 99.1%. |
+| Scope match at rank 1 | 99.6%. |
+| Plain generation token-F1 | 0.3391 on the sampled generation evaluation. |
+| RAG generation token-F1 | 0.8460 on the same sampled evaluation. |
 
-```text
-1. Load PPO full inference actor if available.
-2. If unavailable, load base Qwen3-8B + PPO LoRA adapter.
-3. Initialize local handbook RAG.
-4. Enable official UM/FSKTM web agent if the question may need external/latest information.
-5. Run through TensorTalkHarnessCore.
-6. Display answer with evidence trace.
-```
+The retrieval results show why the project moved beyond closed-book SFT. The model can speak in the right academic tone after fine-tuning, but RAG and harness checks make the answers more evidence-grounded and easier to audit.
 
-Strict runtime requirement:
+## Repository Map
 
 ```text
-Non-PPO fallback is forbidden in the final Improved Model demo.
+UM_Handbook/
+  Baseline_1_SFT_QWEN3_UM_Handbook_.ipynb
+  Baseline_2_RAG_SFT_QWEN3_UM_Handbook_A100_intelligent_harness_agent.ipynb
+  Improved_Model_PPO_QWEN3_UM_Handbook_RAG_Agent_Harness.ipynb
+  UM_Handbook_Markdown_Preprocess.py
+  UM_Source_Chunk_Dataset_Builder.py
+  UM_SFT_QA_Dataset_Builder_from_Index.py
+  Dataset/
+    SFT_Dataset/
+    RAG/
+    Source Chunk Dataset/
+  outputs/
+    baseline2_rag_harness_agent/
+      lora_adapter/
+      retrieval_eval/
+      generation_eval/
+      rag_augmented_dataset/
 ```
 
----
-
-# 18. Relation to LangChain / LangGraph / LangSmith Concepts
-
-This project does not claim to be a LangChain implementation. Instead, it uses a from-scratch notebook implementation that follows similar engineering ideas.
-
-Official LangChain ecosystem references that influenced the design include:
+## Loading the Baseline 2 Adapter
 
-- [LangChain Agents documentation](https://docs.langchain.com/oss/javascript/langchain/agents): agents combine language models with tools and can iteratively work toward a goal.
-- [LangGraph Overview](https://docs.langchain.com/oss/python/langgraph/overview): LangGraph focuses on durable execution, streaming, human-in-the-loop, persistence, and orchestration for agent workflows.
-- [LangGraph Graph API](https://docs.langchain.com/oss/python/langgraph/graph-api): agent workflows can be modeled through state, nodes, and edges.
-- [LangGraph Workflows and Agents](https://docs.langchain.com/oss/python/langgraph/workflows-agents): workflows use predetermined code paths, while agents are more dynamic in tool usage and process control.
-- [LangSmith Evaluation](https://docs.langchain.com/langsmith/evaluation): evaluation can be structured around datasets, evaluators, and experiments.
-- [LangSmith Evaluation Types](https://docs.langchain.com/langsmith/evaluation-types): evaluation may include benchmarking, unit tests, regression tests, LLM-as-judge evaluators, code evaluators, and online monitoring.
-- [LangSmith Application-specific Evaluation Approaches](https://docs.langchain.com/langsmith/evaluation-approaches): autonomous agents are commonly discussed in terms of tool calling, memory, and planning.
+The Baseline 2 adapter is stored in a subfolder of this repository. A typical PEFT loading flow is:
 
-TensorTalk maps these ideas into a custom system:
+```python
+import torch
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
 
-| LangChain ecosystem idea | TensorTalk implementation |
-|---|---|
-| Agent uses model + tools | Qwen3 model + local RAG + official web search + URL validator + evidence judge |
-| State | Trace dictionaries, evidence bundles, routing flags, model/backend status |
-| Nodes | Planning, retrieval, web discovery, filtering, judging, generation, grounding, completeness checking |
-| Edges | Conditional retry, official-web route, local-RAG fallback, weak-evidence fallback, final-answer route |
-| Planning | Query classification, scope detection, entity-aware routing, web/RAG decision |
-| Generation | SFT/PPO Qwen3 actor generates with accepted evidence |
-| Evaluation | Evidence judge, answer grounding judge, completeness guard, fake URL checks, smoke tests |
-| Observability | TensorTalk collapsed trace panels and diagnostic logs |
-| Regression/smoke testing | PEKOM route test, residential-college URL test, CCNA synthetic URL test |
+base_model_id = "Qwen/Qwen3-8B"
+adapter_repo = "TensorCat/TensorTalk"
+adapter_subfolder = "UM_Handbook/outputs/baseline2_rag_harness_agent/lora_adapter"
 
-This is why the project can be described as:
+tokenizer = AutoTokenizer.from_pretrained(
+    adapter_repo,
+    subfolder=adapter_subfolder,
+    trust_remote_code=True,
+)
 
-> A from-scratch LangChain/LangGraph-inspired RAG agent harness for UM Handbook QA, with a Planning → Generation → Evaluation control loop.
+base_model = AutoModelForCausalLM.from_pretrained(
+    base_model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+    trust_remote_code=True,
+)
+
+model = PeftModel.from_pretrained(
+    base_model,
+    adapter_repo,
+    subfolder=adapter_subfolder,
+)
+model.eval()
+```
+
+For the full TensorTalk behavior shown in the screenshot, use the adapter together with the RAG knowledge base, FAISS retriever, official-source web helper, and harness checks from the notebooks. The model weights alone do not include the live retrieval index or web-agent runtime.
+
+## Intended Use
+
+TensorTalk is intended for research, education, and demonstration of:
+
+- handbook question answering for UM FSKTM content;
+- RAG-grounded answer generation;
+- metadata-aware retrieval and reranking;
+- controlled agent behavior over official web sources;
+- harness engineering for evidence checks, fake URL detection, retries, and fallback;
+- comparing closed-book SFT against retrieval-grounded and reward-shaped systems.
 
----
-
-# 19. Summary
-
-TensorTalk demonstrates a staged LLM system development workflow:
-
-```text
-Baseline 1:
-Qwen3-8B learns handbook QA through closed-book SFT.
-
-Baseline 2:
-The system adds RAG, dense retrieval, metadata-aware reranking, official web search, and Harness Engineering.
-
-Improved Model:
-The system adds full 1000-row rule-reward PPO post-training, strict artifact verification, and a PPO-only final harness runtime.
-```
+## Out-of-Scope Use
 
-The most important contribution is not only that the model can answer handbook questions, but that the system is controlled, evidence-aware, source-constrained, traceable, and evaluated through a clear baseline progression.
+Do not use TensorTalk as an official university policy authority, legal adviser, disciplinary decision system, or fully autonomous student-support system. University policies can change, and final answers should be checked against the latest official UM/FSKTM documents when used for real administrative decisions.
+
+## Limitations
+
+- The strongest behavior comes from the full runtime pipeline, not from the adapter by itself.
+- RAG quality depends on the handbook chunks, retrieval metadata, and official-source availability.
+- The web helper is intentionally constrained to trusted domains; it is not a general web search assistant.
+- PPO in this project is rule-reward post-training, not a large-scale human-feedback RLHF pipeline.
+- Some notebook paths reflect the original training environment and may need local path adjustment before rerunning.
 
-The final system should be understood as:
+## License
 
-> **A Qwen3-8B based UM Handbook RAG Agent, improved with rule-reward PPO and controlled by Harness Engineering.**
+This project is released under the Apache 2.0 license.