Upload folder using huggingface_hub

Dockerfile

@@ -0,0 +1,15 @@
+FROM python:3.12-slim
+
+WORKDIR /app
+
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+COPY . .
+
+EXPOSE 7860
+
+CMD ["streamlit", "run", "app.py", \
+     "--server.port=7860", \
+     "--server.address=0.0.0.0", \
+     "--server.headless=true"]

KnowledgeValueLab.md

@@ -0,0 +1,430 @@
+# Knowledge Value Lab (KVL)
+
+## A Framework for Measuring the Marginal Value of Knowledge Assets for AI Systems
+
+### Concept Note and Implementation Plan
+
+## Executive Summary
+
+Organizations around the world are investing heavily in creating, curating, digitizing, licensing, and publishing knowledge assets for use in artificial intelligence systems.
+
+These assets include:
+
+* Research papers
+* Technical reports
+* Books
+* Policy documents
+* Government publications
+* Educational materials
+* Domain-specific knowledge bases
+* Datasets
+* Web archives
+
+Despite growing investments in AI-ready content, there is currently no widely accepted method for answering a fundamental question:
+
+**How much value does a newly available knowledge asset contribute to AI systems?**
+
+A document may contain information that is already embedded in existing foundation models, in which case its marginal contribution is small. Alternatively, it may contain unique knowledge that significantly improves retrieval systems, AI assistants, decision-support tools, and downstream applications.
+
+Knowledge Value Lab (KVL) is a proposed framework and software platform designed to measure the marginal value of knowledge assets for AI systems. The platform evaluates individual documents, datasets, repositories, and collections using a standardized methodology that combines knowledge novelty, retrieval performance, answer quality, grounding, and user demand.
+
+The result is a transparent and reproducible "Knowledge Value Score" that quantifies the contribution of information assets to AI ecosystems.
+
+---
+
+# 1. Motivation
+
+The emergence of foundation models has fundamentally changed how knowledge is consumed and utilized.
+
+Historically, the value of a document was measured through indicators such as:
+
+* Citations
+* Downloads
+* Sales
+* Views
+* Academic impact
+
+These measures provide limited insight into the role of knowledge within AI systems.
+
+A document that is rarely cited may substantially improve an AI assistant's ability to answer questions. Conversely, a highly cited document may contribute little additional value if its contents are already extensively represented within existing training corpora.
+
+Organizations increasingly face decisions regarding:
+
+* Which content should be digitized?
+* Which repositories should be prioritized?
+* Which datasets deserve funding?
+* Which knowledge assets should be licensed for AI applications?
+* Which public data investments create the greatest societal return?
+
+KVL seeks to provide evidence-based answers to these questions.
+
+---
+
+# 2. Vision
+
+To create a standardized framework for measuring the contribution of knowledge assets to AI systems, enabling informed decisions about data investments, content sharing, and knowledge infrastructure development.
+
+---
+
+# 3. Core Research Questions
+
+### Knowledge Novelty
+
+How much information contained in an asset is already known by contemporary AI models?
+
+### Retrieval Utility
+
+How much does the asset improve information retrieval systems?
+
+### Generation Utility
+
+How much does the asset improve AI-generated responses?
+
+### Attribution Utility
+
+Can improvements be directly attributed to the asset?
+
+### Demand Utility
+
+How frequently is the knowledge needed by users?
+
+### Social Utility
+
+What societal value may arise from making the knowledge available?
+
+---
+
+# 4. Conceptual Framework
+
+KVL treats every knowledge asset as a potential contributor to AI capability.
+
+The framework estimates value across five dimensions.
+
+## Dimension 1: Knowledge Novelty
+
+Measures whether the information contained within a document is already represented in existing AI models.
+
+Examples:
+
+* Recently published research
+* Local knowledge
+* Proprietary content
+* Specialized technical documentation
+* Low-resource language materials
+
+may receive high novelty scores.
+
+Widely distributed information may receive lower scores.
+
+### Outputs
+
+Knowledge Novelty Score
+
+0–100
+
+---
+
+## Dimension 2: Retrieval Utility
+
+Measures whether the asset improves search and retrieval systems.
+
+Typical evaluation metrics include:
+
+* Recall@K
+* Mean Reciprocal Rank
+* nDCG
+* Context Precision
+* Context Recall
+
+### Outputs
+
+Retrieval Utility Score
+
+0–100
+
+---
+
+## Dimension 3: Generation Utility
+
+Measures whether access to the asset improves AI-generated outputs.
+
+Applications include:
+
+* Question answering
+* Summarization
+* Advisory systems
+* Research assistants
+* Educational tutors
+* Enterprise knowledge assistants
+
+Evaluation criteria include:
+
+* Accuracy
+* Completeness
+* Specificity
+* Relevance
+* Actionability
+* Safety
+
+### Outputs
+
+Generation Utility Score
+
+0–100
+
+---
+
+## Dimension 4: Attribution and Grounding
+
+Measures whether observed improvements genuinely originate from the asset.
+
+Key questions include:
+
+* Is the document being retrieved?
+* Is evidence from the document being used?
+* Are generated outputs properly grounded?
+
+### Outputs
+
+Grounding Score
+
+0–100
+
+---
+
+## Dimension 5: Demand Utility
+
+Measures the practical importance of the knowledge.
+
+Examples include:
+
+* Frequency of related user queries
+* Coverage of unmet information needs
+* Relevance to priority domains
+* Geographic relevance
+* Language coverage
+
+### Outputs
+
+Demand Utility Score
+
+0–100
+
+---
+
+# 5. Knowledge Value Score
+
+The overall score combines all dimensions into a single measure.
+
+KVS =
+
+30% Knowledge Novelty
+
+20% Retrieval Utility
+
+25% Generation Utility
+
+15% Grounding Utility
+
+10% Demand Utility
+
+Result:
+
+0–100
+
+Classification:
+
+0–20 Minimal Value
+
+21–40 Incremental Value
+
+41–60 Moderate Value
+
+61–80 High Value
+
+81–100 Transformational Value
+
+---
+
+# 6. System Architecture
+
+## Module A: Knowledge Novelty Engine
+
+Functions:
+
+* Claim extraction
+* Question generation
+* Closed-book model evaluation
+* Cross-model comparison
+* Novelty estimation
+
+Outputs:
+
+Knowledge Novelty Score
+
+---
+
+## Module B: Retrieval Evaluation Engine
+
+Functions:
+
+* Index creation
+* Retrieval benchmarking
+* Search quality assessment
+* Comparative experiments
+
+Outputs:
+
+Retrieval Utility Score
+
+---
+
+## Module C: Generation Evaluation Engine
+
+Functions:
+
+* Response generation
+* Multi-model testing
+* Quality assessment
+* Human and AI judging
+
+Outputs:
+
+Generation Utility Score
+
+---
+
+## Module D: Attribution Engine
+
+Functions:
+
+* Citation analysis
+* Evidence tracing
+* Source attribution
+* Grounding verification
+
+Outputs:
+
+Grounding Score
+
+---
+
+## Module E: Demand Analysis Engine
+
+Functions:
+
+* Query log analysis
+* Topic modeling
+* Gap detection
+* User demand estimation
+
+Outputs:
+
+Demand Utility Score
+
+---
+
+# 7. User Experience
+
+Users upload:
+
+* PDF
+* Word documents
+* Web pages
+* Datasets
+* Knowledge collections
+
+The platform automatically:
+
+1. Ingests content
+2. Extracts claims
+3. Generates evaluation tasks
+4. Executes experiments
+5. Computes scores
+6. Produces a report
+
+Typical runtime:
+
+Minutes to hours depending on corpus size.
+
+---
+
+# 8. Dashboard Outputs
+
+The platform generates a Knowledge Value Report containing:
+
+### Overall Knowledge Value Score
+
+### Knowledge Novelty Assessment
+
+### Retrieval Impact Analysis
+
+### Generation Impact Analysis
+
+### Attribution Assessment
+
+### Demand Analysis
+
+### Recommended Actions
+
+Examples:
+
+* Publish openly
+* Prioritize indexing
+* Translate into additional languages
+* Integrate into retrieval systems
+* Acquire licensing rights
+* Merge with related collections
+
+---
+
+# 9. Extension to Repository-Level Evaluation
+
+The framework can be applied to:
+
+* Digital libraries
+* Academic repositories
+* Government archives
+* Corporate knowledge bases
+* Publisher collections
+* Data commons
+* Open data platforms
+
+This enables comparative analyses such as:
+
+* Which repository contributes the most novel knowledge?
+* Which collection generates the largest gains in AI performance?
+* Which public data investments generate the greatest value?
+
+---
+
+# 10. Social Return on Knowledge
+
+An optional extension estimates downstream societal value.
+
+Knowledge assets are evaluated not only by their impact on AI performance but also by their contribution to real-world outcomes.
+
+Examples:
+
+Document → Improved AI Output → Better Decision → Improved Outcome
+
+Possible outcome domains include:
+
+* Education
+* Healthcare
+* Agriculture
+* Public administration
+* Climate adaptation
+* Scientific research
+
+This extension enables estimation of a Social Return on Knowledge (SRK) score.
+
+---
+
+# 11. Long-Term Vision
+
+Knowledge Value Lab aims to become a standard for measuring the value of knowledge in the AI era.
+
+Just as citation metrics transformed scholarly communication and web analytics transformed digital publishing, KVL seeks to establish a new class of metrics that quantify how knowledge contributes to artificial intelligence systems.
+
+The ultimate goal is to enable governments, publishers, researchers, funders, and technology developers to make evidence-based decisions about the creation, sharing, preservation, and financing of knowledge assets in a world increasingly mediated by AI.

README.md

@@ -1,10 +1,46 @@
 ---
 title: Knowledge Value Lab
-emoji: 🦀
-colorFrom: pink
-colorTo: blue
+emoji: 🔬
+colorFrom: blue
+colorTo: indigo
 sdk: docker
+app_port: 7860
 pinned: false
+license: apache-2.0
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Knowledge Value Lab (KVL)
+
+**Measuring the Marginal Value of Knowledge Assets for AI Systems**
+
+KVL quantifies how much a knowledge document contributes to AI systems across five dimensions, producing a single weighted **Knowledge Value Score (KVS)**.
+
+## How to Use
+
+1. Upload a Markdown (`.md`) document
+2. Click **Evaluate Knowledge Value**
+3. Review the scored report and download it
+
+## Dimensions
+
+| Dimension | Weight | What it measures |
+|---|---|---|
+| Knowledge Novelty | 30% | How much of the document is unknown to the base model |
+| Retrieval Utility | 20% | How well the document surfaces in RAG search |
+| Generation Utility | 25% | How much RAG answers improve over baseline |
+| Attribution & Grounding | 15% | How faithfully answers are grounded in the document |
+| Demand Utility | 10% | How frequently this knowledge is needed by users |
+
+## Score Classifications
+
+| Score | Classification |
+|---|---|
+| 81–100 | Transformational Value |
+| 61–80 | High Value |
+| 41–60 | Moderate Value |
+| 21–40 | Incremental Value |
+| 0–20 | Minimal Value |
+
+## Important Note
+
+Knowledge Novelty and Generation Utility scores are **model-relative** — they measure value against specific AI models and will change when models are updated. Always report scores alongside the model names and evaluation date shown in each report.

app.py

@@ -0,0 +1,508 @@
+"""Knowledge Value Lab — Streamlit prototype."""
+
+from __future__ import annotations
+import os
+import time
+import anthropic
+import streamlit as st
+from dotenv import load_dotenv
+from datetime import datetime
+
+load_dotenv()
+
+from kvl import ingestor, scorer, report
+from kvl.modules import novelty, retrieval, generation, attribution, demand
+from kvl.config import DIMENSION_META, KVS_CLASSIFICATION, MODELS, SENSITIVITY_COLOR, model_meta
+
+# ── Page config ───────────────────────────────────────────────────────────────
+st.set_page_config(
+    page_title="Knowledge Value Lab",
+    page_icon="🔬",
+    layout="wide",
+)
+
+# ── CSS ───────────────────────────────────────────────────────────────────────
+st.markdown("""
+<style>
+.kvs-box {
+    background: linear-gradient(135deg, #1e3a5f 0%, #0d2137 100%);
+    border-radius: 12px;
+    padding: 28px 36px;
+    text-align: center;
+    margin-bottom: 8px;
+}
+.kvs-number { font-size: 64px; font-weight: 800; color: #f0f4ff; line-height: 1; }
+.kvs-label  { font-size: 16px; color: #8ab4f8; margin-top: 4px; }
+.kvs-class  { font-size: 22px; font-weight: 600; margin-top: 8px; }
+.kvs-meta   { font-size: 12px; color: #556; margin-top: 10px; font-family: monospace; }
+.model-chip {
+    display: inline-block;
+    background: #1a2740;
+    border: 1px solid #2a4060;
+    border-radius: 4px;
+    padding: 2px 8px;
+    font-size: 11px;
+    font-family: monospace;
+    color: #8ab4f8;
+    margin: 2px;
+}
+.sens-badge {
+    display: inline-block;
+    border-radius: 4px;
+    padding: 1px 7px;
+    font-size: 11px;
+    font-weight: 600;
+}
+.warn-box {
+    background: #1a1500;
+    border-left: 3px solid #f8961e;
+    border-radius: 4px;
+    padding: 10px 14px;
+    font-size: 13px;
+    margin: 8px 0 16px 0;
+    color: #fff;
+}
+</style>
+""", unsafe_allow_html=True)
+
+
+# ── Sidebar ───────────────────────────────────────────────────────────────────
+with st.sidebar:
+    st.markdown("## 🔬 Knowledge Value Lab")
+    st.markdown(
+        "KVL measures the **marginal value** of a knowledge document to an AI system "
+        "across five independent dimensions, producing a single weighted **Knowledge Value Score (KVS)**."
+    )
+
+    st.divider()
+    st.markdown("### Models Used")
+    for key, m in MODELS.items():
+        st.markdown(
+            f"<span class='model-chip'>{m['display']}</span>  \n"
+            f"<span style='font-size:11px;color:#888;'>{m['role']}</span>",
+            unsafe_allow_html=True,
+        )
+        st.markdown("")
+
+    st.markdown(
+        "<div class='warn-box'>"
+        "⚠️ <strong>Scores are model-relative.</strong> "
+        "Knowledge Novelty and Generation Utility reflect this document's value "
+        "to the <em>specific models above</em>. Scores will change when models are updated. "
+        "Always report scores alongside the model names and evaluation date."
+        "</div>",
+        unsafe_allow_html=True,
+    )
+
+    st.divider()
+    st.markdown("### Score Classifications")
+    for threshold, label, desc in KVS_CLASSIFICATION:
+        hi = threshold + 19 if threshold < 81 else 100
+        st.markdown(f"**{threshold}–{hi}** — {label}")
+        st.caption(desc)
+
+    st.divider()
+    st.markdown("### Metric Guide")
+    for key, dmeta in DIMENSION_META.items():
+        sens = dmeta["model_sensitivity"]
+        sc = SENSITIVITY_COLOR[sens]
+        with st.expander(f"{dmeta['name']}  ·  {int(dmeta['weight']*100)}%"):
+            st.markdown(
+                f"<span class='sens-badge' style='background:{sc}22;color:{sc};border:1px solid {sc}55;'>"
+                f"Model sensitivity: {sens}</span>",
+                unsafe_allow_html=True,
+            )
+            st.markdown(dmeta["description"])
+            st.markdown(f"**How measured:** {dmeta['how_measured']}")
+            st.markdown(f"*{dmeta['sensitivity_note']}*")
+            st.markdown(f"**High score:** {dmeta['high_means']}")
+            st.markdown(f"**Low score:** {dmeta['low_means']}")
+
+
+# ── Header ────────────────────────────────────────────────────────────────────
+st.title("🔬 Knowledge Value Lab")
+st.markdown(
+    "**Measuring the Marginal Value of Knowledge Assets for AI Systems**  \n"
+    "Upload a Markdown document to receive a quantified Knowledge Value Score across five dimensions."
+)
+st.divider()
+
+
+# ── Cached resources ────────────────────────────────���─────────────────────────
+@st.cache_resource(show_spinner="Loading embedding model (all-MiniLM-L6-v2)...")
+def load_embedder():
+    from sentence_transformers import SentenceTransformer
+    return SentenceTransformer("all-MiniLM-L6-v2")
+
+
+@st.cache_resource
+def load_client():
+    api_key = os.getenv("ANTHROPIC_API_KEY")
+    if not api_key:
+        st.error("ANTHROPIC_API_KEY not found. Add it to your .env file.")
+        st.stop()
+    return anthropic.Anthropic(api_key=api_key)
+
+
+# ── Upload section ────────────────────────────────────────────────────────────
+col_upload, col_preview = st.columns([1, 1], gap="large")
+
+with col_upload:
+    st.markdown("### Upload Document")
+    uploaded = st.file_uploader(
+        "Choose a Markdown file",
+        type=["md"],
+        help="Upload a .md file to evaluate its knowledge value for AI systems.",
+        label_visibility="collapsed",
+    )
+
+    if uploaded:
+        md_text = uploaded.read().decode("utf-8")
+        doc = ingestor.parse(md_text)
+        st.success(
+            f"**{doc.title}**  \n"
+            f"{doc.word_count:,} words · {len(doc.sections)} sections · {len(doc.chunks)} chunks"
+        )
+        run = st.button("▶  Evaluate Knowledge Value", type="primary", use_container_width=True)
+    else:
+        st.info("Drag and drop a `.md` file above, or click to browse.")
+        run = False
+
+with col_preview:
+    st.markdown("### Document Preview")
+    if uploaded:
+        preview_text = md_text[:1200] + ("…" if len(md_text) > 1200 else "")
+        st.markdown(
+            f"<div style='background:#0e1117;border:1px solid #2a2a3a;border-radius:8px;"
+            f"padding:16px;font-size:13px;max-height:280px;overflow:auto;white-space:pre-wrap;color:#fff;'>"
+            f"{preview_text}</div>",
+            unsafe_allow_html=True,
+        )
+    else:
+        st.markdown(
+            "<div style='background:#0e1117;border:1px solid #2a2a3a;border-radius:8px;"
+            "padding:40px;text-align:center;color:#555;'>No document uploaded yet</div>",
+            unsafe_allow_html=True,
+        )
+
+
+# ── Evaluation pipeline ───────────────────────────────────────────────────────
+if run and uploaded:
+    st.divider()
+    st.markdown("### Evaluation in Progress")
+
+    client = load_client()
+    embedder = load_embedder()
+
+    steps = [
+        "Module A: Knowledge Novelty",
+        "Module B: Retrieval Utility",
+        "Module C: Generation Utility",
+        "Module D: Attribution & Grounding",
+        "Module E: Demand Utility",
+        "Computing Knowledge Value Score",
+    ]
+    progress_bar = st.progress(0)
+    status_text = st.empty()
+    step_statuses = {s: "pending" for s in steps}
+    step_container = st.container()
+
+    def render_steps():
+        with step_container:
+            for s, state in step_statuses.items():
+                icon  = {"pending": "○", "running": "⟳", "done": "✓"}[state]
+                color = {"pending": "#555", "running": "#8ab4f8", "done": "#4caf87"}[state]
+                st.markdown(
+                    f"<span style='color:{color};font-family:monospace;'>{icon}  {s}</span>",
+                    unsafe_allow_html=True,
+                )
+
+    def sub_progress(msg: str):
+        status_text.markdown(f"*{msg}*")
+
+    module_results = {}
+    eval_start = datetime.now()
+    t0 = time.time()
+
+    step_statuses[steps[0]] = "running"; render_steps(); progress_bar.progress(5)
+    module_results["novelty"] = novelty.evaluate(client, doc, progress_cb=sub_progress)
+    step_statuses[steps[0]] = "done"; progress_bar.progress(20)
+
+    step_statuses[steps[1]] = "running"; render_steps()
+    module_results["retrieval"] = retrieval.evaluate(client, doc, embedder, progress_cb=sub_progress)
+    step_statuses[steps[1]] = "done"; progress_bar.progress(40)
+
+    step_statuses[steps[2]] = "running"; render_steps()
+    module_results["generation"] = generation.evaluate(client, doc, progress_cb=sub_progress)
+    step_statuses[steps[2]] = "done"; progress_bar.progress(60)
+
+    step_statuses[steps[3]] = "running"; render_steps()
+    module_results["attribution"] = attribution.evaluate(
+        client, doc, module_results["generation"], embedder, progress_cb=sub_progress
+    )
+    step_statuses[steps[3]] = "done"; progress_bar.progress(80)
+
+    step_statuses[steps[4]] = "running"; render_steps()
+    module_results["demand"] = demand.evaluate(client, doc, progress_cb=sub_progress)
+    step_statuses[steps[4]] = "done"; progress_bar.progress(92)
+
+    step_statuses[steps[5]] = "running"; render_steps()
+    status_text.markdown("*Computing weighted Knowledge Value Score...*")
+    dim_scores = {k: module_results[k]["score"] for k in module_results}
+    kvs_result = scorer.compute(dim_scores)
+    step_statuses[steps[5]] = "done"; progress_bar.progress(100)
+    render_steps()
+
+    elapsed = round(time.time() - t0)
+    status_text.markdown(f"*Evaluation complete in {elapsed}s.*")
+    eval_date_str = eval_start.strftime("%Y-%m-%d %H:%M UTC")
+    meta = model_meta(eval_date_str)
+
+    # ── Results ───────────────────────────────────────────────────────────────
+    st.divider()
+    st.markdown("## Knowledge Value Report")
+
+    kvs = kvs_result["kvs"]
+    classification = kvs_result["classification"]
+    color_map = {
+        "Transformational Value": "#ffd166",
+        "High Value":             "#06d6a0",
+        "Moderate Value":         "#8ab4f8",
+        "Incremental Value":      "#f8961e",
+        "Minimal Value":          "#ef476f",
+    }
+    badge_color = color_map.get(classification, "#8ab4f8")
+
+    # KVS hero with model metadata
+    st.markdown(
+        f"""<div class="kvs-box">
+            <div class="kvs-number">{kvs}</div>
+            <div class="kvs-label">Knowledge Value Score / 100</div>
+            <div class="kvs-class" style="color:{badge_color};">{classification}</div>
+            <div class="kvs-meta">
+                Evaluated {eval_date_str}<br>
+                Judge: {MODELS['judge']['display']} &nbsp;·&nbsp;
+                Worker: {MODELS['worker']['display']} &nbsp;·&nbsp;
+                Embeddings: {MODELS['embedder']['display']}
+            </div>
+        </div>""",
+        unsafe_allow_html=True,
+    )
+
+    # Model-relativity warning
+    st.markdown(
+        "<div class='warn-box'>"
+        "⚠️ <strong>Score validity:</strong> "
+        "Knowledge Novelty and Generation Utility are <strong>model-relative</strong> — "
+        "they reflect this document's marginal value to the models listed above. "
+        "Scores will change if the underlying models are updated or replaced. "
+        "Always report scores alongside model names and evaluation date."
+        "</div>",
+        unsafe_allow_html=True,
+    )
+
+    # ── Dimension breakdown ───────────────────────────────────────────────────
+    st.markdown("### Dimension Breakdown")
+    st.caption(
+        "Each dimension is scored 0–100 and weighted by its contribution to the overall KVS. "
+        "The sensitivity badge shows how much the score depends on the specific AI model used."
+    )
+
+    dims = [
+        ("Knowledge Novelty",  "novelty",     0.30),
+        ("Retrieval Utility",  "retrieval",   0.20),
+        ("Generation Utility", "generation",  0.25),
+        ("Attribution",        "attribution", 0.15),
+        ("Demand Utility",     "demand",      0.10),
+    ]
+
+    for label, key, weight in dims:
+        sc = dim_scores[key]
+        contrib = kvs_result["weighted_contributions"][key]
+        filled = round(sc / 100 * 20)
+        bar_str = "█" * filled + "░" * (20 - filled)
+        pct = int(weight * 100)
+        sens = DIMENSION_META[key]["model_sensitivity"]
+        sc_color = SENSITIVITY_COLOR[sens]
+
+        col1, col2 = st.columns([4, 1])
+        with col1:
+            st.markdown(
+                f"**{label}** &nbsp;"
+                f"<span class='sens-badge' style='background:{sc_color}22;color:{sc_color};"
+                f"border:1px solid {sc_color}55;'>sensitivity: {sens}</span>  \n"
+                f"`{bar_str}` &nbsp; **{sc}/100** "
+                f"<span style='color:#888;font-size:13px;'> ×{pct}% = {contrib} pts</span>",
+                unsafe_allow_html=True,
+            )
+        with col2:
+            st.metric(label="score", value=str(sc), label_visibility="collapsed")
+
+    # ── Detailed analysis expanders ───────────────────────────────────────────
+    st.markdown("### Detailed Analysis")
+
+    # Module A
+    dmeta = DIMENSION_META["novelty"]
+    with st.expander(f"Module A — Knowledge Novelty  ·  {dim_scores['novelty']}/100"):
+        sens = dmeta["model_sensitivity"]
+        sc = SENSITIVITY_COLOR[sens]
+        st.markdown(
+            f"<span class='sens-badge' style='background:{sc}22;color:{sc};border:1px solid {sc}55;'>"
+            f"Model sensitivity: {sens}</span>",
+            unsafe_allow_html=True,
+        )
+        st.markdown(f"**What this measures:** {dmeta['description']}")
+        st.markdown(f"**How it's measured:** {dmeta['how_measured']}")
+        st.markdown(f"**Models used:** {', '.join(dmeta['models_used'])}")
+        st.info(dmeta["sensitivity_note"])
+        st.divider()
+        st.markdown(f"**Result:** {module_results['novelty']['summary']}")
+        details = module_results["novelty"].get("details", [])
+        if details:
+            st.markdown("**Claim analysis** (🟢 novel · 🟡 partial · 🔴 already known):")
+            for d in details:
+                known_pct = round(d["known_score"] * 100)
+                icon = "🟢" if d["known_score"] < 0.4 else ("🟡" if d["known_score"] < 0.7 else "🔴")
+                st.markdown(
+                    f"{icon} **{d['claim'][:130]}**  \n"
+                    f"*Known to model: {known_pct}% — {d['reason']}*"
+                )
+
+    # Module B
+    dmeta = DIMENSION_META["retrieval"]
+    with st.expander(f"Module B — Retrieval Utility  ·  {dim_scores['retrieval']}/100"):
+        sens = dmeta["model_sensitivity"]
+        sc = SENSITIVITY_COLOR[sens]
+        st.markdown(
+            f"<span class='sens-badge' style='background:{sc}22;color:{sc};border:1px solid {sc}55;'>"
+            f"Model sensitivity: {sens}</span>",
+            unsafe_allow_html=True,
+        )
+        st.markdown(f"**What this measures:** {dmeta['description']}")
+        st.markdown(f"**How it's measured:** {dmeta['how_measured']}")
+        st.markdown(f"**Models used:** {', '.join(dmeta['models_used'])}")
+        st.info(dmeta["sensitivity_note"])
+        st.divider()
+        st.markdown(f"**Result:** {module_results['retrieval']['summary']}")
+        details = module_results["retrieval"].get("details", [])
+        if details:
+            st.caption(
+                "**Recall@3** — fraction of queries where the correct chunk appears in top 3 results (1.0 = perfect).  \n"
+                "**MRR** — Mean Reciprocal Rank; how high the correct chunk ranks on average (1.0 = always first)."
+            )
+            st.table({
+                "Query":    [d["query"] for d in details],
+                "Recall@3": [f"{d['recall_at_3']:.2f}" for d in details],
+                "MRR":      [f"{d['reciprocal_rank']:.2f}" for d in details],
+            })
+
+    # Module C
+    dmeta = DIMENSION_META["generation"]
+    with st.expander(f"Module C — Generation Utility  ·  {dim_scores['generation']}/100"):
+        sens = dmeta["model_sensitivity"]
+        sc = SENSITIVITY_COLOR[sens]
+        st.markdown(
+            f"<span class='sens-badge' style='background:{sc}22;color:{sc};border:1px solid {sc}55;'>"
+            f"Model sensitivity: {sens}</span>",
+            unsafe_allow_html=True,
+        )
+        st.markdown(f"**What this measures:** {dmeta['description']}")
+        st.markdown(f"**How it's measured:** {dmeta['how_measured']}")
+        st.markdown(f"**Models used:** {', '.join(dmeta['models_used'])}")
+        st.info(dmeta["sensitivity_note"])
+        st.divider()
+        st.markdown(f"**Result:** {module_results['generation']['summary']}")
+        for d in module_results["generation"].get("details", []):
+            st.markdown(f"**Q: {d['question']}**")
+            c1, c2 = st.columns(2)
+            with c1:
+                st.markdown("*Baseline — no document:*")
+                st.markdown(f"> {d['baseline_answer'][:350]}")
+            with c2:
+                st.markdown("*RAG — with document:*")
+                st.markdown(f"> {d['rag_answer'][:350]}")
+            st.caption(
+                f"Improvement: **{d['improvement']}/100** &nbsp;|&nbsp; "
+                f"Accuracy: {d['accuracy']}/5 &nbsp;|&nbsp; "
+                f"Completeness: {d['completeness']}/5 &nbsp;|&nbsp; "
+                f"Specificity: {d['specificity']}/5  \n{d['reason']}"
+            )
+            st.divider()
+
+    # Module D
+    dmeta = DIMENSION_META["attribution"]
+    with st.expander(f"Module D — Attribution & Grounding  ·  {dim_scores['attribution']}/100"):
+        sens = dmeta["model_sensitivity"]
+        sc = SENSITIVITY_COLOR[sens]
+        st.markdown(
+            f"<span class='sens-badge' style='background:{sc}22;color:{sc};border:1px solid {sc}55;'>"
+            f"Model sensitivity: {sens}</span>",
+            unsafe_allow_html=True,
+        )
+        st.markdown(f"**What this measures:** {dmeta['description']}")
+        st.markdown(f"**How it's measured:** {dmeta['how_measured']}")
+        st.markdown(f"**Models used:** {', '.join(dmeta['models_used'])}")
+        st.info(dmeta["sensitivity_note"])
+        st.divider()
+        st.markdown(f"**Result:** {module_results['attribution']['summary']}")
+        for d in module_results["attribution"].get("details", []):
+            halluc = "⚠️ Hallucination detected" if d.get("hallucination_detected") else "✓ No hallucination"
+            st.markdown(
+                f"**Q: {d['question'][:110]}**  \n"
+                f"Grounding: **{round(d['grounding_fraction']*100)}%** &nbsp;|&nbsp; "
+                f"Semantic similarity: {d['semantic_similarity']} &nbsp;|&nbsp; {halluc}"
+            )
+            if d.get("ungrounded_claims"):
+                st.caption("Ungrounded claims: " + "; ".join(d["ungrounded_claims"][:3]))
+            if d.get("reason"):
+                st.caption(d["reason"])
+
+    # Module E
+    dmeta = DIMENSION_META["demand"]
+    with st.expander(f"Module E — Demand Utility  ·  {dim_scores['demand']}/100"):
+        sens = dmeta["model_sensitivity"]
+        sc = SENSITIVITY_COLOR[sens]
+        st.markdown(
+            f"<span class='sens-badge' style='background:{sc}22;color:{sc};border:1px solid {sc}55;'>"
+            f"Model sensitivity: {sens}</span>",
+            unsafe_allow_html=True,
+        )
+        st.markdown(f"**What this measures:** {dmeta['description']}")
+        st.markdown(f"**How it's measured:** {dmeta['how_measured']}")
+        st.markdown(f"**Models used:** {', '.join(dmeta['models_used'])}")
+        st.info(dmeta["sensitivity_note"])
+        st.divider()
+        st.markdown(f"**Result:** {module_results['demand']['summary']}")
+        topics = module_results["demand"].get("topics", [])
+        if topics:
+            st.caption(
+                "**Query Freq** — estimated user query frequency for this topic (1 = rare, 10 = very common).  \n"
+                "**Priority Domain** — whether this is a high-impact sector (health, climate, food, policy, etc.).  \n"
+                "**Unmet Need** — whether existing AI models fall short in covering this topic."
+            )
+            st.table({
+                "Topic":             [t.get("topic", "") for t in topics],
+                "Query Freq (1-10)": [t.get("query_frequency", "-") for t in topics],
+                "Priority Domain":   ["Yes" if t.get("priority_domain") else "No" for t in topics],
+                "Unmet Need":        ["Yes" if t.get("unmet_need") else "No" for t in topics],
+                "Rationale":         [t.get("rationale", "")[:80] for t in topics],
+            })
+
+    # ── Recommendations ───────────────────────────────────────────────────────
+    st.markdown("### Recommended Actions")
+    for rec in kvs_result["recommendations"]:
+        st.markdown(f"- {rec}")
+
+    # ── Download ──────────────────────────────────────────────────────────────
+    st.divider()
+    report_md = report.generate(doc.title, kvs_result, module_results, meta)
+    st.download_button(
+        label="⬇  Download Full Report (Markdown)",
+        data=report_md,
+        file_name=f"kvl_report_{doc.title[:40].replace(' ', '_')}.md",
+        mime="text/markdown",
+        use_container_width=True,
+    )
+
+elif not uploaded:
+    st.info("Upload a `.md` file above to begin evaluation.")

kvl/config.py

@@ -0,0 +1,184 @@
+"""Centralized model configuration and dimension metadata for KVL."""
+
+from __future__ import annotations
+from datetime import datetime
+
+# ── Model identifiers ─────────────────────────────────────────────────────────
+
+MODELS = {
+    "judge": {
+        "id":      "claude-sonnet-4-6",
+        "display": "Claude Sonnet 4.6",
+        "role":    "Evaluation judge — novelty scoring, grounding assessment, demand analysis",
+        "modules": ["A", "D", "E"],
+    },
+    "worker": {
+        "id":      "claude-haiku-4-5-20251001",
+        "display": "Claude Haiku 4.5",
+        "role":    "Closed-book QA, baseline & RAG answer generation, query generation",
+        "modules": ["A", "B", "C"],
+    },
+    "embedder": {
+        "id":      "sentence-transformers/all-MiniLM-L6-v2",
+        "display": "all-MiniLM-L6-v2",
+        "role":    "Text embeddings for retrieval index and semantic similarity",
+        "modules": ["B", "D"],
+    },
+}
+
+
+def model_meta(eval_date: str | None = None) -> dict:
+    return {
+        "judge":    MODELS["judge"],
+        "worker":   MODELS["worker"],
+        "embedder": MODELS["embedder"],
+        "eval_date": eval_date or datetime.now().strftime("%Y-%m-%d %H:%M UTC"),
+        "framework_version": "KVL v0.1",
+    }
+
+
+# ── Per-dimension metadata ────────────────────────────────────────────────────
+
+DIMENSION_META = {
+    "novelty": {
+        "name":              "Knowledge Novelty",
+        "weight":            0.30,
+        "model_sensitivity": "High",
+        "sensitivity_note":  (
+            "Directly tied to the specific model evaluated. A document scoring 96/100 "
+            "against Claude Sonnet may score significantly lower against a model trained "
+            "on domain-specific corpora. Re-evaluate when the underlying model is updated."
+        ),
+        "description": (
+            "Measures how much of the document's content is already embedded in the AI "
+            "model's pre-training. A high novelty score means the model cannot reliably "
+            "answer questions about this document from its training alone — the document "
+            "contributes genuinely new knowledge to the system."
+        ),
+        "how_measured": (
+            "12–15 specific factual claims are extracted from the document. For each claim, "
+            "a question is generated and posed to the model **without** the document "
+            "(closed-book). A judge model scores how well the model's answer matches the "
+            "document's claim. **Novelty = 100 × (1 − fraction already known)**."
+        ),
+        "high_means": "Document contains knowledge rare or absent in the model's training data.",
+        "low_means":  "Document content is widely represented in the model's training corpus.",
+        "models_used": ["Claude Haiku 4.5 (closed-book QA)", "Claude Sonnet 4.6 (judge)"],
+    },
+    "retrieval": {
+        "name":              "Retrieval Utility",
+        "weight":            0.20,
+        "model_sensitivity": "Low",
+        "sensitivity_note":  (
+            "The most model-independent dimension. Embeddings use a fixed local model "
+            "(all-MiniLM-L6-v2); only test-query generation involves the LLM. "
+            "Scores are stable across LLM versions."
+        ),
+        "description": (
+            "Measures how effectively the document surfaces in response to relevant user "
+            "queries in a retrieval-augmented system. High retrieval utility means the "
+            "document's content consistently appears in top search results when users "
+            "ask about its topics."
+        ),
+        "how_measured": (
+            "The document is split into ~400-word chunks and embedded using "
+            "sentence-transformers. 8 representative queries are generated from the document, "
+            "and for each query the system measures whether the most relevant chunk appears "
+            "in the top-3 results. **Score = weighted Recall@3 (60%) + MRR (40%)**."
+        ),
+        "high_means": "Well-structured content that maps cleanly to user search queries.",
+        "low_means":  "Poor chunk boundaries, dense prose, or missing structure — can be improved through document restructuring.",
+        "models_used": ["all-MiniLM-L6-v2 (embeddings)", "Claude Haiku 4.5 (query generation)"],
+    },
+    "generation": {
+        "name":              "Generation Utility",
+        "weight":            0.25,
+        "model_sensitivity": "High",
+        "sensitivity_note":  (
+            "Highly model-relative. A model with stronger prior knowledge will show less "
+            "improvement from the document, yielding a lower generation utility score "
+            "even if the document is objectively high quality. Re-evaluate when the model changes."
+        ),
+        "description": (
+            "Measures the uplift in AI-generated answer quality when the document is "
+            "provided as context (RAG) versus answering from the model's prior knowledge "
+            "alone. This directly quantifies how much the document improves what the "
+            "AI system can tell users."
+        ),
+        "how_measured": (
+            "8 questions are derived from the document. Each is answered twice: once "
+            "**without** the document (baseline) and once **with** the document as "
+            "context (RAG). A judge model scores the improvement across accuracy, "
+            "completeness, and specificity on a 0–100 scale."
+        ),
+        "high_means": "Document substantially improves AI answers — essential for advisory and QA applications.",
+        "low_means":  "Model can already answer adequately without the document.",
+        "models_used": ["Claude Haiku 4.5 (answer generation)", "Claude Sonnet 4.6 (judge)"],
+    },
+    "attribution": {
+        "name":              "Attribution & Grounding",
+        "weight":            0.15,
+        "model_sensitivity": "Moderate",
+        "sensitivity_note":  (
+            "The LLM judge's grounding assessment varies across models. The semantic "
+            "similarity component (sentence-transformers) is model-independent, providing "
+            "a stable floor for the score."
+        ),
+        "description": (
+            "Measures whether AI-generated answers are genuinely grounded in the document, "
+            "rather than mixing in unverified claims or hallucinations. High grounding means "
+            "the document's evidence is being faithfully used and answers are trustworthy "
+            "for deployment in high-stakes domains."
+        ),
+        "how_measured": (
+            "RAG answers from the Generation module are analysed by a judge model to "
+            "identify which claims are traceable to specific document passages. Semantic "
+            "similarity between the answer and document context provides a second signal. "
+            "A score penalty is applied if hallucination is detected. "
+            "**Score = 70% × LLM grounding fraction + 30% × semantic similarity − hallucination penalty**."
+        ),
+        "high_means": "Answers closely follow document — high trustworthiness for regulated or high-stakes deployment.",
+        "low_means":  "Answers stray from document content — document may lack specificity or model over-relies on prior knowledge.",
+        "models_used": ["Claude Sonnet 4.6 (grounding judge)", "all-MiniLM-L6-v2 (semantic similarity)"],
+    },
+    "demand": {
+        "name":              "Demand Utility",
+        "weight":            0.10,
+        "model_sensitivity": "Low",
+        "sensitivity_note":  (
+            "Primarily reflects the document's subject matter rather than the model's "
+            "knowledge state. Scores are relatively stable across LLM versions, though "
+            "different models may assess domain importance differently."
+        ),
+        "description": (
+            "Estimates how frequently the knowledge in this document would be needed by "
+            "real users — considering topic popularity, domain priority (healthcare, climate, "
+            "agriculture, policy), geographic relevance, and coverage of unmet information needs."
+        ),
+        "how_measured": (
+            "5–8 topics are extracted from the document. For each, the model estimates "
+            "query frequency (1–10), whether the domain is a priority area, and whether "
+            "significant unmet information needs exist. A holistic demand score from the "
+            "judge is blended with the topic-level estimates "
+            "(40% topic average + 60% holistic score)."
+        ),
+        "high_means": "Broad audience or urgent unmet need — wide deployment and open access recommended.",
+        "low_means":  "Narrow or specialised audience — targeted distribution channels are more appropriate.",
+        "models_used": ["Claude Haiku 4.5 (topic extraction)", "Claude Sonnet 4.6 (demand assessment)"],
+    },
+}
+
+
+SENSITIVITY_COLOR = {
+    "High":     "#ef476f",
+    "Moderate": "#f8961e",
+    "Low":      "#06d6a0",
+}
+
+KVS_CLASSIFICATION = [
+    (81, "Transformational Value", "Document adds exceptional, rare knowledge and dramatically improves AI system performance."),
+    (61, "High Value",             "Document adds significant novel knowledge with strong retrieval and generation impact."),
+    (41, "Moderate Value",         "Document offers meaningful contributions in at least some dimensions."),
+    (21, "Incremental Value",      "Document provides marginal improvements; content may be partially redundant."),
+    (0,  "Minimal Value",          "Little or no measurable improvement to AI systems from this document."),
+]

kvl/ingestor.py

@@ -0,0 +1,80 @@
+"""Parse and chunk Markdown documents for KVL evaluation."""
+
+from __future__ import annotations
+import re
+from dataclasses import dataclass, field
+
+
+@dataclass
+class Chunk:
+    text: str
+    index: int
+    section: str = ""
+
+
+@dataclass
+class Document:
+    raw: str
+    title: str
+    sections: list[dict]       # [{"heading": str, "body": str}]
+    chunks: list[Chunk]
+    word_count: int
+    char_count: int
+
+
+def _extract_title(text: str) -> str:
+    for line in text.splitlines():
+        line = line.strip()
+        if line.startswith("# "):
+            return line[2:].strip()
+    return "Untitled Document"
+
+
+def _split_sections(text: str) -> list[dict]:
+    """Split on any heading level, preserving body text."""
+    pattern = re.compile(r"^(#{1,6})\s+(.+)$", re.MULTILINE)
+    matches = list(pattern.finditer(text))
+    sections = []
+    for i, m in enumerate(matches):
+        heading = m.group(2).strip()
+        start = m.end()
+        end = matches[i + 1].start() if i + 1 < len(matches) else len(text)
+        body = text[start:end].strip()
+        if body:
+            sections.append({"heading": heading, "body": body})
+    if not sections:
+        sections.append({"heading": "Content", "body": text.strip()})
+    return sections
+
+
+def _chunk_text(sections: list[dict], chunk_size: int = 400, overlap: int = 50) -> list[Chunk]:
+    """Sliding-window word-based chunking over section bodies."""
+    chunks: list[Chunk] = []
+    idx = 0
+    for sec in sections:
+        words = sec["body"].split()
+        pos = 0
+        while pos < len(words):
+            window = words[pos: pos + chunk_size]
+            text = " ".join(window)
+            chunks.append(Chunk(text=text, index=idx, section=sec["heading"]))
+            idx += 1
+            if pos + chunk_size >= len(words):
+                break
+            pos += chunk_size - overlap
+    return chunks
+
+
+def parse(md_text: str) -> Document:
+    text = md_text.strip()
+    title = _extract_title(text)
+    sections = _split_sections(text)
+    chunks = _chunk_text(sections)
+    return Document(
+        raw=text,
+        title=title,
+        sections=sections,
+        chunks=chunks,
+        word_count=len(text.split()),
+        char_count=len(text),
+    )

kvl/modules/attribution.py

@@ -0,0 +1,109 @@
+"""Module D: Attribution & Grounding — verifies that RAG answers are grounded in the document."""
+
+from __future__ import annotations
+import json
+import numpy as np
+import anthropic
+from kvl.ingestor import Document
+
+_GROUNDING_PROMPT = """Analyze whether an AI-generated answer is properly grounded in the provided source document.
+
+Source document excerpt:
+{context}
+
+AI-generated answer:
+{answer}
+
+Evaluate grounding quality:
+1. What fraction of specific claims in the answer can be traced to the source document? (0.0-1.0)
+2. Does the answer avoid hallucinating facts not in the document?
+3. Are the answer's assertions supported by evidence in the document?
+
+Return ONLY JSON:
+{{
+  "grounding_fraction": <float 0-1>,
+  "hallucination_detected": <bool>,
+  "grounded_claims": ["list of claims that are in the document"],
+  "ungrounded_claims": ["list of claims NOT found in the document"],
+  "reason": "one sentence summary"
+}}"""
+
+
+def _call_claude(client: anthropic.Anthropic, prompt: str) -> str:
+    msg = client.messages.create(
+        model="claude-sonnet-4-6",
+        max_tokens=1024,
+        messages=[{"role": "user", "content": prompt}],
+        system="You are an expert fact-checker assessing source attribution in AI-generated text.",
+    )
+    return msg.content[0].text.strip()
+
+
+def _semantic_overlap(answer: str, context: str, embedder) -> float:
+    """Cosine similarity between answer and document context as a grounding signal."""
+    embs = embedder.encode([answer, context], normalize_embeddings=True, show_progress_bar=False)
+    return float(np.dot(embs[0], embs[1]))
+
+
+def evaluate(client: anthropic.Anthropic, doc: Document, generation_results: dict, embedder, progress_cb=None) -> dict:
+    """Return grounding score (0-100) using outputs from the generation module."""
+    details_list = generation_results.get("details", [])
+    if not details_list:
+        return {"score": 50, "details": [], "summary": "No generation results to assess grounding."}
+
+    context = " ".join(doc.raw.split()[:4000])
+    results = []
+    grounding_scores = []
+
+    for i, item in enumerate(details_list):
+        if progress_cb:
+            progress_cb(f"Checking grounding for answer {i+1}/{len(details_list)}...")
+
+        rag_answer = item.get("rag_answer", "")
+        if not rag_answer:
+            continue
+
+        raw = _call_claude(client, _GROUNDING_PROMPT.format(context=context, answer=rag_answer))
+        raw = raw.strip()
+        if raw.startswith("```"):
+            raw = "\n".join(raw.split("\n")[1:])
+            raw = raw.rsplit("```", 1)[0]
+
+        try:
+            judgment = json.loads(raw)
+        except json.JSONDecodeError:
+            judgment = {"grounding_fraction": 0.5, "hallucination_detected": False, "reason": "Parse error."}
+
+        # Combine LLM grounding fraction with semantic overlap
+        llm_grounding = judgment.get("grounding_fraction", 0.5)
+        semantic_sim = _semantic_overlap(rag_answer, context, embedder)
+        # Penalise if hallucination detected
+        hallucination_penalty = 0.2 if judgment.get("hallucination_detected", False) else 0.0
+        combined = (0.7 * llm_grounding + 0.3 * semantic_sim) - hallucination_penalty
+        combined = max(0.0, min(1.0, combined))
+
+        grounding_scores.append(combined)
+        results.append({
+            "question": item.get("question", ""),
+            "answer": rag_answer,
+            "grounding_fraction": llm_grounding,
+            "semantic_similarity": round(semantic_sim, 3),
+            "hallucination_detected": judgment.get("hallucination_detected", False),
+            "grounded_claims": judgment.get("grounded_claims", []),
+            "ungrounded_claims": judgment.get("ungrounded_claims", []),
+            "reason": judgment.get("reason", ""),
+            "combined_score": round(combined, 3),
+        })
+
+    if not grounding_scores:
+        return {"score": 50, "details": results, "summary": "No grounding assessments completed."}
+
+    avg_grounding = sum(grounding_scores) / len(grounding_scores)
+    score = round(avg_grounding * 100)
+
+    hallucinations = sum(1 for r in results if r.get("hallucination_detected"))
+    return {
+        "score": score,
+        "details": results,
+        "summary": f"Average grounding: {score}/100. Hallucinations detected in {hallucinations}/{len(results)} answers.",
+    }

kvl/modules/demand.py

@@ -0,0 +1,124 @@
+"""Module E: Demand Utility — estimates how frequently this knowledge is needed by users."""
+
+from __future__ import annotations
+import json
+import anthropic
+from kvl.ingestor import Document
+
+_TOPIC_PROMPT = """Analyze this document and identify its main topics and themes.
+
+For each topic, estimate:
+1. How frequently real users would search for or need this information (1-10 scale)
+2. Whether this is a high-priority domain (healthcare, climate, food security, education, policy, etc.)
+3. Whether there are significant unmet information needs in this area
+
+Return ONLY a JSON array of objects:
+[
+  {{
+    "topic": "topic name",
+    "description": "what aspect of the document this covers",
+    "query_frequency": <int 1-10>,
+    "priority_domain": <bool>,
+    "unmet_need": <bool>,
+    "rationale": "one sentence"
+  }},
+  ...
+]
+
+Extract 5-8 topics.
+
+Document:
+{document}"""
+
+_DEMAND_SUMMARY_PROMPT = """Based on this knowledge document's topics and characteristics, assess its overall demand utility.
+
+Topics identified:
+{topics}
+
+Document summary:
+{summary}
+
+Evaluate:
+1. How broad vs. specialized is the audience for this knowledge?
+2. Are there gaps in existing AI model knowledge that this document fills?
+3. What is the geographic or linguistic scope?
+4. How actionable is this knowledge for typical users?
+
+Return ONLY JSON:
+{{
+  "audience_breadth": <int 1-10>,
+  "knowledge_gap_fill": <int 1-10>,
+  "geographic_relevance": <int 1-10>,
+  "actionability": <int 1-10>,
+  "demand_score": <int 0-100>,
+  "summary": "two sentence demand assessment"
+}}"""
+
+
+def _call_claude(client: anthropic.Anthropic, prompt: str, model: str = "claude-sonnet-4-6") -> str:
+    msg = client.messages.create(
+        model=model,
+        max_tokens=1024,
+        messages=[{"role": "user", "content": prompt}],
+        system="You are an expert in knowledge management and information demand analysis.",
+    )
+    return msg.content[0].text.strip()
+
+
+def _parse_json(raw: str) -> dict | list | None:
+    raw = raw.strip()
+    if raw.startswith("```"):
+        raw = "\n".join(raw.split("\n")[1:])
+        raw = raw.rsplit("```", 1)[0]
+    try:
+        return json.loads(raw)
+    except json.JSONDecodeError:
+        return None
+
+
+def evaluate(client: anthropic.Anthropic, doc: Document, progress_cb=None) -> dict:
+    """Return demand utility score (0-100) and topic analysis."""
+    if progress_cb:
+        progress_cb("Extracting document topics and themes...")
+
+    text = " ".join(doc.raw.split()[:5000])
+    raw_topics = _call_claude(client, _TOPIC_PROMPT.format(document=text), model="claude-haiku-4-5-20251001")
+    topics = _parse_json(raw_topics)
+
+    if not topics or not isinstance(topics, list):
+        return {"score": 50, "topics": [], "summary": "Could not extract topics from document."}
+
+    # Compute a base score from topic frequency ratings
+    avg_frequency = sum(t.get("query_frequency", 5) for t in topics) / len(topics)
+    priority_bonus = sum(1 for t in topics if t.get("priority_domain", False)) / len(topics)
+    unmet_bonus = sum(1 for t in topics if t.get("unmet_need", False)) / len(topics)
+
+    # Base score from topic analysis
+    base_score = (avg_frequency / 10) * 60 + priority_bonus * 20 + unmet_bonus * 20
+
+    if progress_cb:
+        progress_cb("Estimating overall demand utility...")
+
+    # Get holistic demand assessment from Claude
+    doc_summary = f"Title: {doc.title}. Words: {doc.word_count}. Sections: {len(doc.sections)}."
+    topics_str = json.dumps(topics, indent=2)
+    raw_demand = _call_claude(client, _DEMAND_SUMMARY_PROMPT.format(topics=topics_str, summary=doc_summary))
+    demand_assessment = _parse_json(raw_demand)
+
+    if demand_assessment and isinstance(demand_assessment, dict):
+        llm_score = demand_assessment.get("demand_score", base_score)
+        # Blend base score (40%) with LLM holistic score (60%)
+        final_score = round(0.4 * base_score + 0.6 * llm_score)
+        assessment_summary = demand_assessment.get("summary", "")
+    else:
+        final_score = round(base_score)
+        assessment_summary = ""
+
+    final_score = max(0, min(100, final_score))
+
+    return {
+        "score": final_score,
+        "topics": topics,
+        "assessment": demand_assessment or {},
+        "summary": assessment_summary or f"Demand estimated from {len(topics)} topic areas. Base score: {round(base_score)}/100.",
+    }

kvl/modules/generation.py

@@ -0,0 +1,143 @@
+"""Module C: Generation Utility — measures improvement in AI responses when using the document."""
+
+from __future__ import annotations
+import json
+import anthropic
+from kvl.ingestor import Document
+
+_QA_GEN_PROMPT = """Generate {n} specific questions whose answers can ONLY be found in this document (not general knowledge).
+Focus on questions about specific findings, methods, results, or recommendations in the document.
+
+Return ONLY a JSON array of question strings.
+
+Document:
+{document}"""
+
+_JUDGE_PROMPT = """Compare two AI-generated answers to the same question. One is generated without any document context (baseline), the other with a relevant document as context (RAG).
+
+Question: {question}
+
+Baseline answer (no document): {baseline}
+
+RAG answer (with document context): {rag_answer}
+
+Rate the RAG answer improvement over baseline on these criteria (1-5 each):
+- accuracy: Is the RAG answer more factually correct?
+- completeness: Does the RAG answer cover more relevant aspects?
+- specificity: Is the RAG answer more specific and precise?
+
+Also provide an overall improvement score (0-100) where:
+- 0 = RAG answer is no better or worse than baseline
+- 50 = moderate improvement
+- 100 = dramatically better, document was essential
+
+Return ONLY JSON: {{"accuracy": int, "completeness": int, "specificity": int, "improvement": int, "reason": "one sentence"}}"""
+
+
+def _call_claude(client: anthropic.Anthropic, system: str, user: str, model: str = "claude-sonnet-4-6") -> str:
+    msg = client.messages.create(
+        model=model,
+        max_tokens=1024,
+        messages=[{"role": "user", "content": user}],
+        system=system,
+    )
+    return msg.content[0].text.strip()
+
+
+def _generate_questions(client: anthropic.Anthropic, doc: Document, n: int = 8) -> list[str]:
+    text = " ".join(doc.raw.split()[:5000])
+    raw = _call_claude(
+        client,
+        "You generate evaluation questions for knowledge documents.",
+        _QA_GEN_PROMPT.format(n=n, document=text),
+        model="claude-haiku-4-5-20251001",
+    )
+    raw = raw.strip()
+    if raw.startswith("```"):
+        raw = "\n".join(raw.split("\n")[1:])
+        raw = raw.rsplit("```", 1)[0]
+    try:
+        questions = json.loads(raw)
+        return [q for q in questions if isinstance(q, str)][:n]
+    except json.JSONDecodeError:
+        return []
+
+
+def _baseline_answer(client: anthropic.Anthropic, question: str) -> str:
+    return _call_claude(
+        client,
+        "Answer based only on your pre-trained knowledge. Be honest if you don't know.",
+        question,
+        model="claude-haiku-4-5-20251001",
+    )
+
+
+def _rag_answer(client: anthropic.Anthropic, question: str, context: str) -> str:
+    return _call_claude(
+        client,
+        "Answer the question using the provided document context. Be specific and cite details from the document.",
+        f"Document context:\n{context}\n\nQuestion: {question}",
+        model="claude-haiku-4-5-20251001",
+    )
+
+
+def _judge(client: anthropic.Anthropic, question: str, baseline: str, rag: str) -> dict:
+    raw = _call_claude(
+        client,
+        "You are an expert evaluator comparing AI answer quality.",
+        _JUDGE_PROMPT.format(question=question, baseline=baseline, rag_answer=rag),
+    )
+    raw = raw.strip()
+    if raw.startswith("```"):
+        raw = "\n".join(raw.split("\n")[1:])
+        raw = raw.rsplit("```", 1)[0]
+    try:
+        return json.loads(raw)
+    except json.JSONDecodeError:
+        return {"accuracy": 3, "completeness": 3, "specificity": 3, "improvement": 50, "reason": "Parse error."}
+
+
+def evaluate(client: anthropic.Anthropic, doc: Document, progress_cb=None) -> dict:
+    """Return generation utility score (0-100) and per-question details."""
+    if progress_cb:
+        progress_cb("Generating evaluation questions...")
+
+    questions = _generate_questions(client, doc)
+    if not questions:
+        return {"score": 50, "details": [], "summary": "Could not generate test questions."}
+
+    # Use the full document as RAG context (capped for token limits)
+    context = " ".join(doc.raw.split()[:4000])
+
+    results = []
+    improvement_scores = []
+
+    for i, question in enumerate(questions):
+        if progress_cb:
+            progress_cb(f"Evaluating question {i+1}/{len(questions)}: baseline vs RAG...")
+
+        baseline = _baseline_answer(client, question)
+        rag = _rag_answer(client, question, context)
+        judgment = _judge(client, question, baseline, rag)
+
+        improvement = judgment.get("improvement", 50)
+        improvement_scores.append(improvement)
+        results.append({
+            "question": question,
+            "baseline_answer": baseline,
+            "rag_answer": rag,
+            "accuracy": judgment.get("accuracy", 3),
+            "completeness": judgment.get("completeness", 3),
+            "specificity": judgment.get("specificity", 3),
+            "improvement": improvement,
+            "reason": judgment.get("reason", ""),
+        })
+
+    avg_improvement = sum(improvement_scores) / len(improvement_scores)
+    score = round(avg_improvement)
+
+    return {
+        "score": score,
+        "details": results,
+        "summary": f"Average RAG improvement: {score}/100 across {len(questions)} questions.",
+    }

kvl/modules/novelty.py

@@ -0,0 +1,122 @@
+"""Module A: Knowledge Novelty — measures how much of the document Claude already knows."""
+
+from __future__ import annotations
+import json
+import anthropic
+from kvl.ingestor import Document
+
+_CLAIM_PROMPT = """You are analyzing a document to evaluate how novel its knowledge is to AI language models.
+
+Extract exactly {n} specific, verifiable factual claims from the document below. Focus on:
+- Precise facts, findings, statistics, or assertions
+- Claims that are specific (not general background knowledge)
+- Claims that could be tested with a yes/no or short answer question
+
+Return ONLY a JSON array of objects with keys "claim" and "question", where "question" is the specific question whose answer is the claim.
+
+Example format:
+[
+  {{"claim": "The study found a 34% reduction in soil erosion.", "question": "What reduction in soil erosion did the study find?"}},
+  ...
+]
+
+Document:
+{document}"""
+
+_JUDGE_PROMPT = """You are evaluating whether an AI model's answer demonstrates prior knowledge of a specific claim.
+
+Claim from document: {claim}
+AI model's closed-book answer: {answer}
+
+Rate how well the model's answer already captures the claim WITHOUT having seen the document.
+Score 0-1 where:
+- 1.0 = model's answer is accurate and complete — this is not novel knowledge
+- 0.5 = model has partial or approximate knowledge
+- 0.0 = model doesn't know this — this IS novel knowledge
+
+Return ONLY a JSON object: {{"score": <float 0-1>, "reason": "<one sentence>"}}"""
+
+
+def _call_claude(client: anthropic.Anthropic, system: str, user: str, model: str = "claude-sonnet-4-6") -> str:
+    msg = client.messages.create(
+        model=model,
+        max_tokens=2048,
+        messages=[{"role": "user", "content": user}],
+        system=system,
+    )
+    return msg.content[0].text.strip()
+
+
+def _extract_claims(client: anthropic.Anthropic, doc: Document, n: int = 12) -> list[dict]:
+    # Use the full document text but cap at ~6000 words to stay within limits
+    text = " ".join(doc.raw.split()[:6000])
+    prompt = _CLAIM_PROMPT.format(n=n, document=text)
+    raw = _call_claude(client, "You extract factual claims from documents.", prompt)
+    # Strip markdown code fences if present
+    raw = raw.strip()
+    if raw.startswith("```"):
+        raw = "\n".join(raw.split("\n")[1:])
+        raw = raw.rsplit("```", 1)[0]
+    try:
+        claims = json.loads(raw)
+        return claims[:n]
+    except json.JSONDecodeError:
+        return []
+
+
+def _closed_book_answer(client: anthropic.Anthropic, question: str) -> str:
+    return _call_claude(
+        client,
+        "Answer the question using only your pre-trained knowledge. Do not make up information. If unsure, say so.",
+        question,
+        model="claude-haiku-4-5-20251001",  # cheaper for bulk closed-book queries
+    )
+
+
+def _judge_novelty(client: anthropic.Anthropic, claim: str, answer: str) -> dict:
+    raw = _call_claude(
+        client,
+        "You are an expert evaluator assessing AI knowledge coverage.",
+        _JUDGE_PROMPT.format(claim=claim, answer=answer),
+    )
+    raw = raw.strip()
+    if raw.startswith("```"):
+        raw = "\n".join(raw.split("\n")[1:])
+        raw = raw.rsplit("```", 1)[0]
+    try:
+        return json.loads(raw)
+    except json.JSONDecodeError:
+        return {"score": 0.5, "reason": "Could not parse judge response."}
+
+
+def evaluate(client: anthropic.Anthropic, doc: Document, progress_cb=None) -> dict:
+    """Return novelty score (0-100) and detailed results."""
+    if progress_cb:
+        progress_cb("Extracting factual claims from document...")
+
+    claims = _extract_claims(client, doc)
+    if not claims:
+        return {"score": 50, "details": [], "summary": "Could not extract claims from document."}
+
+    results = []
+    for i, item in enumerate(claims):
+        if progress_cb:
+            progress_cb(f"Testing claim {i+1}/{len(claims)} against model knowledge...")
+        answer = _closed_book_answer(client, item["question"])
+        judgment = _judge_novelty(client, item["claim"], answer)
+        results.append({
+            "claim": item["claim"],
+            "question": item["question"],
+            "model_answer": answer,
+            "known_score": judgment["score"],
+            "reason": judgment.get("reason", ""),
+        })
+
+    avg_known = sum(r["known_score"] for r in results) / len(results)
+    novelty_score = round((1 - avg_known) * 100)
+
+    return {
+        "score": novelty_score,
+        "details": results,
+        "summary": f"Tested {len(results)} claims. Model already knows ~{round(avg_known*100)}% of this knowledge.",
+    }

kvl/modules/retrieval.py

@@ -0,0 +1,132 @@
+"""Module B: Retrieval Utility — measures how well the document supports information retrieval."""
+
+from __future__ import annotations
+import json
+import numpy as np
+import anthropic
+from kvl.ingestor import Document
+
+_QUERY_PROMPT = """Generate {n} realistic search queries that a user would type to find information in this document.
+Queries should be specific, not generic. Mix question and keyword styles.
+
+Return ONLY a JSON array of strings.
+
+Example: ["What is the impact of X on Y?", "methodology for measuring Z", ...]
+
+Document excerpt:
+{excerpt}"""
+
+
+def _call_claude(client: anthropic.Anthropic, prompt: str) -> str:
+    msg = client.messages.create(
+        model="claude-haiku-4-5-20251001",
+        max_tokens=1024,
+        messages=[{"role": "user", "content": prompt}],
+    )
+    return msg.content[0].text.strip()
+
+
+def _generate_queries(client: anthropic.Anthropic, doc: Document, n: int = 8) -> list[str]:
+    excerpt = " ".join(doc.raw.split()[:3000])
+    raw = _call_claude(client, _QUERY_PROMPT.format(n=n, excerpt=excerpt))
+    raw = raw.strip()
+    if raw.startswith("```"):
+        raw = "\n".join(raw.split("\n")[1:])
+        raw = raw.rsplit("```", 1)[0]
+    try:
+        queries = json.loads(raw)
+        return [q for q in queries if isinstance(q, str)][:n]
+    except json.JSONDecodeError:
+        return []
+
+
+def _build_index(chunks: list, embedder) -> tuple:
+    """Embed all chunks and build a FAISS flat index."""
+    import faiss
+    texts = [c.text for c in chunks]
+    embeddings = embedder.encode(texts, normalize_embeddings=True, show_progress_bar=False)
+    embeddings = np.array(embeddings, dtype="float32")
+    dim = embeddings.shape[1]
+    index = faiss.IndexFlatIP(dim)  # inner product = cosine when normalized
+    index.add(embeddings)
+    return index, embeddings
+
+
+def _retrieve(query: str, index, embedder, k: int = 3) -> list[int]:
+    """Return top-k chunk indices for a query."""
+    q_emb = embedder.encode([query], normalize_embeddings=True, show_progress_bar=False)
+    q_emb = np.array(q_emb, dtype="float32")
+    _, indices = index.search(q_emb, k)
+    return indices[0].tolist()
+
+
+def _recall_at_k(retrieved: list[int], relevant: int) -> float:
+    return 1.0 if relevant in retrieved else 0.0
+
+
+def _reciprocal_rank(retrieved: list[int], relevant: int) -> float:
+    for rank, idx in enumerate(retrieved, start=1):
+        if idx == relevant:
+            return 1.0 / rank
+    return 0.0
+
+
+def evaluate(client: anthropic.Anthropic, doc: Document, embedder, progress_cb=None) -> dict:
+    """Return retrieval utility score (0-100) and detailed results."""
+    if not doc.chunks:
+        return {"score": 0, "details": [], "summary": "No chunks to index."}
+
+    if progress_cb:
+        progress_cb("Building retrieval index...")
+
+    index, _ = _build_index(doc.chunks, embedder)
+
+    if progress_cb:
+        progress_cb("Generating retrieval test queries...")
+
+    queries = _generate_queries(client, doc)
+    if not queries:
+        return {"score": 50, "details": [], "summary": "Could not generate test queries."}
+
+    results = []
+    recall_scores = []
+    mrr_scores = []
+
+    for i, query in enumerate(queries):
+        if progress_cb:
+            progress_cb(f"Evaluating retrieval query {i+1}/{len(queries)}...")
+
+        retrieved_indices = _retrieve(query, index, embedder, k=3)
+
+        # Find the most semantically relevant chunk by re-ranking with cosine sim
+        q_emb = embedder.encode([query], normalize_embeddings=True, show_progress_bar=False)
+        q_emb = np.array(q_emb, dtype="float32")
+        chunk_texts = [c.text for c in doc.chunks]
+        chunk_embs = embedder.encode(chunk_texts, normalize_embeddings=True, show_progress_bar=False)
+        sims = np.dot(chunk_embs, q_emb.T).flatten()
+        best_chunk = int(np.argmax(sims))
+
+        r_at_3 = _recall_at_k(retrieved_indices, best_chunk)
+        rr = _reciprocal_rank(retrieved_indices, best_chunk)
+        recall_scores.append(r_at_3)
+        mrr_scores.append(rr)
+
+        results.append({
+            "query": query,
+            "retrieved_chunks": retrieved_indices,
+            "best_chunk": best_chunk,
+            "recall_at_3": r_at_3,
+            "reciprocal_rank": rr,
+        })
+
+    avg_recall = sum(recall_scores) / len(recall_scores)
+    avg_mrr = sum(mrr_scores) / len(mrr_scores)
+    # Weighted combination: recall@3 (60%) + MRR (40%), mapped to 0-100
+    raw_score = 0.6 * avg_recall + 0.4 * avg_mrr
+    score = round(raw_score * 100)
+
+    return {
+        "score": score,
+        "details": results,
+        "summary": f"Recall@3: {avg_recall:.2f} | MRR: {avg_mrr:.2f} across {len(queries)} queries.",
+    }

kvl/report.py

@@ -0,0 +1,213 @@
+"""Generate a Markdown Knowledge Value Report from evaluation results."""
+
+from __future__ import annotations
+from datetime import datetime
+from kvl.config import DIMENSION_META, KVS_CLASSIFICATION, MODELS
+
+
+def _bar(score: int, width: int = 20) -> str:
+    filled = round(score / 100 * width)
+    return "█" * filled + "░" * (width - filled)
+
+
+def _sensitivity_badge(level: str) -> str:
+    icons = {"High": "🔴", "Moderate": "🟡", "Low": "🟢"}
+    return f"{icons.get(level, '')} {level}"
+
+
+def generate(doc_title: str, kvs_result: dict, module_results: dict, meta: dict | None = None) -> str:
+    now = datetime.now().strftime("%Y-%m-%d %H:%M")
+    kvs = kvs_result["kvs"]
+    classification = kvs_result["classification"]
+    dim_scores = kvs_result["dimension_scores"]
+    contributions = kvs_result["weighted_contributions"]
+    recommendations = kvs_result["recommendations"]
+
+    eval_date = (meta or {}).get("eval_date", now)
+    framework_version = (meta or {}).get("framework_version", "KVL v0.1")
+
+    lines = [
+        "# Knowledge Value Report",
+        "",
+        f"**Document:** {doc_title}  ",
+        f"**Evaluated:** {eval_date}  ",
+        f"**Framework:** {framework_version}",
+        "",
+    ]
+
+    # ── Model Metadata block ──────────────────────────────────────────────────
+    lines += [
+        "## Evaluation Models",
+        "",
+        "> ⚠️ **Score validity notice:** Knowledge Novelty and Generation Utility scores are",
+        "> **model-relative** — they measure the marginal value of this document to the specific",
+        "> AI models listed below. Scores will change if these models are updated or replaced.",
+        "> Retrieval Utility and Demand Utility are largely model-independent.",
+        "> Re-evaluate after any major model version change.",
+        "",
+        "| Role | Model | Dimensions |",
+        "|---|---|---|",
+        f"| Evaluation judge | {MODELS['judge']['display']} (`{MODELS['judge']['id']}`) | Novelty scoring, Grounding, Demand |",
+        f"| Answer generation | {MODELS['worker']['display']} (`{MODELS['worker']['id']}`) | Closed-book QA, Baseline & RAG answers, Queries |",
+        f"| Text embeddings | {MODELS['embedder']['display']} | Retrieval index, Semantic similarity |",
+        "",
+        "---",
+        "",
+    ]
+
+    # ── KVS Summary ───────────────────────────────────────────────────────────
+    lines += [
+        "## Overall Knowledge Value Score",
+        "",
+        "```",
+        f"  {kvs} / 100   —   {classification}",
+        f"  {_bar(kvs)}",
+        "```",
+        "",
+        "| Band | Range | Description |",
+        "|---|---|---|",
+    ]
+    for threshold, label, desc in KVS_CLASSIFICATION:
+        marker = " ◀ this document" if label == classification else ""
+        lines.append(f"| **{label}** | {threshold}–{threshold+19 if threshold < 81 else 100} | {desc}{marker} |")
+    lines += ["", "---", ""]
+
+    # ── Dimension table ───────────────────────────────────────────────────────
+    lines += [
+        "## Dimension Scores",
+        "",
+        "| Dimension | Score | Bar | Weight | Contribution | Model Sensitivity |",
+        "|---|---|---|---|---|---|",
+    ]
+    for key, dmeta in DIMENSION_META.items():
+        sc = dim_scores[key]
+        contrib = contributions[key]
+        sens = _sensitivity_badge(dmeta["model_sensitivity"])
+        lines.append(
+            f"| {dmeta['name']} | {sc}/100 | `{_bar(sc, 10)}` "
+            f"| {int(dmeta['weight']*100)}% | {contrib} | {sens} |"
+        )
+    lines += ["", "---", ""]
+
+    # ── Per-dimension detail ──────────────────────────────────────────────────
+    section_order = ["novelty", "retrieval", "generation", "attribution", "demand"]
+    section_titles = {
+        "novelty":     "Knowledge Novelty",
+        "retrieval":   "Retrieval Utility",
+        "generation":  "Generation Utility",
+        "attribution": "Attribution & Grounding",
+        "demand":      "Demand Utility",
+    }
+
+    for key in section_order:
+        dmeta = DIMENSION_META[key]
+        sc = dim_scores[key]
+        sens = _sensitivity_badge(dmeta["model_sensitivity"])
+        lines += [
+            f"## {section_titles[key]}",
+            "",
+            f"**Score: {sc}/100** &nbsp;|&nbsp; Model sensitivity: {sens}",
+            "",
+            f"_{dmeta['description']}_",
+            "",
+            f"**How measured:** {dmeta['how_measured']}",
+            "",
+            f"**Models used:** {', '.join(dmeta['models_used'])}",
+            "",
+            f"**Model sensitivity note:** {dmeta['sensitivity_note']}",
+            "",
+            f"{module_results.get(key, {}).get('summary', '')}",
+            "",
+        ]
+
+        if key == "novelty":
+            details = module_results.get("novelty", {}).get("details", [])
+            if details:
+                lines += [
+                    f"<details><summary>Claim-by-claim breakdown ({len(details)} claims)</summary>",
+                    "",
+                    "| Claim | Known Score | Notes |",
+                    "|---|---|---|",
+                ]
+                for d in details:
+                    lines.append(f"| {d['claim'][:80]} | {d['known_score']:.2f} | {d['reason'][:60]} |")
+                lines += ["", "</details>", ""]
+
+        elif key == "retrieval":
+            details = module_results.get("retrieval", {}).get("details", [])
+            if details:
+                lines += [
+                    f"<details><summary>Query-by-query results ({len(details)} queries)</summary>",
+                    "",
+                    "| Query | Recall@3 | MRR |",
+                    "|---|---|---|",
+                ]
+                for d in details:
+                    lines.append(f"| {d['query'][:80]} | {d['recall_at_3']:.2f} | {d['reciprocal_rank']:.2f} |")
+                lines += ["", "</details>", ""]
+
+        elif key == "generation":
+            details = module_results.get("generation", {}).get("details", [])
+            if details:
+                lines += [
+                    f"<details><summary>Question-by-question results ({len(details)} questions)</summary>",
+                    "",
+                    "| Question | Improvement | Reason |",
+                    "|---|---|---|",
+                ]
+                for d in details:
+                    lines.append(f"| {d['question'][:80]} | {d['improvement']}/100 | {d['reason'][:60]} |")
+                lines += ["", "</details>", ""]
+
+        elif key == "attribution":
+            details = module_results.get("attribution", {}).get("details", [])
+            if details:
+                lines += [
+                    f"<details><summary>Per-answer grounding ({len(details)} answers)</summary>",
+                    "",
+                    "| Question | Grounding % | Hallucination | Semantic Sim |",
+                    "|---|---|---|---|",
+                ]
+                for d in details:
+                    halluc = "Yes ⚠️" if d.get("hallucination_detected") else "No"
+                    lines.append(
+                        f"| {d.get('question','')[:70]} "
+                        f"| {round(d['grounding_fraction']*100)}% "
+                        f"| {halluc} "
+                        f"| {d['semantic_similarity']} |"
+                    )
+                lines += ["", "</details>", ""]
+
+        elif key == "demand":
+            topics = module_results.get("demand", {}).get("topics", [])
+            if topics:
+                lines += [
+                    "| Topic | Query Freq (1-10) | Priority Domain | Unmet Need |",
+                    "|---|---|---|---|",
+                ]
+                for t in topics:
+                    lines.append(
+                        f"| {t.get('topic','')[:50]} "
+                        f"| {t.get('query_frequency','-')}/10 "
+                        f"| {'Yes' if t.get('priority_domain') else 'No'} "
+                        f"| {'Yes' if t.get('unmet_need') else 'No'} |"
+                    )
+                lines.append("")
+
+        lines += ["---", ""]
+
+    # ── Recommendations ───────────────────────────────────────────────────────
+    lines += ["## Recommended Actions", ""]
+    for rec in recommendations:
+        lines.append(f"- {rec}")
+
+    lines += [
+        "",
+        "---",
+        "",
+        f"*Generated by {framework_version} · Evaluated {eval_date}*  ",
+        f"*Judge: {MODELS['judge']['display']} · Worker: {MODELS['worker']['display']} · Embeddings: {MODELS['embedder']['display']}*  ",
+        "*Scores are model-relative. Re-evaluate after major model updates.*",
+    ]
+
+    return "\n".join(lines)

kvl/scorer.py

@@ -0,0 +1,96 @@
+"""KVS aggregation — combines five dimension scores into the Knowledge Value Score."""
+
+from __future__ import annotations
+
+WEIGHTS = {
+    "novelty": 0.30,
+    "retrieval": 0.20,
+    "generation": 0.25,
+    "attribution": 0.15,
+    "demand": 0.10,
+}
+
+CLASSIFICATION = [
+    (81, "Transformational Value"),
+    (61, "High Value"),
+    (41, "Moderate Value"),
+    (21, "Incremental Value"),
+    (0,  "Minimal Value"),
+]
+
+RECOMMENDATIONS = {
+    "high_novelty":       "Prioritize indexing in AI retrieval systems — contains knowledge not in foundation models.",
+    "low_novelty":        "Widely known content; consider whether curation effort is justified.",
+    "high_retrieval":     "Well-structured for retrieval — integrate directly into RAG pipelines.",
+    "low_retrieval":      "Improve chunking, structure, or metadata to boost retrievability.",
+    "high_generation":    "Strongly improves AI-generated answers — valuable for advisory and QA systems.",
+    "low_generation":     "Limited generation uplift — content may be too abstract or redundant.",
+    "high_attribution":   "Answers are well-grounded in document — high trustworthiness for deployment.",
+    "low_attribution":    "Grounding is weak — review document structure and specificity.",
+    "high_demand":        "High user demand — wide deployment and open access recommended.",
+    "low_demand":         "Narrow or specialized demand — consider targeted distribution.",
+    "translate":          "Consider translation into additional languages to broaden impact.",
+    "open_access":        "Recommend open access publication to maximize societal return.",
+}
+
+
+def classify(score: int) -> str:
+    for threshold, label in CLASSIFICATION:
+        if score >= threshold:
+            return label
+    return "Minimal Value"
+
+
+def compute(scores: dict[str, int]) -> dict:
+    """Compute weighted KVS from dimension scores."""
+    kvs = sum(scores[dim] * weight for dim, weight in WEIGHTS.items())
+    kvs = round(kvs)
+
+    weighted_contributions = {
+        dim: round(scores[dim] * weight, 1)
+        for dim, weight in WEIGHTS.items()
+    }
+
+    recommendations = _recommend(scores)
+
+    return {
+        "kvs": kvs,
+        "classification": classify(kvs),
+        "dimension_scores": scores,
+        "weighted_contributions": weighted_contributions,
+        "recommendations": recommendations,
+    }
+
+
+def _recommend(scores: dict[str, int]) -> list[str]:
+    recs = []
+    if scores.get("novelty", 50) >= 65:
+        recs.append(RECOMMENDATIONS["high_novelty"])
+    else:
+        recs.append(RECOMMENDATIONS["low_novelty"])
+
+    if scores.get("retrieval", 50) < 50:
+        recs.append(RECOMMENDATIONS["low_retrieval"])
+    else:
+        recs.append(RECOMMENDATIONS["high_retrieval"])
+
+    if scores.get("generation", 50) >= 60:
+        recs.append(RECOMMENDATIONS["high_generation"])
+    else:
+        recs.append(RECOMMENDATIONS["low_generation"])
+
+    if scores.get("attribution", 50) < 55:
+        recs.append(RECOMMENDATIONS["low_attribution"])
+    else:
+        recs.append(RECOMMENDATIONS["high_attribution"])
+
+    if scores.get("demand", 50) >= 60:
+        recs.append(RECOMMENDATIONS["high_demand"])
+        recs.append(RECOMMENDATIONS["open_access"])
+    else:
+        recs.append(RECOMMENDATIONS["low_demand"])
+
+    if scores.get("novelty", 50) >= 70:
+        recs.append(RECOMMENDATIONS["translate"])
+
+    return recs

requirements.txt

@@ -0,0 +1,7 @@
+anthropic>=0.40.0
+streamlit>=1.40.0
+sentence-transformers>=3.0.0
+faiss-cpu>=1.8.0
+python-dotenv>=1.0.0
+markdown-it-py>=3.0.0
+numpy>=1.26.0