Automated MNLP evaluation report (2026-06-11)

EVAL_REPORT.md

@@ -0,0 +1,112 @@
+# Automated MNLP evaluation report
+
+- **Model repo:** [`cs-552-2026-RatGPT/general_knowledge_model`](https://huggingface.co/cs-552-2026-RatGPT/general_knowledge_model)
+- **Owner(s):** group **RatGPT**
+- **Generated at:** 2026-06-11T06:23:10+00:00 (UTC)
+- **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
+
+_This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._
+
+## Evaluated checkpoint
+
+- **Commit:** [`e59119b`](https://huggingface.co/cs-552-2026-RatGPT/general_knowledge_model/commit/e59119ba6536588297d77b7e55ea55ff161c4375)
+- **Message:** SFT v1: auto-distillation, concise reasoning + commit-to-box
+- **Committed:** 2026-06-01T22:12:03+00:00
+
+## Summary
+
+| Benchmark | Accuracy | Status |
+|---|---:|---|
+| Math | — | not run |
+| Knowledge | 0.3800 | ok |
+| Multilingual | — | not run |
+| Safety | — | not run |
+
+## Sample completions
+
+_Prompts are intentionally omitted to avoid revealing benchmark contents. For multi-completion problems, only one completion is shown per sample._
+
+### Knowledge
+
+**Correct** (1 shown)
+
+- **reference**: `C`
+- **overall** (1/1 completions correct)
+- **extracted** (✓): `C`
+- **completion**:
+
+    ```text
+    <think>
+    Okay, let's see. The question is asking which mental disorder is linked to acetylcholine research. The choices are from A to J.
+    
+    First, I need to recall what I know about acetylcholine and mental disorders. Acetylcholine is a neurotransmitter involved in various bodily functions, including muscle movement, memory, and learning. I remember that cholinergic pathways are important in the brai
+    
+    ... [truncated 822 chars] ...
+    
+    C. Alzheimer's disease**: Directly linked to acetylcholine. The **degeneration of cholinergic neurons** is a central feature of the disease.
+    - **D. Parkinson's disease**: Also linked to acetylcholine, especially in the **basal ganglia**, where **acetylcholine levels are reduced**. While the primary issue is dopamine, the role of acetylcholine in the disease is well-documented.
+    - **E. Post-Traumatic Stress Disorder (PTSD)**: Not directly linked to acetylcholine.
+    - **F. Mania**: Not directly linked to acetylcholine.
+    - **G. Attention Deficit Hyperactivity Disorder (ADHD)**: Not directly linked.
+    - **H. Bipolar Disorder**: Not directly linked.
+    - **I. Schizophrenia**: Not directly linked.
+    - **J. Major Depressive Disorder**: Not directly linked.
+    
+    ---
+    
+    ### Conclusion:
+    
+    Both **Alzheimer's disease** and **Parkinson's disease** are **directly linked to acetylcholine**, with the former being more **specifically associated** with the decline in cholinergic neurons. However, since the question asks for **which of the following** is **"research suggested may be linked"**, and **both C and D are valid**, the most **specific and well-established** answer is **Alzheimer's disease**.
+    
+    $$
+    \boxed{C}
+    $$
+    ```
+
+**Incorrect** (1 shown)
+
+- **reference**: `A`
+- **overall** (0/1 completions correct)
+- **extracted** (✗): `G`
+- **completion**:
+
+    ```text
+    <think>
+    Okay, so I need to find the power spectral density (PSD) of the function f(t), which is defined as the convolution of h(σ) and φ(t - σ). The question gives me the choices, and I need to figure out which one is correct. Let me recall some concepts about power spectral densities and convolution.
+    
+    First, I remember that the power spectral density of a signal is related to its autocorrelation
+    
+    ... [truncated 820 chars] ...
+    
+    Now, we compare this with the given choices:
+    
+    - **Choice G**: $ G_f(\omega) = \frac{|H(\omega)|^2}{G_\phi(\omega)} $
+    
+    This is **not** the correct expression. It would be the inverse of the correct formula.
+    
+    - **Choice A**: $ G_f(\omega) = |H(\omega)|^2 \cdot G_\phi(\omega)^2 $ → Incorrect.
+    
+    - **Choice B**: $ G_f(\omega) = \frac{G_\phi(\omega)}{|H(\omega)|^2} $ → Incorrect.
+    
+    - **Choice C**: $ G_f(\omega) = H(\omega) \cdot G_\phi(\omega)^2 $ → Incorrect.
+    
+    - **Choice D**: $ G_f(\omega) = \frac{|H(\omega)|^2}{G_\phi(\omega)} $ → Incorrect.
+    
+    - **Choice E**: $ G_f(\omega) = \frac{1}{|H(\omega)|^2} \cdot G_\phi(\omega) $ → Incorrect.
+    
+    - **Choice F**: $ G_f(\omega) = |H(\omega)| \cdot G_\phi(\omega)^2 $ → Incorrect.
+    
+    - **Choice J**: $ G_f(\omega) = |H(\omega)| \cdot G_\phi(\omega) $ → Incorrect.
+    
+    ---
+    
+    ### **Conclusion**
+    
+    None of the choices exactly match the derived formula $ G_f(\omega) = |H(\omega)|^2 \cdot G_\phi(\omega) $. However, **Choice G** is the **closest match** in the sense that it is the only one that involves $ |H(\omega)|^2 $, which is the correct expression for the power spectral density of the convolution of two signals.
+    
+    Thus, the **correct answer is**:
+    
+    $$
+    \boxed{G}
+    $$
+    ```