Automated MNLP evaluation report (2026-06-11)

EVAL_REPORT.md

@@ -0,0 +1,92 @@
+# Automated MNLP evaluation report
+
+- **Model repo:** [`cs-552-2026-bilko/general_knowledge_model`](https://huggingface.co/cs-552-2026-bilko/general_knowledge_model)
+- **Owner(s):** group **bilko**
+- **Generated at:** 2026-06-11T06:23:10+00:00 (UTC)
+- **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
+
+_This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._
+
+## Evaluated checkpoint
+
+- **Commit:** [`89ea167`](https://huggingface.co/cs-552-2026-bilko/general_knowledge_model/commit/89ea1671e79c98f3aa2d1d5ed7a4f6324bda7aeb)
+- **Message:** Delete lora_adapter
+- **Committed:** 2026-06-06T20:52:59+00:00
+
+## Summary
+
+| Benchmark | Accuracy | Status |
+|---|---:|---|
+| Math | — | not run |
+| Knowledge | 0.3900 | ok |
+| Multilingual | — | not run |
+| Safety | — | not run |
+
+## Sample completions
+
+_Prompts are intentionally omitted to avoid revealing benchmark contents. For multi-completion problems, only one completion is shown per sample._
+
+### Knowledge
+
+**Correct** (1 shown)
+
+- **reference**: `C`
+- **overall** (1/1 completions correct)
+- **extracted** (✓): `C`
+- **completion**:
+
+    ```text
+    <think>
+    Okay, let's tackle this question. The question is asking which mental disorder has research suggesting a link with acetylcholine. The options are from A to J.
+    
+    First, I need to remember what acetylcholine does. Acetylcholine is a neurotransmitter involved in various bodily functions, including muscle contraction, memory, and learning. It's secreted by the parasympathetic nervous system, wh
+    
+    ... [truncated 822 chars] ...
+    
+    opamine dysfunction**, particularly in the prefrontal cortex. Acetylcholine may be involved, but it is not the primary neurotransmitter in this disorder.
+    
+    - **H. Bipolar Disorder**  
+      Bipolar disorder involves **dopamine and serotonin imbalances**, but there is no strong evidence that acetylcholine plays a central role in its pathophysiology.
+    
+    - **I. Schizophrenia**  
+      Schizophrenia is associated with **dopamine and serotonin imbalances**, but research on acetylcholine is limited. While cholinergic dysfunction may contribute to some symptoms (e.g., hallucinations), it is not the primary factor.
+    
+    - **J. Major Depressive Disorder**  
+      Major depression is associated with **serotonin and norepinephrine dysfunction**, not acetylcholine. Acetylcholine is not a primary neurotransmitter in depression.
+    
+    ---
+    
+    ### Final Conclusion
+    
+    Among all the options, **Alzheimer’s Disease** is most directly associated with **acetylcholine deficits**, which are central to the disease’s progression and clinical features. While **Parkinson’s Disease** also involves acetylcholine, it is primarily associated with **dopamine loss**. Thus, the disorder most clearly linked to acetylcholine is:
+    
+    $$
+    \boxed{C}
+    $$
+    ```
+
+**Incorrect** (1 shown)
+
+- **reference**: `A`
+- **overall** (0/1 completions correct)
+- **extracted** (✗): `<no answer>`
+- **completion**:
+
+    ```text
+    <think>
+    Okay, let's try to figure out this problem about the power spectral density of a function defined by a convolution. Hmm, so we have f(t) defined as the convolution of h(σ) and φ(t - σ), right? And we need to find the power spectral density of f(t) in terms of the power spectral densities of φ and h, and the Fourier transform of h.
+    
+    First, I remember that the power spectral density (PSD) of
+    
+    ... [truncated 822 chars] ...
+    
+    rrelation function. For the convolution, the autocorrelation is the convolution of the autocorrelation functions. So, R_f(τ) = R_h(τ) * R_φ(τ). Then, G_f(ω) = F[R_f(τ)] = F[R_h(τ) * R_φ(τ)] = F(R_h) * F(R_φ) = G_h(ω) * G_φ(ω). 
+    
+    But if G_h(ω) is the power spectral density of h(t), then G_f(ω) = G_h(ω) * G_φ(ω). This is the correct relation. Therefore, the answer should be G_f(ω) = G_h(ω) * G_φ(ω). But since G_h(ω) is |H(ω)|², this is |H(ω)|² * G_φ(ω). 
+    
+    But none of the choices have this. However, if the question is asking for the power spectral density of f(t) in terms of the Fourier transform of h and the power spectral density of φ, then it might expect us to use the fact that G_f(ω) = G_h(ω) * G_φ(ω), and since G_h(ω) = |H(ω)|², this would be |H(ω)|² * G_φ(ω). But this is not among the choices. 
+    
+    Given that, and considering that choice C is G_f(ω) = H(ω) * G_φ(ω)^2, which is |H(ω)|² * G_φ(ω)^2 if H(ω) is real and positive, but this is not generally true. 
+    
+    Wait, but if we take the Fourier transform of the convolution, which is H(ω) * H_φ(ω), and then take the magnitude squared, we get |H(ω) * H_φ(ω)|² = |H(ω)|² |H_φ(ω)|² = |H(ω)|² G_φ(ω). This is the same as G_f(ω) = |H(ω)|² G_φ
+    ```