vedanshmathur7 commited on
Commit
ebef106
·
verified ·
1 Parent(s): e1c64a2

Sync from GitHub via hub-sync

Browse files
reports/evaluation_report.html CHANGED
@@ -75,7 +75,8 @@ strong {
75
  <h1>Comparative Evaluation Report: Local Open-Source vs. Hosted Frontier Assistants</h1>
76
  <p><strong>Model Stack:</strong> Local <code>Qwen2.5:1.5b-instruct</code> (Ollama) vs. Hosted <code>Llama-3.1-8b-instant</code> (Groq API)<br />
77
  <strong>Evaluation Scope:</strong> 39 benchmark prompts spanning Factuality (15), Jailbreak/Adversarial (12), and Stereotype/Bias (12).<br />
78
- <strong>Scoring Engine:</strong> Automated LLM-as-a-Judge (<code>Llama-3.1-8b-instant</code>) executing multidimensional evaluation on a 1–5 scale.</p>
 
79
  <hr />
80
  <h2>1. Executive Summary</h2>
81
  <p>This report evaluates the operational trade-offs, response quality, safety resilience, and cost profiles of deploying a lightweight, local open-source software (OSS) model versus a hosted frontier model. </p>
 
75
  <h1>Comparative Evaluation Report: Local Open-Source vs. Hosted Frontier Assistants</h1>
76
  <p><strong>Model Stack:</strong> Local <code>Qwen2.5:1.5b-instruct</code> (Ollama) vs. Hosted <code>Llama-3.1-8b-instant</code> (Groq API)<br />
77
  <strong>Evaluation Scope:</strong> 39 benchmark prompts spanning Factuality (15), Jailbreak/Adversarial (12), and Stereotype/Bias (12).<br />
78
+ <strong>Scoring Engine:</strong> Automated LLM-as-a-Judge (<code>Llama-3.1-8b-instant</code>) executing multidimensional evaluation on a 1–5 scale.<br />
79
+ <strong>Live Interactive Demo:</strong> <a href="https://huggingface.co/spaces/vedanshmathur7/ModelArena">https://huggingface.co/spaces/vedanshmathur7/ModelArena</a></p>
80
  <hr />
81
  <h2>1. Executive Summary</h2>
82
  <p>This report evaluates the operational trade-offs, response quality, safety resilience, and cost profiles of deploying a lightweight, local open-source software (OSS) model versus a hosted frontier model. </p>
reports/evaluation_report.pdf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:16bded9ab1a2edd72cf796078e9e0c7f443c694fa73d2dc486144de281bbbf58
3
- size 443726
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a10d18efe75c94212f98f4f9e7d61ed345c81ae6dcb672d820517ca4294a85a
3
+ size 444980
reports/report.md CHANGED
@@ -2,7 +2,9 @@
2
 
3
  **Model Stack:** Local `Qwen2.5:1.5b-instruct` (Ollama) vs. Hosted `Llama-3.1-8b-instant` (Groq API)
4
  **Evaluation Scope:** 39 benchmark prompts spanning Factuality (15), Jailbreak/Adversarial (12), and Stereotype/Bias (12).
5
- **Scoring Engine:** Automated LLM-as-a-Judge (`Llama-3.1-8b-instant`) executing multidimensional evaluation on a 1–5 scale.
 
 
6
 
7
  ---
8
 
 
2
 
3
  **Model Stack:** Local `Qwen2.5:1.5b-instruct` (Ollama) vs. Hosted `Llama-3.1-8b-instant` (Groq API)
4
  **Evaluation Scope:** 39 benchmark prompts spanning Factuality (15), Jailbreak/Adversarial (12), and Stereotype/Bias (12).
5
+ **Scoring Engine:** Automated LLM-as-a-Judge (`Llama-3.1-8b-instant`) executing multidimensional evaluation on a 1–5 scale.
6
+ **Live Interactive Demo:** [https://huggingface.co/spaces/vedanshmathur7/ModelArena](https://huggingface.co/spaces/vedanshmathur7/ModelArena)
7
+
8
 
9
  ---
10