Spaces:
Sleeping
Sleeping
Sync from GitHub via hub-sync
Browse files- reports/evaluation_report.html +2 -1
- reports/evaluation_report.pdf +2 -2
- reports/report.md +3 -1
reports/evaluation_report.html
CHANGED
|
@@ -75,7 +75,8 @@ strong {
|
|
| 75 |
<h1>Comparative Evaluation Report: Local Open-Source vs. Hosted Frontier Assistants</h1>
|
| 76 |
<p><strong>Model Stack:</strong> Local <code>Qwen2.5:1.5b-instruct</code> (Ollama) vs. Hosted <code>Llama-3.1-8b-instant</code> (Groq API)<br />
|
| 77 |
<strong>Evaluation Scope:</strong> 39 benchmark prompts spanning Factuality (15), Jailbreak/Adversarial (12), and Stereotype/Bias (12).<br />
|
| 78 |
-
<strong>Scoring Engine:</strong> Automated LLM-as-a-Judge (<code>Llama-3.1-8b-instant</code>) executing multidimensional evaluation on a 1–5 scale.</
|
|
|
|
| 79 |
<hr />
|
| 80 |
<h2>1. Executive Summary</h2>
|
| 81 |
<p>This report evaluates the operational trade-offs, response quality, safety resilience, and cost profiles of deploying a lightweight, local open-source software (OSS) model versus a hosted frontier model. </p>
|
|
|
|
| 75 |
<h1>Comparative Evaluation Report: Local Open-Source vs. Hosted Frontier Assistants</h1>
|
| 76 |
<p><strong>Model Stack:</strong> Local <code>Qwen2.5:1.5b-instruct</code> (Ollama) vs. Hosted <code>Llama-3.1-8b-instant</code> (Groq API)<br />
|
| 77 |
<strong>Evaluation Scope:</strong> 39 benchmark prompts spanning Factuality (15), Jailbreak/Adversarial (12), and Stereotype/Bias (12).<br />
|
| 78 |
+
<strong>Scoring Engine:</strong> Automated LLM-as-a-Judge (<code>Llama-3.1-8b-instant</code>) executing multidimensional evaluation on a 1–5 scale.<br />
|
| 79 |
+
<strong>Live Interactive Demo:</strong> <a href="https://huggingface.co/spaces/vedanshmathur7/ModelArena">https://huggingface.co/spaces/vedanshmathur7/ModelArena</a></p>
|
| 80 |
<hr />
|
| 81 |
<h2>1. Executive Summary</h2>
|
| 82 |
<p>This report evaluates the operational trade-offs, response quality, safety resilience, and cost profiles of deploying a lightweight, local open-source software (OSS) model versus a hosted frontier model. </p>
|
reports/evaluation_report.pdf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8a10d18efe75c94212f98f4f9e7d61ed345c81ae6dcb672d820517ca4294a85a
|
| 3 |
+
size 444980
|
reports/report.md
CHANGED
|
@@ -2,7 +2,9 @@
|
|
| 2 |
|
| 3 |
**Model Stack:** Local `Qwen2.5:1.5b-instruct` (Ollama) vs. Hosted `Llama-3.1-8b-instant` (Groq API)
|
| 4 |
**Evaluation Scope:** 39 benchmark prompts spanning Factuality (15), Jailbreak/Adversarial (12), and Stereotype/Bias (12).
|
| 5 |
-
**Scoring Engine:** Automated LLM-as-a-Judge (`Llama-3.1-8b-instant`) executing multidimensional evaluation on a 1–5 scale.
|
|
|
|
|
|
|
| 6 |
|
| 7 |
---
|
| 8 |
|
|
|
|
| 2 |
|
| 3 |
**Model Stack:** Local `Qwen2.5:1.5b-instruct` (Ollama) vs. Hosted `Llama-3.1-8b-instant` (Groq API)
|
| 4 |
**Evaluation Scope:** 39 benchmark prompts spanning Factuality (15), Jailbreak/Adversarial (12), and Stereotype/Bias (12).
|
| 5 |
+
**Scoring Engine:** Automated LLM-as-a-Judge (`Llama-3.1-8b-instant`) executing multidimensional evaluation on a 1–5 scale.
|
| 6 |
+
**Live Interactive Demo:** [https://huggingface.co/spaces/vedanshmathur7/ModelArena](https://huggingface.co/spaces/vedanshmathur7/ModelArena)
|
| 7 |
+
|
| 8 |
|
| 9 |
---
|
| 10 |
|