Spaces:
Running
Running
flatten data
Browse filesThis view is limited to 50 files because it contains too many changes. Β See raw diff
- README.md +15 -14
- data/{models/0-hero_Matter-0.1-7B-DPO-preview.json β 0-hero_Matter-0.1-7B-DPO-preview.json} +0 -0
- data/{models/0-hero_Matter-0.1-7B-boost-DPO-preview.json β 0-hero_Matter-0.1-7B-boost-DPO-preview.json} +0 -0
- data/{models/0-hero_Matter-0.2-7B-DPO.json β 0-hero_Matter-0.2-7B-DPO.json} +0 -0
- data/{models/01-ai_Yi-1.5-34B-32K.json β 01-ai_Yi-1.5-34B-32K.json} +0 -0
- data/{models/01-ai_Yi-1.5-34B-Chat-16K.json β 01-ai_Yi-1.5-34B-Chat-16K.json} +0 -0
- data/{models/01-ai_Yi-1.5-34B-Chat.json β 01-ai_Yi-1.5-34B-Chat.json} +0 -0
- data/{models/01-ai_Yi-1.5-34B.json β 01-ai_Yi-1.5-34B.json} +0 -0
- data/{models/01-ai_Yi-1.5-6B-Chat.json β 01-ai_Yi-1.5-6B-Chat.json} +0 -0
- data/{models/01-ai_Yi-1.5-6B.json β 01-ai_Yi-1.5-6B.json} +0 -0
- data/{models/01-ai_Yi-1.5-9B-32K.json β 01-ai_Yi-1.5-9B-32K.json} +0 -0
- data/{models/01-ai_Yi-1.5-9B-Chat-16K.json β 01-ai_Yi-1.5-9B-Chat-16K.json} +0 -0
- data/{models/01-ai_Yi-1.5-9B-Chat.json β 01-ai_Yi-1.5-9B-Chat.json} +0 -0
- data/{models/01-ai_Yi-1.5-9B.json β 01-ai_Yi-1.5-9B.json} +0 -0
- data/{models/01-ai_Yi-34B-200K.json β 01-ai_Yi-34B-200K.json} +0 -0
- data/{models/01-ai_Yi-34B-Chat.json β 01-ai_Yi-34B-Chat.json} +0 -0
- data/{models/01-ai_Yi-6B-200K.json β 01-ai_Yi-6B-200K.json} +0 -0
- data/{models/01-ai_Yi-6B-Chat.json β 01-ai_Yi-6B-Chat.json} +0 -0
- data/{models/01-ai_Yi-9B-200K.json β 01-ai_Yi-9B-200K.json} +0 -0
- data/{models/01-ai_Yi-9B.json β 01-ai_Yi-9B.json} +0 -0
- data/{models/01-ai_Yi-Coder-9B-Chat.json β 01-ai_Yi-Coder-9B-Chat.json} +0 -0
- data/{models/01-ai_yi-34b.json β 01-ai_yi-34b.json} +0 -0
- data/{models/01-ai_yi-6b.json β 01-ai_yi-6b.json} +0 -0
- data/{models/01-ai_yi-large-preview.json β 01-ai_yi-large-preview.json} +0 -0
- data/{models/1-800-LLMs_Qwen-2.5-14B-Hindi-Custom-Instruct.json β 1-800-LLMs_Qwen-2.5-14B-Hindi-Custom-Instruct.json} +0 -0
- data/{models/1-800-LLMs_Qwen-2.5-14B-Hindi.json β 1-800-LLMs_Qwen-2.5-14B-Hindi.json} +0 -0
- data/{models/1024m_PHI-4-Hindi.json β 1024m_PHI-4-Hindi.json} +0 -0
- data/{models/1024m_QWEN-14B-B100.json β 1024m_QWEN-14B-B100.json} +0 -0
- data/{models/152334H_miqu-1-70b-sf.json β 152334H_miqu-1-70b-sf.json} +0 -0
- data/{models/1TuanPham_T-VisStar-7B-v0.1.json β 1TuanPham_T-VisStar-7B-v0.1.json} +0 -0
- data/{models/1TuanPham_T-VisStar-v0.1.json β 1TuanPham_T-VisStar-v0.1.json} +0 -0
- data/{models/3rd-Degree-Burn_L-3.1-Science-Writer-8B.json β 3rd-Degree-Burn_L-3.1-Science-Writer-8B.json} +0 -0
- data/{models/3rd-Degree-Burn_Llama-3.1-8B-Squareroot-v1.json β 3rd-Degree-Burn_Llama-3.1-8B-Squareroot-v1.json} +0 -0
- data/{models/3rd-Degree-Burn_Llama-3.1-8B-Squareroot.json β 3rd-Degree-Burn_Llama-3.1-8B-Squareroot.json} +0 -0
- data/{models/3rd-Degree-Burn_Llama-Squared-8B.json β 3rd-Degree-Burn_Llama-Squared-8B.json} +0 -0
- data/{models/4season_final_model_test_v2.json β 4season_final_model_test_v2.json} +0 -0
- data/{models/AALF_FuseChat-Llama-3.1-8B-Instruct-preview.json β AALF_FuseChat-Llama-3.1-8B-Instruct-preview.json} +0 -0
- data/{models/AALF_FuseChat-Llama-3.1-8B-SFT-preview.json β AALF_FuseChat-Llama-3.1-8B-SFT-preview.json} +0 -0
- data/{models/AALF_gemma-2-27b-it-SimPO-37K-100steps.json β AALF_gemma-2-27b-it-SimPO-37K-100steps.json} +0 -0
- data/{models/AALF_gemma-2-27b-it-SimPO-37K.json β AALF_gemma-2-27b-it-SimPO-37K.json} +0 -0
- data/{models/AELLM_gemma-2-aeria-infinity-9b.json β AELLM_gemma-2-aeria-infinity-9b.json} +0 -0
- data/{models/AELLM_gemma-2-lyco-infinity-9b.json β AELLM_gemma-2-lyco-infinity-9b.json} +0 -0
- data/{models/AGI-0_Art-v0-3B.json β AGI-0_Art-v0-3B.json} +0 -0
- data/{models/AGI-0_Artificium-llama3.1-8B-001.json β AGI-0_Artificium-llama3.1-8B-001.json} +0 -0
- data/{models/AGI-0_smartllama3.1-8B-001.json β AGI-0_smartllama3.1-8B-001.json} +0 -0
- data/{models/AI-MO_NuminaMath-7B-CoT.json β AI-MO_NuminaMath-7B-CoT.json} +0 -0
- data/{models/AI-MO_NuminaMath-7B-TIR.json β AI-MO_NuminaMath-7B-TIR.json} +0 -0
- data/{models/AI-Sweden-Models_Llama-3-8B-instruct.json β AI-Sweden-Models_Llama-3-8B-instruct.json} +0 -0
- data/{models/AI-Sweden-Models_gpt-sw3-40b.json β AI-Sweden-Models_gpt-sw3-40b.json} +0 -0
- data/{models/AI4free_Dhanishtha.json β AI4free_Dhanishtha.json} +0 -0
README.md
CHANGED
|
@@ -16,7 +16,7 @@ tags:
|
|
| 16 |
|
| 17 |
# Eval Cards
|
| 18 |
|
| 19 |
-
This repository is a Next.js application for
|
| 20 |
|
| 21 |
## Project Goals
|
| 22 |
|
|
@@ -29,15 +29,16 @@ The Eval Cards project aims to:
|
|
| 29 |
|
| 30 |
## For External Collaborators
|
| 31 |
|
| 32 |
-
### Making Changes to
|
| 33 |
|
| 34 |
-
|
| 35 |
|
| 36 |
-
Key
|
| 37 |
-
- **`
|
| 38 |
-
- **`
|
| 39 |
-
- **`
|
| 40 |
-
- **`
|
|
|
|
| 41 |
|
| 42 |
### Standards and Frameworks Used
|
| 43 |
|
|
@@ -49,13 +50,13 @@ This ensures consistency with international AI governance standards and facilita
|
|
| 49 |
|
| 50 |
### Contributing Evaluation Data
|
| 51 |
|
| 52 |
-
Evaluation data files are stored in `
|
| 53 |
|
| 54 |
To add a new evaluation:
|
| 55 |
-
1. Create a new JSON file in `
|
| 56 |
-
2. Follow the
|
| 57 |
-
3. Ensure
|
| 58 |
-
4.
|
| 59 |
|
| 60 |
### Development Setup
|
| 61 |
|
|
@@ -100,4 +101,4 @@ Visit `http://localhost:3000` to verify.
|
|
| 100 |
2. Push this repository to the Space Git (or upload files through the UI). The Space will build the Docker image using the included `Dockerfile` and serve your app on port 3000.
|
| 101 |
|
| 102 |
Notes:
|
| 103 |
-
- If your build needs native dependencies (e.g. `sharp`), the Docker image may require extra apt packages; update the Dockerfile accordingly.
|
|
|
|
| 16 |
|
| 17 |
# Eval Cards
|
| 18 |
|
| 19 |
+
This repository is a Next.js application for viewing and authoring AI evaluations. It provides a comprehensive platform for documenting and sharing AI system evaluations across multiple dimensions including capabilities and risks.
|
| 20 |
|
| 21 |
## Project Goals
|
| 22 |
|
|
|
|
| 29 |
|
| 30 |
## For External Collaborators
|
| 31 |
|
| 32 |
+
### Making Changes to Evaluation Categories and Schema
|
| 33 |
|
| 34 |
+
All evaluation categories, form fields, and data structures are centrally managed in the `schema/` folder. **This is the primary location for making structural changes to the evaluation framework.**
|
| 35 |
|
| 36 |
+
Key schema files:
|
| 37 |
+
- **`schema/evaluation-schema.json`** - Defines all evaluation categories (capabilities and risks)
|
| 38 |
+
- **`schema/output-schema.json`** - Defines the complete data structure for evaluation outputs
|
| 39 |
+
- **`schema/system-info-schema.json`** - Defines form field options for system information
|
| 40 |
+
- **`schema/category-details.json`** - Contains detailed descriptions and criteria for each category
|
| 41 |
+
- **`schema/form-hints.json`** - Provides help text and guidance for form fields
|
| 42 |
|
| 43 |
### Standards and Frameworks Used
|
| 44 |
|
|
|
|
| 50 |
|
| 51 |
### Contributing Evaluation Data
|
| 52 |
|
| 53 |
+
Evaluation data files are stored in `public/benchmarks/` as JSON files. Each file represents a complete evaluation of an AI system and must conform to the schema defined in `schema/output-schema.json`.
|
| 54 |
|
| 55 |
To add a new evaluation:
|
| 56 |
+
1. Create a new JSON file in `public/benchmarks/`
|
| 57 |
+
2. Follow the structure defined in `schema/output-schema.json`
|
| 58 |
+
3. Ensure all required fields are populated
|
| 59 |
+
4. Validate against the schema before submission
|
| 60 |
|
| 61 |
### Development Setup
|
| 62 |
|
|
|
|
| 101 |
2. Push this repository to the Space Git (or upload files through the UI). The Space will build the Docker image using the included `Dockerfile` and serve your app on port 3000.
|
| 102 |
|
| 103 |
Notes:
|
| 104 |
+
- If your build needs native dependencies (e.g. `sharp`), the Docker image may require extra apt packages; update the Dockerfile accordingly.
|
data/{models/0-hero_Matter-0.1-7B-DPO-preview.json β 0-hero_Matter-0.1-7B-DPO-preview.json}
RENAMED
|
File without changes
|
data/{models/0-hero_Matter-0.1-7B-boost-DPO-preview.json β 0-hero_Matter-0.1-7B-boost-DPO-preview.json}
RENAMED
|
File without changes
|
data/{models/0-hero_Matter-0.2-7B-DPO.json β 0-hero_Matter-0.2-7B-DPO.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-1.5-34B-32K.json β 01-ai_Yi-1.5-34B-32K.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-1.5-34B-Chat-16K.json β 01-ai_Yi-1.5-34B-Chat-16K.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-1.5-34B-Chat.json β 01-ai_Yi-1.5-34B-Chat.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-1.5-34B.json β 01-ai_Yi-1.5-34B.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-1.5-6B-Chat.json β 01-ai_Yi-1.5-6B-Chat.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-1.5-6B.json β 01-ai_Yi-1.5-6B.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-1.5-9B-32K.json β 01-ai_Yi-1.5-9B-32K.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-1.5-9B-Chat-16K.json β 01-ai_Yi-1.5-9B-Chat-16K.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-1.5-9B-Chat.json β 01-ai_Yi-1.5-9B-Chat.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-1.5-9B.json β 01-ai_Yi-1.5-9B.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-34B-200K.json β 01-ai_Yi-34B-200K.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-34B-Chat.json β 01-ai_Yi-34B-Chat.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-6B-200K.json β 01-ai_Yi-6B-200K.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-6B-Chat.json β 01-ai_Yi-6B-Chat.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-9B-200K.json β 01-ai_Yi-9B-200K.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-9B.json β 01-ai_Yi-9B.json}
RENAMED
|
File without changes
|
data/{models/01-ai_Yi-Coder-9B-Chat.json β 01-ai_Yi-Coder-9B-Chat.json}
RENAMED
|
File without changes
|
data/{models/01-ai_yi-34b.json β 01-ai_yi-34b.json}
RENAMED
|
File without changes
|
data/{models/01-ai_yi-6b.json β 01-ai_yi-6b.json}
RENAMED
|
File without changes
|
data/{models/01-ai_yi-large-preview.json β 01-ai_yi-large-preview.json}
RENAMED
|
File without changes
|
data/{models/1-800-LLMs_Qwen-2.5-14B-Hindi-Custom-Instruct.json β 1-800-LLMs_Qwen-2.5-14B-Hindi-Custom-Instruct.json}
RENAMED
|
File without changes
|
data/{models/1-800-LLMs_Qwen-2.5-14B-Hindi.json β 1-800-LLMs_Qwen-2.5-14B-Hindi.json}
RENAMED
|
File without changes
|
data/{models/1024m_PHI-4-Hindi.json β 1024m_PHI-4-Hindi.json}
RENAMED
|
File without changes
|
data/{models/1024m_QWEN-14B-B100.json β 1024m_QWEN-14B-B100.json}
RENAMED
|
File without changes
|
data/{models/152334H_miqu-1-70b-sf.json β 152334H_miqu-1-70b-sf.json}
RENAMED
|
File without changes
|
data/{models/1TuanPham_T-VisStar-7B-v0.1.json β 1TuanPham_T-VisStar-7B-v0.1.json}
RENAMED
|
File without changes
|
data/{models/1TuanPham_T-VisStar-v0.1.json β 1TuanPham_T-VisStar-v0.1.json}
RENAMED
|
File without changes
|
data/{models/3rd-Degree-Burn_L-3.1-Science-Writer-8B.json β 3rd-Degree-Burn_L-3.1-Science-Writer-8B.json}
RENAMED
|
File without changes
|
data/{models/3rd-Degree-Burn_Llama-3.1-8B-Squareroot-v1.json β 3rd-Degree-Burn_Llama-3.1-8B-Squareroot-v1.json}
RENAMED
|
File without changes
|
data/{models/3rd-Degree-Burn_Llama-3.1-8B-Squareroot.json β 3rd-Degree-Burn_Llama-3.1-8B-Squareroot.json}
RENAMED
|
File without changes
|
data/{models/3rd-Degree-Burn_Llama-Squared-8B.json β 3rd-Degree-Burn_Llama-Squared-8B.json}
RENAMED
|
File without changes
|
data/{models/4season_final_model_test_v2.json β 4season_final_model_test_v2.json}
RENAMED
|
File without changes
|
data/{models/AALF_FuseChat-Llama-3.1-8B-Instruct-preview.json β AALF_FuseChat-Llama-3.1-8B-Instruct-preview.json}
RENAMED
|
File without changes
|
data/{models/AALF_FuseChat-Llama-3.1-8B-SFT-preview.json β AALF_FuseChat-Llama-3.1-8B-SFT-preview.json}
RENAMED
|
File without changes
|
data/{models/AALF_gemma-2-27b-it-SimPO-37K-100steps.json β AALF_gemma-2-27b-it-SimPO-37K-100steps.json}
RENAMED
|
File without changes
|
data/{models/AALF_gemma-2-27b-it-SimPO-37K.json β AALF_gemma-2-27b-it-SimPO-37K.json}
RENAMED
|
File without changes
|
data/{models/AELLM_gemma-2-aeria-infinity-9b.json β AELLM_gemma-2-aeria-infinity-9b.json}
RENAMED
|
File without changes
|
data/{models/AELLM_gemma-2-lyco-infinity-9b.json β AELLM_gemma-2-lyco-infinity-9b.json}
RENAMED
|
File without changes
|
data/{models/AGI-0_Art-v0-3B.json β AGI-0_Art-v0-3B.json}
RENAMED
|
File without changes
|
data/{models/AGI-0_Artificium-llama3.1-8B-001.json β AGI-0_Artificium-llama3.1-8B-001.json}
RENAMED
|
File without changes
|
data/{models/AGI-0_smartllama3.1-8B-001.json β AGI-0_smartllama3.1-8B-001.json}
RENAMED
|
File without changes
|
data/{models/AI-MO_NuminaMath-7B-CoT.json β AI-MO_NuminaMath-7B-CoT.json}
RENAMED
|
File without changes
|
data/{models/AI-MO_NuminaMath-7B-TIR.json β AI-MO_NuminaMath-7B-TIR.json}
RENAMED
|
File without changes
|
data/{models/AI-Sweden-Models_Llama-3-8B-instruct.json β AI-Sweden-Models_Llama-3-8B-instruct.json}
RENAMED
|
File without changes
|
data/{models/AI-Sweden-Models_gpt-sw3-40b.json β AI-Sweden-Models_gpt-sw3-40b.json}
RENAMED
|
File without changes
|
data/{models/AI4free_Dhanishtha.json β AI4free_Dhanishtha.json}
RENAMED
|
File without changes
|