aayushiii18 commited on
Commit
0d43549
·
verified ·
1 Parent(s): 761efa6

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +69 -0
README.md ADDED
@@ -0,0 +1,69 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - legal
5
+ - bert
6
+ - legal-bert
7
+ - contract-risk
8
+ - cuad
9
+ - pytorch
10
+ pipeline_tag: text-classification
11
+ library_name: transformers
12
+ ---
13
+
14
+ # RiskLoop Representative Model Checkpoints
15
+
16
+ This repository hosts the **reproduced representative model checkpoints** for [RiskLoop](https://github.com/aayushiii18/RiskLoop), an NLP system for automated contract risk detection based on Legal-BERT fine-tuning on the CUAD (Contract Understanding Atticus Dataset) benchmark.
17
+
18
+ > [!IMPORTANT]
19
+ > **Provenance & Reproduction Note:**
20
+ > These checkpoints are **reproduced representative checkpoints** created under strictly controlled, frozen training conditions (Condition A) and evaluated in RiskLoop's frozen Phase 6 test evaluation.
21
+ > They are **NOT** the original historical official checkpoints from earlier development phases, as those original historical binaries were lost. These reproduced models represent the exact frozen representative checkpoints used to generate the Phase 6 decision analysis and final test metrics.
22
+
23
+ ---
24
+
25
+ ## 1. Model Architecture & Training Details
26
+
27
+ - **Backbone Model:** `nlpaueb/legal-bert-base-uncased`
28
+ - **Model Type:** Single-task span classification models (QA-style start/end logit heads over transformer sequence outputs).
29
+ - **Training Strategy:** Condition A (single-task fine-tuning with fixed seeds, 4 epochs, max sequence length 512, document stride 256).
30
+
31
+ ---
32
+
33
+ ## 2. Model Checkpoint Registry & Integrity Hashes
34
+
35
+ The repository contains three frozen model binary files (`.pt` PyTorch state dictionaries):
36
+
37
+ | Checkpoint Filename | Target Contract Task | Experimental Condition | Random Seed | File Size (Bytes) | SHA-256 Hash |
38
+ | :--- | :--- | :--- | :--- | :--- | :--- |
39
+ | `run_03_best_model.pt` | **Cap On Liability** | Condition A | `44` | 438,001,498 | `8689ef00a0718713f4ae8bdf8b48d3441b5d9a2ccd13ae322185862551c6f674` |
40
+ | `run_04_best_model.pt` | **Anti-Assignment** | Condition A | `42` | 438,001,498 | `ea6a75254798342e0cc3925dd8adca1ba9b5bb60c6a631ddf14057748c01947c` |
41
+ | `run_07_best_model.pt` | **Termination For Convenience** | Condition A | `42` | 438,001,498 | `a8a28902ef3dbc28e706135e8599f01cfa8c0e9c4c05cc081818e3027deda119` |
42
+
43
+ ---
44
+
45
+ ## 3. Frozen Phase 6 Test Evaluation Metrics
46
+
47
+ The reproduced representative checkpoints achieved the following metrics on the held-out frozen CUAD test evaluation set (Phase 6):
48
+
49
+ | Task Name | Representative Checkpoint | Test F1 Score | Test ROC-AUC | Test PR-AUC |
50
+ | :--- | :--- | :--- | :--- | :--- |
51
+ | **Cap On Liability** | `run_03_best_model.pt` | `0.7727273` | `0.9960492` | `0.9147721` |
52
+ | **Anti-Assignment** | `run_04_best_model.pt` | `0.8083624` | `0.9944473` | `0.9146714` |
53
+ | **Termination For Convenience** | `run_07_best_model.pt` | `0.7483871` | `0.9939074` | `0.7926948` |
54
+
55
+ ---
56
+
57
+ ## 4. Inference Score Interpretation
58
+
59
+ In RiskLoop's inference engine and Streamlit demo interface:
60
+ - Output scores are **uncalibrated raw logit deltas** ($\text{logit}_1 - \text{logit}_0$).
61
+ - High positive scores indicate strong model activation for clause presence.
62
+ - Scores are **not** calibrated probabilities or confidence percentages.
63
+
64
+ ---
65
+
66
+ ## 5. Usage & Legal Disclaimer
67
+
68
+ - **Intended Use:** Portfolio evaluation, academic research, and interactive open-source demonstration of contract risk detection.
69
+ - **Legal Disclaimer:** This software and model predictions do **NOT** constitute legal advice, formal contract audit, or legal guarantee. Users should consult qualified legal professionals for actual contract review.