Hugging Face's logo

peter2000
/
laya-vulnerability-groups

Text Classification
Transformers
Safetensors
Laya
multi-label
climate
vulnerability
rlcd
Model card Files
xet
 Community
peter2000 commited on
Commit
ac55aa8
·
verified ·
1 Parent(s): a112bc2

Laya fine-tuned on GIZ vulnerability data: macro-F1 0.654 (zero-shot 0.108)

Browse files
Files changed (3) hide show
  1. README.md +21 -21
  2. model.safetensors +1 -1
  3. rl_agent_config.json +2 -2
README.md CHANGED
@@ -14,33 +14,33 @@ fine-tuned with the official RLCD recipe on [GIZ/vulnerability_training_data_ful
14
  Each vulnerability group is asked as one binary (`noul`) typed question; all 17 are answered in a single forward pass.
15
  Evaluate with `laya.load("peter2000/laya-vulnerability-groups")` and `agent.predict(state, questions)`.
16
 
17
- ## Test-set metrics (held-out 5 rows, threshold 0.5)
18
 
19
  | metric | value |
20
  |---|---|
21
- | macro-F1 | 0.0000 |
22
- | micro-F1 | 0.0000 |
23
- | ECE | 0.0688 |
24
- | subset accuracy | 0.0000 |
25
 
26
  Per-label F1:
27
 
28
  | label | F1 |
29
  |---|---|
30
- | Agricultural communities | 0.0000 |
31
- | Coastal communities | 0.0000 |
32
- | Ethnic, racial or other minorities | 0.0000 |
33
- | Fishery communities | 0.0000 |
34
- | Informal sector workers | 0.0000 |
35
- | Members of indigenous and local communities | 0.0000 |
36
- | Migrants and displaced persons | 0.0000 |
37
- | Older persons | 0.0000 |
38
  | Other | 0.0000 |
39
- | Persons living in poverty | 0.0000 |
40
- | Persons with disabilities | 0.0000 |
41
- | Persons with pre-existing health conditions | 0.0000 |
42
- | Residents of drought-prone regions | 0.0000 |
43
- | Rural populations | 0.0000 |
44
- | Sexual minorities (LGBTQI+) | 0.0000 |
45
- | Urban populations | 0.0000 |
46
- | Women and other genders | 0.0000 |
 
14
  Each vulnerability group is asked as one binary (`noul`) typed question; all 17 are answered in a single forward pass.
15
  Evaluate with `laya.load("peter2000/laya-vulnerability-groups")` and `agent.predict(state, questions)`.
16
 
17
+ ## Test-set metrics (held-out 95 rows, threshold 0.5)
18
 
19
  | metric | value |
20
  |---|---|
21
+ | macro-F1 | 0.6544 |
22
+ | micro-F1 | 0.6694 |
23
+ | ECE | 0.0263 |
24
+ | subset accuracy | 0.4737 |
25
 
26
  Per-label F1:
27
 
28
  | label | F1 |
29
  |---|---|
30
+ | Agricultural communities | 0.9565 |
31
+ | Coastal communities | 0.5000 |
32
+ | Ethnic, racial or other minorities | 0.6000 |
33
+ | Fishery communities | 0.4000 |
34
+ | Informal sector workers | 1.0000 |
35
+ | Members of indigenous and local communities | 0.9333 |
36
+ | Migrants and displaced persons | 0.5714 |
37
+ | Older persons | 0.8889 |
38
  | Other | 0.0000 |
39
+ | Persons living in poverty | 0.3333 |
40
+ | Persons with disabilities | 1.0000 |
41
+ | Persons with pre-existing health conditions | 0.8000 |
42
+ | Residents of drought-prone regions | 0.5714 |
43
+ | Rural populations | 0.8000 |
44
+ | Sexual minorities (LGBTQI+) | 0.3333 |
45
+ | Urban populations | 0.5714 |
46
+ | Women and other genders | 0.8649 |
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a87929b1f9599ac659d97a89662a6f8806ff5ae15ad1c1e27e89e55ae24187e4
3
  size 842609220
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:42a9aa146309525f3635caa14f437d3da4f8c43ed9939c5a7dc3c15f1cc7b6ee
3
  size 842609220
rl_agent_config.json CHANGED
@@ -13,13 +13,13 @@
13
  "temperature": [
14
  1.2,
15
  1.2,
16
- 1.0
17
  ],
18
  "temperature_by_options": {
19
  "choice:3-5": 1.7601518630981445,
20
  "choice:6-10": 1.0000158548355103,
21
  "score:3-5": 1.2514300346374512,
22
- "noul:2": 1.0,
23
  "choice:11+": 0.10058280825614929,
24
  "choice:2": 1.9063563346862793
25
  },
 
13
  "temperature": [
14
  1.2,
15
  1.2,
16
+ 4.928757667541504
17
  ],
18
  "temperature_by_options": {
19
  "choice:3-5": 1.7601518630981445,
20
  "choice:6-10": 1.0000158548355103,
21
  "score:3-5": 1.2514300346374512,
22
+ "noul:2": 4.928757667541504,
23
  "choice:11+": 0.10058280825614929,
24
  "choice:2": 1.9063563346862793
25
  },