added the cap @ 16 learning objectives
Browse files- config/llm_config.py +2 -1
- config/system_prompt_texts.py +8 -6
- test samples.md +5 -2
config/llm_config.py
CHANGED
|
@@ -44,7 +44,8 @@ llms = {
|
|
| 44 |
"GPT-4 Turbo (low temp)": create_openai_llm("gpt-4-turbo-2024-04-09", LOW),
|
| 45 |
|
| 46 |
# OpenAI reasoning models (no temperature)
|
| 47 |
-
"o1": create_openai_reasoning_llm("o1-2024-12-17", reasoning_effort="
|
|
|
|
| 48 |
"o3-mini (low reasoning_effort)": create_openai_reasoning_llm("o3-mini", reasoning_effort="low"),
|
| 49 |
"o3-mini (medium reasoning_effort)": create_openai_reasoning_llm("o3-mini", reasoning_effort="medium"),
|
| 50 |
"o3-mini (high reasoning_effort)": create_openai_reasoning_llm("o3-mini", reasoning_effort="high"),
|
|
|
|
| 44 |
"GPT-4 Turbo (low temp)": create_openai_llm("gpt-4-turbo-2024-04-09", LOW),
|
| 45 |
|
| 46 |
# OpenAI reasoning models (no temperature)
|
| 47 |
+
"o1 (low reasoning_effort)": create_openai_reasoning_llm("o1-2024-12-17", reasoning_effort="low"),
|
| 48 |
+
"o1 (high reasoning_effort)": create_openai_reasoning_llm("o1-2024-12-17", reasoning_effort="high"),
|
| 49 |
"o3-mini (low reasoning_effort)": create_openai_reasoning_llm("o3-mini", reasoning_effort="low"),
|
| 50 |
"o3-mini (medium reasoning_effort)": create_openai_reasoning_llm("o3-mini", reasoning_effort="medium"),
|
| 51 |
"o3-mini (high reasoning_effort)": create_openai_reasoning_llm("o3-mini", reasoning_effort="high"),
|
config/system_prompt_texts.py
CHANGED
|
@@ -273,10 +273,10 @@ Before extracting learning objectives:
|
|
| 273 |
</examples>
|
| 274 |
|
| 275 |
### Content Quality
|
| 276 |
-
* **Self-contained**: Must be understandable without relying on any outside context
|
| 277 |
-
* **Falsifiable**: Must be unambiguously, demonstrably true or false
|
| 278 |
* **Factually Equivalent**: Must represent exactly the knowledge as written
|
| 279 |
-
* **Specific**: Must express the smallest coherent, testable knowledge unit, instead of several things at once
|
| 280 |
|
| 281 |
<examples>
|
| 282 |
<self_contained_example>
|
|
@@ -299,7 +299,9 @@ Before extracting learning objectives:
|
|
| 299 |
</examples>
|
| 300 |
|
| 301 |
### Additional notes
|
| 302 |
-
*
|
|
|
|
|
|
|
| 303 |
|
| 304 |
### Language Precision
|
| 305 |
* Avoid universal terms ("always", "never") unless 100% accurate, and there are in fact no exceptions
|
|
@@ -308,7 +310,7 @@ Before extracting learning objectives:
|
|
| 308 |
* Use "important" only when there is no other option to say something more meaningful ("X is important" doesn't say much)
|
| 309 |
|
| 310 |
## Quality Assurance Process
|
| 311 |
-
1. Extract all potential learning objectives
|
| 312 |
2. For each objective, verify it meets ALL requirements
|
| 313 |
3. Refine and potentially split objectives until each one is:
|
| 314 |
- Maximally specific
|
|
@@ -337,7 +339,7 @@ Before extracting learning objectives:
|
|
| 337 |
- De student weet dat je dagelijks oefent om zo objectief (zonder je eigen mening) mogelijk te observeren.
|
| 338 |
- De student weet dat je aan de zorgvrager vraagt of jouw observatie klopt.
|
| 339 |
</better>
|
| 340 |
-
<explanation>Now they are stated mirroring the language of the text, without introducing new, potentially difficult words.</explanation>
|
| 341 |
</language_level_examples>
|
| 342 |
</examples>
|
| 343 |
|
|
|
|
| 273 |
</examples>
|
| 274 |
|
| 275 |
### Content Quality
|
| 276 |
+
* **Self-contained**: Must be understandable without relying on any outside context, like 'previous' learning objectives (they will in fact be presented to the student non-chronologically, so cannot build on each other)
|
| 277 |
+
* **Falsifiable**: Must be unambiguously, demonstrably true or false
|
| 278 |
* **Factually Equivalent**: Must represent exactly the knowledge as written
|
| 279 |
+
* **Specific**: Must express the smallest coherent, testable knowledge unit, instead of several things at once. For composite knowledge, like
|
| 280 |
|
| 281 |
<examples>
|
| 282 |
<self_contained_example>
|
|
|
|
| 299 |
</examples>
|
| 300 |
|
| 301 |
### Additional notes
|
| 302 |
+
* If a term is explained in the text, and it's central to the topic, knowing what it means should also become a learning objective
|
| 303 |
+
* If a term is explained in the text, also explain it briefly (between parentheses) in each learning objective that uses it
|
| 304 |
+
* If the text is so knowledge dense that it contains more than 16 potential learning objectives, cap them at around that number and eliminate the less important ones. For non-hierarchical learning objectives (all equally important), just trim randomly
|
| 305 |
|
| 306 |
### Language Precision
|
| 307 |
* Avoid universal terms ("always", "never") unless 100% accurate, and there are in fact no exceptions
|
|
|
|
| 310 |
* Use "important" only when there is no other option to say something more meaningful ("X is important" doesn't say much)
|
| 311 |
|
| 312 |
## Quality Assurance Process
|
| 313 |
+
1. Extract all potential learning objectives (up to ~16 max)
|
| 314 |
2. For each objective, verify it meets ALL requirements
|
| 315 |
3. Refine and potentially split objectives until each one is:
|
| 316 |
- Maximally specific
|
|
|
|
| 339 |
- De student weet dat je dagelijks oefent om zo objectief (zonder je eigen mening) mogelijk te observeren.
|
| 340 |
- De student weet dat je aan de zorgvrager vraagt of jouw observatie klopt.
|
| 341 |
</better>
|
| 342 |
+
<explanation>Now they are stated mirroring the language of the text, without introducing new, potentially difficult words. Words that are explained in the text, need to be explained in each learning objective that reuses it between parentheses as well.</explanation>
|
| 343 |
</language_level_examples>
|
| 344 |
</examples>
|
| 345 |
|
test samples.md
CHANGED
|
@@ -92,10 +92,13 @@ Het hart bestaat uit vier holtes: twee boezems aan de bovenkant en twee kamers a
|
|
| 92 |
- De student weet dat de boezems van het hart aan de bovenkant zitten.
|
| 93 |
- De student weet dat het hart uit twee kamers bestaat.
|
| 94 |
- De student weet dat de kamers van het hart aan de onderkant zitten.
|
| 95 |
-
##
|
| 96 |
- De student weet dat als je een onderbuikgevoel krijgt, je hierop moet letten, moet onderzoeken waar het vandaan komt en het met collega’s moet bespreken.
|
| 97 |
-
|
|
|
|
| 98 |
- De student weet dat de Wet Bopz sinds 1994 in gebruik was.
|
|
|
|
|
|
|
| 99 |
|
| 100 |
|
| 101 |
|
|
|
|
| 92 |
- De student weet dat de boezems van het hart aan de bovenkant zitten.
|
| 93 |
- De student weet dat het hart uit twee kamers bestaat.
|
| 94 |
- De student weet dat de kamers van het hart aan de onderkant zitten.
|
| 95 |
+
## Require "Theorie"
|
| 96 |
- De student weet dat als je een onderbuikgevoel krijgt, je hierop moet letten, moet onderzoeken waar het vandaan komt en het met collega’s moet bespreken.
|
| 97 |
+
- De student weet dat je dagelijks oefent om zo objectief (zonder je eigen mening) mogelijk te observeren.
|
| 98 |
+
## Must be tested approximately (not with 1 year precision)
|
| 99 |
- De student weet dat de Wet Bopz sinds 1994 in gebruik was.
|
| 100 |
+
## Require Extra info (explanation)
|
| 101 |
+
- De student weet dat de Wet Bopz de rechten en plichten van onvrijwillig opgenomen zorgvragers regelde, zoals het behandelplan, verlof en juridische bijstand.
|
| 102 |
|
| 103 |
|
| 104 |
|