no parsing of response
Browse files
chains/exercises/run_fluster_with_diagnosis.py
CHANGED
|
@@ -36,15 +36,12 @@ async def parse_fluster_text_to_exercises(fluster_text: str) -> List[Exercise]:
|
|
| 36 |
|
| 37 |
# 3) Call the LLM with structured output
|
| 38 |
# If your version of langchain_openai supports .with_structured_output(ExerciseSet):
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
# 4) Extract the parsed object
|
| 42 |
-
exercise_set = response.choices[0].message.parsed # This should be an ExerciseSet instance
|
| 43 |
|
| 44 |
if exercise_set is None:
|
| 45 |
# If the LLM refused or the format was invalid, we might get None
|
| 46 |
raise ValueError(
|
| 47 |
-
f"LLM refused or returned invalid structured data.\nRaw content:\n{
|
| 48 |
)
|
| 49 |
|
| 50 |
# 5) Return the exercises list
|
|
|
|
| 36 |
|
| 37 |
# 3) Call the LLM with structured output
|
| 38 |
# If your version of langchain_openai supports .with_structured_output(ExerciseSet):
|
| 39 |
+
exercise_set = await llm_structurize.with_structured_output(ExerciseSet).ainvoke(messages)
|
|
|
|
|
|
|
|
|
|
| 40 |
|
| 41 |
if exercise_set is None:
|
| 42 |
# If the LLM refused or the format was invalid, we might get None
|
| 43 |
raise ValueError(
|
| 44 |
+
f"LLM refused or returned invalid structured data.\nRaw content:\n{exercise_set}"
|
| 45 |
)
|
| 46 |
|
| 47 |
# 5) Return the exercises list
|
config/system_prompt_texts.py
CHANGED
|
@@ -637,7 +637,7 @@ An explanation should sometimes be presented to the student after they've answer
|
|
| 637 |
Each of the 3 exercises must test the very same key fact in the given learning objective (the info that's not in parentheses). Any text between parentheses must only be used in the Theory or Explanation sections of the exercises.
|
| 638 |
|
| 639 |
## Prompt
|
| 640 |
-
The information that's posed in the prompt part of the exercise should only contain information that's also present in the learning objective. Do not reference any source outside of it.
|
| 641 |
|
| 642 |
|
| 643 |
## Theory & Explanation (optional)
|
|
@@ -667,7 +667,7 @@ Output format doesn't matter. Only prioritize thorough reasoning to arrive at hi
|
|
| 667 |
Think long and hard about the ideal three exercises to test the given learning objective. Especially spend a lot of time iteratively coming up with good distractors and a good "false statement", to make sure they optimally satisfy the distractors requirements.
|
| 668 |
As intuition pumps:
|
| 669 |
- To guard against picking distractors that are too obviously false: Really try to imagine a relatively dumb student among the target audience for the specific given learning objective. Would they feasible find each distractor at least somewhat plausible?
|
| 670 |
-
- To guard against picking distractors that are too close to the truth: Try to imagine a panel of experts judging the distractors. All of them should agree that the correct answer is clearly the best answer for this exercise, and none of them should doubt
|
| 671 |
If you're unsure about any of your distractors or "false statements" one way or the other, adapt them accordingly, and run the thought experiment again, until you get all distractors just right.
|
| 672 |
After lots of iterative prep, trying out different things and reasoning through a wide range of potential options, finally return a complete exercise set of 1 bigger multiple choice exercise and 2 smaller True/False statements.
|
| 673 |
"""
|
|
|
|
| 637 |
Each of the 3 exercises must test the very same key fact in the given learning objective (the info that's not in parentheses). Any text between parentheses must only be used in the Theory or Explanation sections of the exercises.
|
| 638 |
|
| 639 |
## Prompt
|
| 640 |
+
The information that's posed in the prompt part of the exercise should only contain information that's also present in the learning objective. Do not reference any source outside of it.
|
| 641 |
|
| 642 |
|
| 643 |
## Theory & Explanation (optional)
|
|
|
|
| 667 |
Think long and hard about the ideal three exercises to test the given learning objective. Especially spend a lot of time iteratively coming up with good distractors and a good "false statement", to make sure they optimally satisfy the distractors requirements.
|
| 668 |
As intuition pumps:
|
| 669 |
- To guard against picking distractors that are too obviously false: Really try to imagine a relatively dumb student among the target audience for the specific given learning objective. Would they feasible find each distractor at least somewhat plausible?
|
| 670 |
+
- To guard against picking distractors that are too close to the truth: Try to imagine a panel of experts judging the distractors. All of them should agree that the correct answer is clearly the best answer for this exercise, and none of them should doubt whether any other choices would also be kinda correct.
|
| 671 |
If you're unsure about any of your distractors or "false statements" one way or the other, adapt them accordingly, and run the thought experiment again, until you get all distractors just right.
|
| 672 |
After lots of iterative prep, trying out different things and reasoning through a wide range of potential options, finally return a complete exercise set of 1 bigger multiple choice exercise and 2 smaller True/False statements.
|
| 673 |
"""
|