BtB-ExpC commited on
Commit
0c92067
·
1 Parent(s): 25f6928

no parsing of response

Browse files
chains/exercises/run_fluster_with_diagnosis.py CHANGED
@@ -36,15 +36,12 @@ async def parse_fluster_text_to_exercises(fluster_text: str) -> List[Exercise]:
36
 
37
  # 3) Call the LLM with structured output
38
  # If your version of langchain_openai supports .with_structured_output(ExerciseSet):
39
- response = await llm_structurize.with_structured_output(ExerciseSet).ainvoke(messages)
40
-
41
- # 4) Extract the parsed object
42
- exercise_set = response.choices[0].message.parsed # This should be an ExerciseSet instance
43
 
44
  if exercise_set is None:
45
  # If the LLM refused or the format was invalid, we might get None
46
  raise ValueError(
47
- f"LLM refused or returned invalid structured data.\nRaw content:\n{response.choices[0].message.content}"
48
  )
49
 
50
  # 5) Return the exercises list
 
36
 
37
  # 3) Call the LLM with structured output
38
  # If your version of langchain_openai supports .with_structured_output(ExerciseSet):
39
+ exercise_set = await llm_structurize.with_structured_output(ExerciseSet).ainvoke(messages)
 
 
 
40
 
41
  if exercise_set is None:
42
  # If the LLM refused or the format was invalid, we might get None
43
  raise ValueError(
44
+ f"LLM refused or returned invalid structured data.\nRaw content:\n{exercise_set}"
45
  )
46
 
47
  # 5) Return the exercises list
config/system_prompt_texts.py CHANGED
@@ -637,7 +637,7 @@ An explanation should sometimes be presented to the student after they've answer
637
  Each of the 3 exercises must test the very same key fact in the given learning objective (the info that's not in parentheses). Any text between parentheses must only be used in the Theory or Explanation sections of the exercises.
638
 
639
  ## Prompt
640
- The information that's posed in the prompt part of the exercise should only contain information that's also present in the learning objective. Do not reference any source outside of it. See the below examples, both for the learning objective:
641
 
642
 
643
  ## Theory & Explanation (optional)
@@ -667,7 +667,7 @@ Output format doesn't matter. Only prioritize thorough reasoning to arrive at hi
667
  Think long and hard about the ideal three exercises to test the given learning objective. Especially spend a lot of time iteratively coming up with good distractors and a good "false statement", to make sure they optimally satisfy the distractors requirements.
668
  As intuition pumps:
669
  - To guard against picking distractors that are too obviously false: Really try to imagine a relatively dumb student among the target audience for the specific given learning objective. Would they feasible find each distractor at least somewhat plausible?
670
- - To guard against picking distractors that are too close to the truth: Try to imagine a panel of experts judging the distractors. All of them should agree that the correct answer is clearly the best answer for this exercise, and none of them should doubt that any other choices would also be kinda correct.
671
  If you're unsure about any of your distractors or "false statements" one way or the other, adapt them accordingly, and run the thought experiment again, until you get all distractors just right.
672
  After lots of iterative prep, trying out different things and reasoning through a wide range of potential options, finally return a complete exercise set of 1 bigger multiple choice exercise and 2 smaller True/False statements.
673
  """
 
637
  Each of the 3 exercises must test the very same key fact in the given learning objective (the info that's not in parentheses). Any text between parentheses must only be used in the Theory or Explanation sections of the exercises.
638
 
639
  ## Prompt
640
+ The information that's posed in the prompt part of the exercise should only contain information that's also present in the learning objective. Do not reference any source outside of it.
641
 
642
 
643
  ## Theory & Explanation (optional)
 
667
  Think long and hard about the ideal three exercises to test the given learning objective. Especially spend a lot of time iteratively coming up with good distractors and a good "false statement", to make sure they optimally satisfy the distractors requirements.
668
  As intuition pumps:
669
  - To guard against picking distractors that are too obviously false: Really try to imagine a relatively dumb student among the target audience for the specific given learning objective. Would they feasible find each distractor at least somewhat plausible?
670
+ - To guard against picking distractors that are too close to the truth: Try to imagine a panel of experts judging the distractors. All of them should agree that the correct answer is clearly the best answer for this exercise, and none of them should doubt whether any other choices would also be kinda correct.
671
  If you're unsure about any of your distractors or "false statements" one way or the other, adapt them accordingly, and run the thought experiment again, until you get all distractors just right.
672
  After lots of iterative prep, trying out different things and reasoning through a wide range of potential options, finally return a complete exercise set of 1 bigger multiple choice exercise and 2 smaller True/False statements.
673
  """