LadyPary commited on
Commit
d85a720
·
verified ·
1 Parent(s): e4dd67a

Update newDemo.py

Browse files
Files changed (1) hide show
  1. newDemo.py +5 -51
newDemo.py CHANGED
@@ -18,53 +18,7 @@ generation_config = genai.GenerationConfig(temperature=0.5)
18
  # Model configuration
19
  model = genai.GenerativeModel(
20
  model_name='gemini-1.5-pro-latest',
21
- system_instruction= """
22
- Answer the user based on these questions and answers.
23
- Give a detailed answer with timestamps.
24
- End your responses with a some examples of follow up question they can ask you.
25
-
26
- Question: 01:15-01:35 is an example of ISI.
27
- Model Answer: Got it. I observe the following behaviors within 01:15-01:35:
28
- It seems to be a turn-taking (e.g., putting a hat on the adult’s head) behavior occuring.
29
- Question: please update this behavior codes to ISI’s definition.
30
- Model Answer: understood!
31
- -------------------------------------------------------
32
- Question: What nonverbal and/or verbal behavior does the child use to initiate a behavioral request [IBR] for a toy at the beginning of the interaction with the examiner?
33
- ---
34
- Model Answer: The child initiates a behavioral request for a toy at the beginning of the interaction with the examiner by looking at the toy and then at the examiner, indicating a desire for the toy.
35
- -------------------------------------------------------
36
- Question: What toy did the child IBR for?
37
- ---
38
- Model Answer: The child initiates a behavioral request for a wind-up toy in the video.
39
- -------------------------------------------------------
40
- Question: How many times does the child IBR throughout her interaction with the examiner, after the initial IBR for a specific toy at the beginning of the interaction?
41
- ---
42
- Model Answer: The child IBRs three times throughout her interaction with the examiner after the initial IBR for a specific toy at the beginning of the interaction.
43
- -------------------------------------------------------
44
- Question: How many times does the child use a single behavior to IBR? What specific behaviors are used (e.g., eye contact, reach) for each IBR? Can you identify the timestamp for each instance?
45
- ---
46
- Model Answer: The child uses a single behavior to IBR three times. The specific behaviors used are:
47
- 1. eye contact: 0 seconds 29 seconds The child looks from the toy to the instructor and back to the toy.
48
- 2. language: 31 seconds 37 seconds The child reaches for the toy and looks at the instructor.
49
- 3. reach: 86 seconds 123 seconds The child gives the toy to the instructor and reaches for it again.
50
- -------------------------------------------------------
51
- Question: How many times does the child initiate joint attention (IJA) during her interaction with the examiner?
52
- ---
53
- Model Answer: The child initiates joint attention (IJA) two times during her interaction with the examiner between 82 seconds and 123 seconds . The first time is when she looks at the toy and then at the examiner, and the second time is when she asks for more of the toy.
54
- -------------------------------------------------------
55
- Question: How many times does the child use a single behavior to IJA with the examiner? What specific behaviors are used for each IJA (e.g., alternating gaze)? Can you identify the timestamp for each instance?
56
- ---
57
- Model Answer: The child uses a single behavior to IJA with the examiner three times. The specific behaviors used for each IJA are alternating gaze, coordinated joint look, and pointing. The timestamps for each instance are: 0 seconds 29 seconds , 31 seconds 123 seconds , and 91 seconds 123 seconds .
58
- -------------------------------------------------------
59
- Question: How many times does the child pair two behaviors to IJA with the examiner? What specific behaviors are paired for each IJA (e.g., speech and coordinated joint look)? Can you identify the timestamp for each instance?
60
- ---
61
- Model Answer: The child pairs two behaviors to IJA with the examiner four times. The specific behaviors paired for each IJA are: 1) speech and coordinated joint look, 2) speech and pointing, 3) speech and giving, and 4) speech and showing. The timestamps for each instance are: 40 seconds 123 seconds
62
- -------------------------------------------------------
63
- """
64
- )
65
-
66
- '''
67
- system_instruction= """
68
  You are an assistant chatbot for a Speech Language Pathologist (SLP).
69
  Your task is to help analyze a provided video of a therapy session and answer questions accurately.
70
  Provide timestamps for specific events or behaviors mentioned. Conclude each response with possible follow-up questions.
@@ -78,7 +32,8 @@ Model Answer: The child pairs two behaviors to IJA with the examiner four times.
78
  4. If the user asks about a behavior, respond with, “My understanding of this behavior is [xxx - AI generated output]. Is this a behavior that you want to track? If it is, please define this behavior and tell me more about it so I can analyze it more accurately according to your practice.”
79
  5. If you receive names, confirm that these are the names of the people from left to right.
80
  """
81
- '''
 
82
 
83
 
84
  '''
@@ -118,8 +73,7 @@ def new_prompt(prompt):
118
  # Append user prompt to chat history
119
  chat_history.append({'role': 'user', 'parts': [prompt]})
120
  try:
121
- response = model.generate_content(chat_history)
122
- '''
123
  if video_file:
124
  # Video exists and is processed
125
  chat_history[-1]['parts'].extend([" from video: ", video_file])
@@ -127,7 +81,7 @@ def new_prompt(prompt):
127
  else:
128
  # No video uploaded yet
129
  response = model.generate_content(chat_history)
130
- '''
131
  # Extract the text content from the response and append it to the chat history
132
  assistant_message = response.candidates[0].content.parts[0].text
133
  chat_history.append({'role': 'model', 'parts': [assistant_message]})
 
18
  # Model configuration
19
  model = genai.GenerativeModel(
20
  model_name='gemini-1.5-pro-latest',
21
+ system_instruction= """
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
  You are an assistant chatbot for a Speech Language Pathologist (SLP).
23
  Your task is to help analyze a provided video of a therapy session and answer questions accurately.
24
  Provide timestamps for specific events or behaviors mentioned. Conclude each response with possible follow-up questions.
 
32
  4. If the user asks about a behavior, respond with, “My understanding of this behavior is [xxx - AI generated output]. Is this a behavior that you want to track? If it is, please define this behavior and tell me more about it so I can analyze it more accurately according to your practice.”
33
  5. If you receive names, confirm that these are the names of the people from left to right.
34
  """
35
+ )
36
+
37
 
38
 
39
  '''
 
73
  # Append user prompt to chat history
74
  chat_history.append({'role': 'user', 'parts': [prompt]})
75
  try:
76
+
 
77
  if video_file:
78
  # Video exists and is processed
79
  chat_history[-1]['parts'].extend([" from video: ", video_file])
 
81
  else:
82
  # No video uploaded yet
83
  response = model.generate_content(chat_history)
84
+
85
  # Extract the text content from the response and append it to the chat history
86
  assistant_message = response.candidates[0].content.parts[0].text
87
  chat_history.append({'role': 'model', 'parts': [assistant_message]})