Spaces:

ibm-granite
/

granite-3.1-8b-instruct

Running on Zero

grahamwhiteuk commited on Jan 17

Commit

35e6309

1 Parent(s): 8d34f84

feat: remove timeeout on streamer

Signed-off-by: Graham White <gwhite@uk.ibm.com>

Files changed (1) hide show

src/app.py CHANGED Viewed

@@ -72,7 +72,7 @@ def generate(
     )
     input_ids = input_ids.to(model.device)
-    streamer = TextIteratorStreamer(tokenizer, timeout=30.0, skip_prompt=True, skip_special_tokens=True)
     generate_kwargs = dict(
         {"input_ids": input_ids},
         streamer=streamer,

     )
     input_ids = input_ids.to(model.device)
+    streamer = TextIteratorStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
     generate_kwargs = dict(
         {"input_ids": input_ids},
         streamer=streamer,