Cleanups

Customizable system prompt

Streaming output

Do hacky thread limit to stop both models from running at once

Limit concurency

Trying to work out concurency

Manage long inputs and outputs

Do instruct models for 2.7 and 7 b sizes

instruct and base chat types

Expose both base and instruct models

Add cuda and sampling pram

Fix gradio build erros

Torch with cuda

Try pre-requirements.txt for installs

Just requirements install

Try two stage install requirements+commands

Install through commands section

Try installing repo through requiremnts.txt

Init comit to test zamba2 7b build scripts

initial commit