Add KV caching to inference, ~5.9x average speedup on CPU

Update app.py

Update app.py

Update requirements.txt

Update app.py

Create app.py

Create requirements.txt

initial commit