Add infinity as example deployment
#22
by michaelfeil - opened
README.md
CHANGED
|
@@ -2701,6 +2701,14 @@ for dv in doc_vecs:
|
|
| 2701 |
```
|
| 2702 |
|
| 2703 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2704 |
|
| 2705 |
# Citation
|
| 2706 |
|
|
|
|
| 2701 |
```
|
| 2702 |
|
| 2703 |
|
| 2704 |
+
## 3. Infinity
|
| 2705 |
+
|
| 2706 |
+
[Infinity](https://github.com/michaelfeil/infinity) is a MIT licensed server for OpenAI-compatible deployment.
|
| 2707 |
+
```
|
| 2708 |
+
docker run --gpus all -v $PWD/data:/app/.cache -p "7997":"7997" \
|
| 2709 |
+
michaelf34/infinity:0.0.68 \
|
| 2710 |
+
v2 --model-id WhereIsAI/UAE-Large-V1 --revision "369c368f70f16a613f19f5598d4f12d9f44235d4" --dtype float16 --batch-size 32 --device cuda --engine torch --port 7997
|
| 2711 |
+
```
|
| 2712 |
|
| 2713 |
# Citation
|
| 2714 |
|