Update instructions for usage with infinity
#13
by
michaelfeil
- opened
README.md
CHANGED
|
@@ -8924,6 +8924,14 @@ print(scores.tolist())
|
|
| 8924 |
# [[55.92064666748047, 1.6549524068832397], [-0.2698777914047241, 49.95653533935547]]
|
| 8925 |
```
|
| 8926 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8927 |
|
| 8928 |
## Evaluation
|
| 8929 |
|
|
|
|
| 8924 |
# [[55.92064666748047, 1.6549524068832397], [-0.2698777914047241, 49.95653533935547]]
|
| 8925 |
```
|
| 8926 |
|
| 8927 |
+
### Usage with infinity
|
| 8928 |
+
|
| 8929 |
+
Instructions for usage with [Infinity](https://github.com/michaelfeil/infinity)
|
| 8930 |
+
```bash
|
| 8931 |
+
docker run --gpus all -v $PWD/data:/app/.cache -e HF_TOKEN=$HF_TOKEN -p "7997":"7997" \
|
| 8932 |
+
michaelf34/infinity:0.0.68 \
|
| 8933 |
+
v2 --model-id BAAI/bge-multilingual-gemma2 --revision "b67c20b19cabf41f74b2aa1469047dade1f42738" --dtype bfloat16 --batch-size 4 --device cuda --engine torch --port 7997 --no-bettertransformer
|
| 8934 |
+
```
|
| 8935 |
|
| 8936 |
## Evaluation
|
| 8937 |
|