Shen-Pandi commited on
Commit
5492f2a
·
verified ·
1 Parent(s): baa6ff8

Update model card for Safetensors and vLLM

Browse files
Files changed (1) hide show
  1. README.md +34 -2
README.md CHANGED
@@ -1,7 +1,39 @@
1
  ---
2
  language: en
3
  tags:
4
- - mlx
5
- library_name: mlx
 
 
 
6
  pipeline_tag: text-generation
7
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  language: en
3
  tags:
4
+ - safetensors
5
+ - vllm
6
+ - insurance
7
+ - reinsurance
8
+ library_name: transformers
9
  pipeline_tag: text-generation
10
  ---
11
+
12
+ # Reinsure-8B (vLLM Compatible)
13
+
14
+ This is a fine-tuned version of Meta's Llama 3.1 8B Instruct model, specifically trained on over 14,000 synthetic and real-world instruction pairs covering Reinsurance and Insurance domains (Catastrophe Modeling, Treaty Structures, Facultative documents, and regulatory compliance).
15
+
16
+ ## Format
17
+ This model has been successfully dequantized and exported to **16-bit Safetensors** format. It is fully compatible with any standard cloud inference engine, including **vLLM** and HuggingFace Text Generation Inference (TGI).
18
+
19
+ ## Deployment
20
+
21
+ This model is intended for deployment on serverless GPU architectures (like **Modal**) using `vLLM` to act as a highly scalable context engine for an API gateway.
22
+
23
+ ### Example vLLM Deployment (Modal)
24
+ ```python
25
+ import modal
26
+
27
+ vllm_image = modal.Image.debian_slim().pip_install("vllm")
28
+ app = modal.App("reinsure-8b")
29
+
30
+ @app.function(image=vllm_image, gpu="L4", container_idle_timeout=300)
31
+ @modal.web_server(8000)
32
+ def serve():
33
+ import sys, subprocess
34
+ subprocess.Popen([
35
+ sys.executable, "-m", "vllm.entrypoints.openai.api_server",
36
+ "--model", "Reinsured-AI/Reinsure-8B",
37
+ "--port", "8000"
38
+ ])
39
+ ```