Instructions to use Sophia-AI/RegTech-14B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Sophia-AI/RegTech-14B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Sophia-AI/RegTech-14B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Sophia-AI/RegTech-14B-Instruct")
model = AutoModelForCausalLM.from_pretrained("Sophia-AI/RegTech-14B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Sophia-AI/RegTech-14B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Sophia-AI/RegTech-14B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sophia-AI/RegTech-14B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Sophia-AI/RegTech-14B-Instruct

SGLang

How to use Sophia-AI/RegTech-14B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Sophia-AI/RegTech-14B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sophia-AI/RegTech-14B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Sophia-AI/RegTech-14B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sophia-AI/RegTech-14B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Sophia-AI/RegTech-14B-Instruct with Docker Model Runner:
```
docker model run hf.co/Sophia-AI/RegTech-14B-Instruct
```

MwSpace commited on Feb 16

Commit

14a198b

verified ·

1 Parent(s): f57c0d6

Upload RegTech-14B-Instruct

Browse files

Files changed (7) hide show

model-00001-of-00006.safetensors +1 -1
model-00002-of-00006.safetensors +1 -1
model-00003-of-00006.safetensors +1 -1
model-00004-of-00006.safetensors +1 -1
model-00005-of-00006.safetensors +1 -1
model-00006-of-00006.safetensors +1 -1
training_report.json +160 -249

model-00001-of-00006.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d60197165349792044d5e1489bbee9ad7e388190555d1659274f1901c55e76d1
 size 4986211280

 version https://git-lfs.github.com/spec/v1
+oid sha256:43fc26c9c5311bfac0ffb290b01589fac271b941da368e3873bc4b1d81e63bc6
 size 4986211280

model-00002-of-00006.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:996e83ae5f630926a0b9382d0651752fe4ce2e5aeb3e293dae14bbdbbf93d494
 size 4954847344

 version https://git-lfs.github.com/spec/v1
+oid sha256:f6ab33ca80c5d2a528877030b9f83c5b84c485b3fd5771a0da39fff3bf2da794
 size 4954847344

model-00003-of-00006.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9f66874e7a940286c158687f731db691dd605a311b0386b099f3af1d8f27ef22
 size 4954847392

 version https://git-lfs.github.com/spec/v1
+oid sha256:b1d255cf1f2bb92712e2d9a8cdab57f8f1752b4c9a2bb2c0e538a0d48c0fa424
 size 4954847392

model-00004-of-00006.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fd5ef99189d0880d8c9334412c09b4478b82547d405e7b11dc900ea173d51fb7
 size 4954847392

 version https://git-lfs.github.com/spec/v1
+oid sha256:5ef6318d2f290564f77ed5414b262e3964b98d26133e1320733ee4a8039cb9ff
 size 4954847392

model-00005-of-00006.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2b6eb5179e0987bbf7c7774737f273ede16f6f542da4d1db39264b4547d08e2a
 size 4954847392

 version https://git-lfs.github.com/spec/v1
+oid sha256:d6cfc7a2def3f023d31c9c67340113053b6df83aa6ef4f2598591f1c12ebba06
 size 4954847392

model-00006-of-00006.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b16bb22f531dc046f89183e79d82ee7bbee454afa6568d439a61942fb97bae24
 size 4734533160

 version https://git-lfs.github.com/spec/v1
+oid sha256:1997e8d1ba66d7c2b4283582c2f8b054eba6f410b402fa8add05456dbdece9a0
 size 4734533160

training_report.json CHANGED Viewed

@@ -3,15 +3,15 @@
   "model_name": "RegTech-14B-Instruct",
   "dataset": "./train.jsonl",
   "env_file": "/home/ubuntu/sophia-core-server/.tuning/.env.14B",
-  "train_samples": 923,
-  "eval_samples": 102,
   "params": {
     "rank": 16,
-    "alpha": 32,
     "dropout": 0.1,
-    "lr": 5e-06,
     "scheduler": "cosine",
-    "epochs": 3,
     "effective_batch": 4,
     "max_seq_length": 4096,
     "neftune_alpha": 5.0,
@@ -26,635 +26,546 @@
     ]
   },
   "results": {
-    "total_steps": 693,
-    "final_train_loss": 1.1265,
-    "best_eval_loss": 1.2247475385665894,
-    "best_eval_step": 640,
-    "elapsed_minutes": 23.5
   },
   "loss_history": {
     "train": [
       [
         5,
-        1.7295
       ],
       [
         10,
-        1.802
       ],
       [
         15,
-        2.0237
       ],
       [
         20,
-        1.7917
       ],
       [
         25,
-        2.0068
       ],
       [
         30,
-        1.9094
       ],
       [
         35,
-        1.8299
       ],
       [
         40,
-        1.7277
       ],
       [
         45,
-        1.8172
       ],
       [
         50,
-        1.7058
       ],
       [
         55,
-        1.3853
       ],
       [
         60,
-        1.7634
       ],
       [
         65,
-        1.5767
       ],
       [
         70,
-        1.754
       ],
       [
         75,
-        1.7128
       ],
       [
         80,
-        1.5807
       ],
       [
         85,
-        1.5355
       ],
       [
         90,
-        1.4244
       ],
       [
         95,
-        1.5826
       ],
       [
         100,
-        1.5446
       ],
       [
         105,
-        1.4907
       ],
       [
         110,
-        1.432
       ],
       [
         115,
-        1.5543
       ],
       [
         120,
-        1.3322
       ],
       [
         125,
-        1.2579
       ],
       [
         130,
-        1.7049
       ],
       [
         135,
-        1.2563
       ],
       [
         140,
-        1.6188
       ],
       [
         145,
-        1.3941
       ],
       [
         150,
-        1.456
       ],
       [
         155,
-        1.5397
       ],
       [
         160,
-        1.3587
       ],
       [
         165,
-        1.4827
       ],
       [
         170,
-        1.4644
       ],
       [
         175,
-        1.1337
       ],
       [
         180,
-        1.302
       ],
       [
         185,
-        1.4177
       ],
       [
         190,
-        1.3671
       ],
       [
         195,
-        1.3418
       ],
       [
         200,
-        1.0722
       ],
       [
         205,
-        1.315
       ],
       [
         210,
-        1.1977
       ],
       [
         215,
-        1.2221
       ],
       [
         220,
-        1.1184
       ],
       [
         225,
-        1.282
       ],
       [
         230,
-        1.1874
       ],
       [
         235,
-        1.4661
       ],
       [
         240,
-        1.1914
       ],
       [
         245,
-        1.1856
       ],
       [
         250,
-        1.0897
       ],
       [
         255,
-        1.1284
       ],
       [
         260,
-        1.27
       ],
       [
         265,
-        1.378
       ],
       [
         270,
-        1.1756
       ],
       [
         275,
-        1.1799
       ],
       [
         280,
-        1.3092
       ],
       [
         285,
-        1.3576
       ],
       [
         290,
-        1.2254
       ],
       [
         295,
-        1.076
       ],
       [
         300,
-        1.0677
       ],
       [
         305,
-        1.3067
       ],
       [
         310,
-        1.1462
       ],
       [
         315,
-        1.2189
       ],
       [
         320,
-        1.1138
       ],
       [
         325,
-        1.2075
       ],
       [
         330,
-        1.1388
       ],
       [
         335,
-        1.1792
       ],
       [
         340,
-        1.1284
       ],
       [
         345,
-        1.3012
       ],
       [
         350,
-        0.9941
       ],
       [
         355,
-        1.1375
       ],
       [
         360,
-        1.2087
       ],
       [
         365,
-        1.2662
       ],
       [
         370,
-        1.2091
       ],
       [
         375,
-        1.0686
       ],
       [
         380,
-        0.9329
       ],
       [
         385,
-        1.0478
       ],
       [
         390,
-        1.0606
       ],
       [
         395,
-        1.0607
       ],
       [
         400,
-        1.1947
       ],
       [
         405,
-        1.158
       ],
       [
         410,
-        1.2246
       ],
       [
         415,
-        1.0888
       ],
       [
         420,
-        1.1178
       ],
       [
         425,
-        1.0749
       ],
       [
         430,
-        1.3176
       ],
       [
         435,
-        1.0919
       ],
       [
         440,
-        1.0837
       ],
       [
         445,
-        0.9845
       ],
       [
         450,
-        1.0481
       ],
       [
         455,
-        1.1088
       ],
       [
         460,
-        0.9743
       ],
       [
         465,
-        1.3255
       ],
       [
         470,
-        1.0583
       ],
       [
         475,
-        1.1048
       ],
       [
         480,
-        0.9751
       ],
       [
         485,
-        0.9574
       ],
       [
         490,
-        1.1434
       ],
       [
         495,
-        1.2023
       ],
       [
         500,
-        1.1073
       ],
       [
         505,
-        1.053
       ],
       [
         510,
-        1.1742
       ],
       [
         515,
-        0.9444
       ],
       [
         520,
-        1.088
       ],
       [
         525,
-        1.0151
       ],
       [
         530,
-        1.0707
-      ],
-      [
-        535,
-        1.0992
-      ],
-      [
-        540,
-        1.0729
-      ],
-      [
-        545,
-        0.9759
-      ],
-      [
-        550,
-        1.0995
-      ],
-      [
-        555,
-        1.1917
-      ],
-      [
-        560,
-        1.0671
-      ],
-      [
-        565,
-        1.1079
-      ],
-      [
-        570,
-        0.8946
-      ],
-      [
-        575,
-        1.0232
-      ],
-      [
-        580,
-        1.0634
-      ],
-      [
-        585,
-        1.0558
-      ],
-      [
-        590,
-        1.1866
-      ],
-      [
-        595,
-        0.9822
-      ],
-      [
-        600,
-        1.0456
-      ],
-      [
-        605,
-        0.8698
-      ],
-      [
-        610,
-        1.1024
-      ],
-      [
-        615,
-        1.1079
-      ],
-      [
-        620,
-        1.0469
-      ],
-      [
-        625,
-        1.0726
-      ],
       [
-        630,
-        1.0963
       ],
       [
-        635,
-        1.0431
       ],
       [
-        640,
-        1.0866
       ],
       [
-        645,
-        1.0284
       ],
       [
-        650,
-        1.2035
       ],
       [
-        655,
-        1.0262
       ],
       [
-        660,
-        0.9705
       ],
       [
-        665,
-        1.0958
       ],
       [
-        670,
-        1.1898
       ],
       [
-        675,
-        1.0921
       ],
       [
-        680,
-        0.9948
       ],
       [
-        685,
-        1.1798
       ],
       [
-        690,
-        1.1265
       ]
     ],
-    "eval": [
       [
         40,
-        2.2252511978149414
       ],
       [
         80,
-        1.9501206874847412
       ],
       [
         120,
-        1.708126187324524
       ],
       [
         160,
-        1.5921595096588135
       ],
       [
         200,
-        1.513823390007019
       ],
       [
         240,
-        1.4531652927398682
       ],
       [
         280,
-        1.400567650794983
       ],
       [
         320,
-        1.356279969215393
       ],
       [
         360,
-        1.3178586959838867
       ],
       [
         400,
-        1.288638949394226
       ],
       [
         440,
-        1.2657065391540527
       ],
       [
         480,
-        1.250612735748291
       ],
       [
         520,
-        1.2387887239456177
-      ],
-      [
-        560,
-        1.2310446500778198
-      ],
-      [
-        600,
-        1.2271382808685303
-      ],
-      [
-        640,
-        1.2247475385665894
-      ],
-      [
-        680,
-        1.2248592376708984
       ]
     ]
   }

   "model_name": "RegTech-14B-Instruct",
   "dataset": "./train.jsonl",
   "env_file": "/home/ubuntu/sophia-core-server/.tuning/.env.14B",
+  "train_samples": 2134,
+  "eval_samples": 237,
   "params": {
     "rank": 16,
+    "alpha": 16,
     "dropout": 0.1,
+    "lr": 2e-05,
     "scheduler": "cosine",
+    "epochs": 1,
     "effective_batch": 4,
     "max_seq_length": 4096,
     "neftune_alpha": 5.0,
     ]
   },
   "results": {
+    "total_steps": 534,
+    "final_train_loss": 1.073,
+    "best_eval_loss": 1.098220944404602,
+    "best_eval_step": 520,
+    "best_token_accuracy": 0.7624,
+    "elapsed_minutes": 19.9
   },
   "loss_history": {
     "train": [
       [
         5,
+        1.8103
       ],
       [
         10,
+        1.9463
       ],
       [
         15,
+        1.7076
       ],
       [
         20,
+        1.8542
       ],
       [
         25,
+        1.9426
       ],
       [
         30,
+        2.026
       ],
       [
         35,
+        1.9177
       ],
       [
         40,
+        1.7516
       ],
       [
         45,
+        1.655
       ],
       [
         50,
+        1.4414
       ],
       [
         55,
+        1.4496
       ],
       [
         60,
+        1.3549
       ],
       [
         65,
+        1.3485
       ],
       [
         70,
+        1.6647
       ],
       [
         75,
+        1.3798
       ],
       [
         80,
+        1.5445
       ],
       [
         85,
+        1.3067
       ],
       [
         90,
+        1.3387
       ],
       [
         95,
+        1.2908
       ],
       [
         100,
+        1.2998
       ],
       [
         105,
+        1.3263
       ],
       [
         110,
+        1.3664
       ],
       [
         115,
+        1.2699
       ],
       [
         120,
+        1.3007
       ],
       [
         125,
+        1.3573
       ],
       [
         130,
+        1.243
       ],
       [
         135,
+        1.3071
       ],
       [
         140,
+        1.2171
       ],
       [
         145,
+        1.1797
       ],
       [
         150,
+        1.1813
       ],
       [
         155,
+        1.3488
       ],
       [
         160,
+        1.1427
       ],
       [
         165,
+        1.2909
       ],
       [
         170,
+        1.1646
       ],
       [
         175,
+        1.0927
       ],
       [
         180,
+        1.1764
       ],
       [
         185,
+        1.1671
       ],
       [
         190,
+        1.2527
       ],
       [
         195,
+        1.0971
       ],
       [
         200,
+        1.0705
       ],
       [
         205,
+        1.1283
       ],
       [
         210,
+        1.1866
       ],
       [
         215,
+        1.1713
       ],
       [
         220,
+        1.0456
       ],
       [
         225,
+        1.137
       ],
       [
         230,
+        1.074
       ],
       [
         235,
+        1.062
       ],
       [
         240,
+        1.0085
       ],
       [
         245,
+        1.0669
       ],
       [
         250,
+        0.9657
       ],
       [
         255,
+        1.0515
       ],
       [
         260,
+        1.0492
       ],
       [
         265,
+        1.1586
       ],
       [
         270,
+        1.0454
       ],
       [
         275,
+        1.1171
       ],
       [
         280,
+        1.0412
       ],
       [
         285,
+        0.9569
       ],
       [
         290,
+        1.0504
       ],
       [
         295,
+        1.0302
       ],
       [
         300,
+        1.0497
       ],
       [
         305,
+        0.9437
       ],
       [
         310,
+        0.9805
       ],
       [
         315,
+        1.0755
       ],
       [
         320,
+        1.0508
       ],
       [
         325,
+        1.0563
       ],
       [
         330,
+        0.9653
       ],
       [
         335,
+        0.9692
       ],
       [
         340,
+        1.1218
       ],
       [
         345,
+        1.0367
       ],
       [
         350,
+        0.9287
       ],
       [
         355,
+        1.0165
       ],
       [
         360,
+        1.0356
       ],
       [
         365,
+        1.0249
       ],
       [
         370,
+        0.9885
       ],
       [
         375,
+        0.9588
       ],
       [
         380,
+        0.977
       ],
       [
         385,
+        0.9921
       ],
       [
         390,
+        0.9863
       ],
       [
         395,
+        1.0569
       ],
       [
         400,
+        0.982
       ],
       [
         405,
+        1.0738
       ],
       [
         410,
+        1.034
       ],
       [
         415,
+        1.067
       ],
       [
         420,
+        1.044
       ],
       [
         425,
+        0.9622
       ],
       [
         430,
+        0.9102
       ],
       [
         435,
+        1.0164
       ],
       [
         440,
+        0.9833
       ],
       [
         445,
+        0.9276
       ],
       [
         450,
+        0.8751
       ],
       [
         455,
+        1.0087
       ],
       [
         460,
+        1.032
       ],
       [
         465,
+        1.0099
       ],
       [
         470,
+        1.0805
       ],
       [
         475,
+        1.0326
       ],
       [
         480,
+        0.968
       ],
       [
         485,
+        0.948
       ],
       [
         490,
+        1.0297
       ],
       [
         495,
+        1.0083
       ],
       [
         500,
+        0.955
       ],
       [
         505,
+        0.967
       ],
       [
         510,
+        0.8891
       ],
       [
         515,
+        0.9298
       ],
       [
         520,
+        0.9516
       ],
       [
         525,
+        0.9553
       ],
       [
         530,
+        1.073
+      ]
+    ],
+    "eval": [
       [
+        40,
+        1.9159308671951294
       ],
       [
+        80,
+        1.5397088527679443
       ],
       [
+        120,
+        1.398988127708435
       ],
       [
+        160,
+        1.3029450178146362
       ],
       [
+        200,
+        1.2370014190673828
       ],
       [
+        240,
+        1.1978071928024292
       ],
       [
+        280,
+        1.1654218435287476
       ],
       [
+        320,
+        1.1381187438964844
       ],
       [
+        360,
+        1.1211411952972412
       ],
       [
+        400,
+        1.107499599456787
       ],
       [
+        440,
+        1.1020454168319702
       ],
       [
+        480,
+        1.0991754531860352
       ],
       [
+        520,
+        1.098220944404602
       ]
     ],
+    "token_accuracy": [
       [
         40,
+        0.6898
       ],
       [
         80,
+        0.7085
       ],
       [
         120,
+        0.7226
       ],
       [
         160,
+        0.7344
       ],
       [
         200,
+        0.7426
       ],
       [
         240,
+        0.7473
       ],
       [
         280,
+        0.7522
       ],
       [
         320,
+        0.7564
       ],
       [
         360,
+        0.7585
       ],
       [
         400,
+        0.7605
       ],
       [
         440,
+        0.7618
       ],
       [
         480,
+        0.7624
       ],
       [
         520,
+        0.7624
       ]
     ]
   }