Instructions to use ZySec-AI/SecurityLLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ZySec-AI/SecurityLLM with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ZySec-AI/SecurityLLM")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ZySec-AI/SecurityLLM")
model = AutoModelForCausalLM.from_pretrained("ZySec-AI/SecurityLLM")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use ZySec-AI/SecurityLLM with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ZySec-AI/SecurityLLM"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ZySec-AI/SecurityLLM",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ZySec-AI/SecurityLLM

SGLang

How to use ZySec-AI/SecurityLLM with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ZySec-AI/SecurityLLM" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ZySec-AI/SecurityLLM",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ZySec-AI/SecurityLLM" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ZySec-AI/SecurityLLM",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ZySec-AI/SecurityLLM with Docker Model Runner:
```
docker model run hf.co/ZySec-AI/SecurityLLM
```

vSiddi commited on Mar 20, 2024

Commit

4736490

1 Parent(s): 9e383a8

New training

Browse files

Files changed (8) hide show

README.md +4 -1
config.json +1 -1
generation_config.json +1 -1
gitattributes.txt +35 -0
model-00001-of-00003.safetensors +1 -1
model-00002-of-00003.safetensors +1 -1
model-00003-of-00003.safetensors +1 -1
tokenizer.json +1 -8

README.md CHANGED Viewed

@@ -7,11 +7,14 @@ tags:
 - threat
 - vulnerability
 - infosec
-- attackio
 - cyber security
 - ai4security
 - llmsecurity
 - cyber
 - cybersec
 - cybersecurity
 ---

 - threat
 - vulnerability
 - infosec
+- zysec.ai
 - cyber security
 - ai4security
 - llmsecurity
 - cyber
+- ai4good
+- aisecurity
+- threat
 - cybersec
 - cybersecurity
 ---

config.json CHANGED Viewed

@@ -21,7 +21,7 @@
   "sliding_window": 4096,
   "tie_word_embeddings": false,
   "torch_dtype": "float16",
-  "transformers_version": "4.37.1",
   "use_cache": true,
   "vocab_size": 32000
 }

   "sliding_window": 4096,
   "tie_word_embeddings": false,
   "torch_dtype": "float16",
+  "transformers_version": "4.39.0.dev0",
   "use_cache": true,
   "vocab_size": 32000
 }

generation_config.json CHANGED Viewed

@@ -2,5 +2,5 @@
   "_from_model_config": true,
   "bos_token_id": 1,
   "eos_token_id": 2,
-  "transformers_version": "4.37.1"
 }

   "_from_model_config": true,
   "bos_token_id": 1,
   "eos_token_id": 2,
+  "transformers_version": "4.39.0.dev0"
 }

gitattributes.txt ADDED Viewed

	@@ -0,0 +1,35 @@

+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

model-00001-of-00003.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:15afdeedaa2b25b218fae2e755cae4744cd8b855de1d43af7289e8f6d8240dc8
 size 4943162240

 version https://git-lfs.github.com/spec/v1
+oid sha256:b3eab801981e3b3b64c23404c0aa929ae25d10bff1e73c83c8bc062080f35889
 size 4943162240

model-00002-of-00003.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7a7eadfdf60f8d23c57bbe9dd041677813bae0be7326fb35659af069d9465b7e
 size 4999819232

 version https://git-lfs.github.com/spec/v1
+oid sha256:a84f9d1f3b350e464b77c0f17450ee911edacb03afbf29626c87c111d3b9a4b3
 size 4999819232

model-00003-of-00003.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6d71cf1f5ff1b742f2caefafc1f05a92d90ffe99148fdea3b1e963f08219fea4
 size 4540516256

 version https://git-lfs.github.com/spec/v1
+oid sha256:68afc98e7c635ccba9a4b6a132fbb860ee989dcc1adcfbe4053f26f40b166566
 size 4540516256

tokenizer.json CHANGED Viewed

@@ -1,14 +1,7 @@
 {
   "version": "1.0",
   "truncation": null,
-  "padding": {
-    "strategy": "BatchLongest",
-    "direction": "Left",
-    "pad_to_multiple_of": null,
-    "pad_id": 2,
-    "pad_type_id": 0,
-    "pad_token": "</s>"
-  },
   "added_tokens": [
     {
       "id": 0,

 {
   "version": "1.0",
   "truncation": null,
+  "padding": null,
   "added_tokens": [
     {
       "id": 0,