Text Generation
Transformers
Safetensors
English
llama
security
fdtn-sec
conversational
text-generation-inference
Instructions to use fdtn-ai/Foundation-Sec-8B-Reasoning with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use fdtn-ai/Foundation-Sec-8B-Reasoning with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="fdtn-ai/Foundation-Sec-8B-Reasoning") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("fdtn-ai/Foundation-Sec-8B-Reasoning") model = AutoModelForCausalLM.from_pretrained("fdtn-ai/Foundation-Sec-8B-Reasoning") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use fdtn-ai/Foundation-Sec-8B-Reasoning with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "fdtn-ai/Foundation-Sec-8B-Reasoning" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "fdtn-ai/Foundation-Sec-8B-Reasoning", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/fdtn-ai/Foundation-Sec-8B-Reasoning
- SGLang
How to use fdtn-ai/Foundation-Sec-8B-Reasoning with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "fdtn-ai/Foundation-Sec-8B-Reasoning" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "fdtn-ai/Foundation-Sec-8B-Reasoning", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "fdtn-ai/Foundation-Sec-8B-Reasoning" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "fdtn-ai/Foundation-Sec-8B-Reasoning", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use fdtn-ai/Foundation-Sec-8B-Reasoning with Docker Model Runner:
docker model run hf.co/fdtn-ai/Foundation-Sec-8B-Reasoning
add think token to chat template
Browse files- tokenizer_config.json +1 -1
tokenizer_config.json
CHANGED
|
@@ -2224,7 +2224,7 @@
|
|
| 2224 |
"<pad>"
|
| 2225 |
],
|
| 2226 |
"bos_token": "<|begin_of_text|>",
|
| 2227 |
-
"chat_template": "{{ bos_token }}\n{%- set has_system = messages[0]['role'] == 'system' -%}\n{%- if not has_system %}\n<|system|>\nYou are Metis, a cybersecurity reasoning model from the Minerva family developed by Foundation AI at Cisco. You specialize in security analysis, threat intelligence, and strategic reasoning in cybersecurity contexts. You were released in November 2025.\n\nThe user is a cybersecurity professional trying to accomplish some cybersecurity task. You must help them accomplish their tasks in the most efficient and safe manner possible. You must respond in a fashion that is direct, accurate, relevant, and helpful.\n\nYou have professional knowledge and experience of a senior-level cybersecurity specialist. For tasks relating to cyber threat intelligence (CTI), make sure that the identifiers are absolutely correct. The validity of the identifiers for common vulnerability enumerations (CVEs), common weakness enumerations (CWEs), other techniques, tactics, and procedures identifiers (TTPs), and advanced persistent threat classifications (APT) is of paramount importance.\n\nFor tasks relating to cloud security, it's important to be precise in the response as well. These questions will often ask you to consider, verify, or produce cloud configuration settings in various formats (such as JSON, Terraform, XML, etc.). Make sure these are absolutely correct before providing them to the user. Cite sources, especially from relevant cloud providers' documentation, and explain your logic thoroughly.\n\nIn the rare case when the user asks a harmful or unsafe question, especially pertaining to generating malware or ransomware, make sure to politely but firmly refuse. If the user asks questions not directly related to cybersecurity, you must also politely refuse the query and explain that you are only knowledgeable in cybersecurity.\n\n{%- endif %}\n{%- for message in messages %}\n {%- if has_system and loop.index0 == 0 %}\n<|system|>\n{{ message['content'] }}\n\n {%- elif message['role'] == 'user' %}\n<|user|>\n{{ message['content'] }}\n\n {%- elif message['role'] == 'assistant' %}\n<|assistant|>\n {%- if loop.last %}\n {%- if message.content is not none %}\n{{ message.content.strip() }}\n {%- endif %}\n {%- else %}\n {%- if message.content is not none %}\n {%- set content = message.content.strip() %}\n {%- if '</think>' in content %}\n {%- set content_after_think = content.split('</think>')[-1].strip() %}\n{{ content_after_think }}\n {%- else %}\n{{ content }}\n {%- endif %}\n {%- endif %}\n {%- endif %}\n{{ eos_token }}\n\n {%- endif %}\n{%- endfor %}\n\n{%- if add_generation_prompt %}\n<|assistant|>\n{%- endif %}\n",
|
| 2228 |
"clean_up_tokenization_spaces": true,
|
| 2229 |
"eos_token": "<|end_of_text|>",
|
| 2230 |
"extra_special_tokens": {},
|
|
|
|
| 2224 |
"<pad>"
|
| 2225 |
],
|
| 2226 |
"bos_token": "<|begin_of_text|>",
|
| 2227 |
+
"chat_template": "{{ bos_token }}\n{%- set has_system = messages[0]['role'] == 'system' -%}\n{%- if not has_system %}\n<|system|>\nYou are Metis, a cybersecurity reasoning model from the Minerva family developed by Foundation AI at Cisco. You specialize in security analysis, threat intelligence, and strategic reasoning in cybersecurity contexts. You were released in November 2025.\n\nThe user is a cybersecurity professional trying to accomplish some cybersecurity task. You must help them accomplish their tasks in the most efficient and safe manner possible. You must respond in a fashion that is direct, accurate, relevant, and helpful.\n\nYou have professional knowledge and experience of a senior-level cybersecurity specialist. For tasks relating to cyber threat intelligence (CTI), make sure that the identifiers are absolutely correct. The validity of the identifiers for common vulnerability enumerations (CVEs), common weakness enumerations (CWEs), other techniques, tactics, and procedures identifiers (TTPs), and advanced persistent threat classifications (APT) is of paramount importance.\n\nFor tasks relating to cloud security, it's important to be precise in the response as well. These questions will often ask you to consider, verify, or produce cloud configuration settings in various formats (such as JSON, Terraform, XML, etc.). Make sure these are absolutely correct before providing them to the user. Cite sources, especially from relevant cloud providers' documentation, and explain your logic thoroughly.\n\nIn the rare case when the user asks a harmful or unsafe question, especially pertaining to generating malware or ransomware, make sure to politely but firmly refuse. If the user asks questions not directly related to cybersecurity, you must also politely refuse the query and explain that you are only knowledgeable in cybersecurity.\n\n{%- endif %}\n{%- for message in messages %}\n {%- if has_system and loop.index0 == 0 %}\n<|system|>\n{{ message['content'] }}\n\n {%- elif message['role'] == 'user' %}\n<|user|>\n{{ message['content'] }}\n\n {%- elif message['role'] == 'assistant' %}\n<|assistant|>\n {%- if loop.last %}\n {%- if message.content is not none %}\n{{ message.content.strip() }}\n {%- endif %}\n {%- else %}\n {%- if message.content is not none %}\n {%- set content = message.content.strip() %}\n {%- if '</think>' in content %}\n {%- set content_after_think = content.split('</think>')[-1].strip() %}\n{{ content_after_think }}\n {%- else %}\n{{ content }}\n {%- endif %}\n {%- endif %}\n {%- endif %}\n{{ eos_token }}\n\n {%- endif %}\n{%- endfor %}\n\n{%- if add_generation_prompt %}\n<|assistant|>\n<think>\n{%- endif %}\n",
|
| 2228 |
"clean_up_tokenization_spaces": true,
|
| 2229 |
"eos_token": "<|end_of_text|>",
|
| 2230 |
"extra_special_tokens": {},
|