Team, you have provided the performance matrics in your model card which is very good and I deployed your model but why your model is predicting every wazuh alert as true positive or am I missing something? Kindly let me know so we can connect to discuss about it.
Hello, Model also repond as true positive when alert severity level is less than 5 even I set temperature at 0.1 after locally deploy that model. Actually it shows every alert as a true positive : (
Using the Wazuh API, we retrieved alerts and sent each alert individually to the model for analysis, processing one request at a time as true positive and false positive alert.
hmhm...interesting, can you share the screenshot or result of the model? include the prompt. I want understand more about this problem, so i can help you solve this. I has been used this model on production and so far the model successfully flag the alert correctly.
are you still using this model on production? The prompt I used is given below and I am also attaching low severity alerts classification in the below image.
first used the below prompt: #"You are an expert cybersecurity analyst integrated into a Security Operations Center (SOC). "
"Classify the following Wazuh alert as a true positive or false positive. "
"Respond only with 'True Positive' or 'False Positive'.\n\n"
f"{alert_json}\n"
when every classified alert returned as true positive then prompt refined as below:
"You are an expert cybersecurity analyst in a SOC.\n" "Use the Wazuh rule level as a key signal:\n" " - Rule level 0-6 : likely routine/informational → lean False Positive\n" " - Rule level 7-11 : moderate threat → consider context carefully\n" " - Rule level 12-14 : high severity → lean True Positive\n" " - Rule level 15+ : critical severity → lean True Positive\n\n" f"Alert severity : {severity}\n" f"Rule level : {rule_level}\n\n" "Full alert JSON:\n" f"{alert_json}\n\n" "Classify this alert. Respond only with 'True Positive' or 'False Positive'."
Still alert classification is true positive even for low priority alerts.
