Sha09090 commited on
Commit
3f9b430
·
verified ·
1 Parent(s): 4c46864

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +55 -0
README.md ADDED
@@ -0,0 +1,55 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ tags:
4
+ - hdfs
5
+ - log-analysis
6
+ - failure-prediction
7
+ license: mit
8
+ pipeline_tag: text-classification
9
+ widget:
10
+ - text: "PacketResponder: error for block blk_12345 terminating"
11
+ ---
12
+
13
+ # 🚀 HDFS Failure Prediction Model
14
+
15
+ **Developed by:** Shashank Choudhary ([@Sha09090](https://huggingface.co/Sha09090))
16
+ **License:** MIT
17
+
18
+ This model detects failures in HDFS logs with **99.95% Accuracy**.
19
+ It was trained on a balanced dataset of ~575k log entries.
20
+
21
+ ## 📊 Benchmarks
22
+
23
+ | Metric | Score |
24
+ | :--- | :--- |
25
+ | **Accuracy** | **99.95%** |
26
+ | **Precision** | **99.99%** |
27
+ | **Recall** | **99.96%** |
28
+
29
+ ## 💻 How to Use
30
+
31
+ ```python
32
+ from transformers import AutoTokenizer, AutoModelForSequenceClassification
33
+ import torch
34
+
35
+ # Load directly from Shashank's Repo
36
+ model_name = "Sha09090/hdfs-failure-prediction"
37
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
38
+ model = AutoModelForSequenceClassification.from_pretrained(model_name)
39
+
40
+ # Predict
41
+ log = "PacketResponder: error for block blk_12345 terminating"
42
+ inputs = tokenizer(log, return_tensors="pt")
43
+ with torch.no_grad():
44
+ logits = model(**inputs).logits
45
+
46
+ print("Failure Probability:", torch.softmax(logits, dim=1)[0][1].item())
47
+
48
+ @misc{hdfs-failure-prediction,
49
+ author = {Shashank Choudhary},
50
+ title = {HDFS Failure Prediction Transformer},
51
+ year = {2026},
52
+ publisher = {Hugging Face},
53
+ journal = {Hugging Face Repository},
54
+ howpublished = {\url{[https://huggingface.co/Sha09090/hdfs-failure-prediction](https://huggingface.co/Sha09090/hdfs-failure-prediction)}}
55
+ }