Update README

README.md

@@ -1,200 +1,241 @@
 ---
-# For reference on model card metadata, see the spec: https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1
-# Doc / guide: https://huggingface.co/docs/hub/model-cards
-{}
+license: cc-by-nc-sa-4.0
 ---
 
-# Model Card for Model ID
+# Model Card for Hallucination Detection Model (HDM-2-3B)
 
-<!-- Provide a quick summary of what the model is/does. -->
+**Paper:** 
+[![Read full-text on arXiv](https://img.shields.io/badge/arXiv-2504.07069-b31b1b.svg)](https://arxiv.org/abs/2504.07069)
+*HalluciNot: Hallucination Detection Through Context and Common Knowledge Verification.*
 
-This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1).
 
-## Model Details
+**Notebook:** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https\://colab.research.google.com/drive/1HclyB06twZVIxuK6AlyifRaf77vO5Yz#scrollTo=UVvBvBMWrDiv)
 
-### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
+**HDM-Bench Dataset:** 🤗 [HuggingFace Dataset](https\://huggingface.co/datasets/AimonLabs/HDM-Bench)
 
 
+## Introduction
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+Most judge models used in the industry today are not specialized for Hallucination evaluation tasks. 
+Developers using them often struggle with score inconsistency, variance, high latencies, high costs, and prompt sensitivity. 
+HDM-2 solves these challenges and at the same time, provides industry-first, state-of-the-art features.
 
-### Model Sources [optional]
 
-<!-- Provide the basic links for the model. -->
+## Highlights:
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+- Outperforms existing baselines on RagTruth, TruthfulQA, and our new HDM-Bench benchmark.
 
-## Uses
+- **Context-based** hallucination evaluations based on user-provided or retrieved documents.
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+- **Common knowledge** contradictions based on widely-accepted common knowledge facts.
 
-### Direct Use
+- **Phrase, token, and sentence-level** Hallucination identification with token-level probability **scores**
+
+- Generalized model that works well across a variety of domains such as Finance, Healthcare, Legal, and Insurance.
+
+- Operates within a **latency** budget of **500ms** on a single L4 GPU, especially beneficial for Agentic use cases.
+
+## Model Overview:
+
+HDM-2 is a modular, production-ready, multi-task hallucination (or inaccuracy) evaluation model designed to validate the factual groundedness of LLM outputs in enterprise environments, for both **contextual** and **common knowledge** evaluations. 
+HDM-2 introduces a novel taxonomy-guided, span-level validation architecture focused on precision, explainability, and adaptability. 
+The figure below shows the workflow (on the left) in which we determine whether a certain LLM response is hallucinated or not and an example (on the right) that shows the taxonomy of an LLM response.
+
+HDM-2 Model Workflow | Example of Enterprise LLM Response Taxonomy
+--- | ---
+![](https://lh7-rt.googleusercontent.com/docsz/AD_4nXdpn0qSjx_A3ax0qXZ3BIBTXAbMphuN1gLPXRQ4m_aTCSaN_hMMS27d0hJeQaZhc0P_iCpnktRsCyT_xB5V7-ofqQwjAvNWkRka_fJAGKfD466PK-jgGoRpDPqT9Ag3MT8XVSGscQ?key=x9HqmDQsJmBeqyuiakDxe8Cs) | ![](https://lh7-rt.googleusercontent.com/docsz/AD_4nXfJzyMnYVlR9sNIV7cDKmY3d_RnQYUBj7Ass6RWfhTt5ds2OJ5os2uPv7loECI_ao7_To3H4WV9UoHhnbJ2Ux-XSFQK76NJzOkiWNuDQQxuaojzgazujJ45KPSyhbtbfNe3msyl6w?key=x9HqmDQsJmBeqyuiakDxe8Cs)
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
 
-[More Information Needed]
+### Enterprise Models
 
-### Downstream Use [optional]
+- The Enterprise version offers a way to incorporate “Enterprise knowledge” into Hallucination evaluations. This means knowledge that is specific to your company (or domain or industry) that might not be present in your context!!
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+- Another important feature covered in the Enterprise version are explanations. Please reach out to us for Enterprise licensing. 
 
-[More Information Needed]
+- Other premium capabilities that will be included in the Enterprise version include improved accuracies, even lower latencies, and additional use cases such as Math and Code.
 
-### Out-of-Scope Use
+- Apart from Hallucinations, we have SOTA models for Prompt/Instruction adherence, RAG Relevance, Reranking (Promptable). The instruction adherence model is general-purpose and extremely low-latency. It performs well with a wide variety of instructions, including safety, style, and format constraints.
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
 
-[More Information Needed]
+### Performance - Model Accuracy
 
-## Bias, Risks, and Limitations
+See paper (linked on top) for more details.
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+|             |               |            |              |
+| :---------: | :-----------: | :--------: | :----------: |
+| **Dataset** | **Precision** | **Recall** | **F1 Score** |
+|   HDMBENCH  |      0.87     |    0.84    |     0.855    |
+|  TruthfulQA |      0.82     |    0.78    |     0.80     |
+|   RagTruth  |      0.85     |    0.81    |     0.83     |
 
-[More Information Needed]
 
-### Recommendations
+### Latency
+
+
+|                         |                      |                        |                         |                     |
+| ----------------------- | -------------------- | ---------------------- | ----------------------- | ------------------- |
+| **Device**              | **Avg. Latency (s)** | **Median Latency (s)** | **95th Percentile (s)** | **Max Latency (s)** |
+| Nvidia A100             | 0.204                | 0.201                  | 0.208                   | 1.32                |
+| Nvidia L4 (recommended) | 0.207                | 0.203                  | 0.220                   | 1.29                |
+| Nvidia T4               | 0.935                | 0.947                  | 1.487                   | 1.605               |
+| CPU                     | 261.92               | 242.76                 | 350.76                  | 356.96              |
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
 
 ## How to Get Started with the Model
 
 Use the code below to get started with the model.
 
-[More Information Needed]
+Install the Inference Code
 
-## Training Details
+    !pip install hdm2 --quiet
 
-### Training Data
+Run the HDM-2 model
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+    # Load the model from HuggingFace into the GPU
 
-[More Information Needed]
+<!---->
 
-### Training Procedure
+    from hdm2 import HallucinationDetectionModel
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+<!---->
 
-#### Preprocessing [optional]
+    hdm_model = HallucinationDetectionModel()
 
-[More Information Needed]
+<!---->
 
+    prompt = "Explain how the heart functions"
+    context = """
+    The heart is a muscular organ that pumps blood throughout the body.
+    It has four chambers: two atria and two ventricles.
+    """
+    response = """The heart is a vital six-chambered organ that pumps blood throughout the human body.
+    It contains three atria and three ventricles that work in harmony to circulate blood.
+    The heart primarily runs on glucose for energy and typically beats at a rate of 20-30 beats per minute in adults.
+    Located in the center-left of the chest, the heart is protected by the ribcage.
+    The average human heart weighs about 5 pounds and will beat approximately 2 million times in a lifetime.
+    """
+    # Ground truth:
+    # Hearts have 4 chambers (not 6), have 2 atria and 2 ventricles (not 3 each),
+    # normal heart rate is 60-100 BPM (not 20-30),
+    # average heart weighs ~10 oz (not 5 pounds),
+    # and beats ~2.5 billion times (not 2 million) in a lifetime
 
-#### Training Hyperparameters
+<!---->
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+    # Detect hallucinations with default parameters
+    results = hdm_model.apply(prompt, context, response)
 
-#### Speeds, Sizes, Times [optional]
+Print the results
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+    # Utility function to help with printing the model output
+    def print_results(results):
+     #print(results)
+     # Print results
+     print(f"\nHallucination severity: {results['adjusted_hallucination_severity']:.4f}")
 
-[More Information Needed]
+<!---->
 
-## Evaluation
+     # Print hallucinated sentences
+     if results['candidate_sentences']:
+         print("\nPotentially hallucinated sentences:")
+         is_ck_hallucinated = False
+         for sentence_result in results['ck_results']:
+             if sentence_result['prediction'] == 1:  # 1 indicates hallucination
+                 print(f"- {sentence_result['text']} (Probability: {sentence_result['hallucination_probability']:.4f})")
+                 is_ck_hallucinated = True
+         if not is_ck_hallucinated:
+           print("No hallucinated sentences detected.")
+     else:
+         print("\nNo hallucinated sentences detected.")
 
-<!-- This section describes the evaluation protocols and provides the results. -->
+<!---->
 
-### Testing Data, Factors & Metrics
+    print_results(results)
 
-#### Testing Data
 
-<!-- This should link to a Dataset Card if possible. -->
+### Model Description
+
+- Model ID: HDM-2-3B
 
-[More Information Needed]
+- Developed by: AIMon Labs, Inc.
 
-#### Factors
+- Model type: 
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+- Language(s) (NLP): English
 
-[More Information Needed]
+- License: CC BY-NC-SA 4.0
 
-#### Metrics
+- License URL: <https://creativecommons.org/licenses/by-nc-sa/4.0/> 
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
 
-[More Information Needed]
+### Model Sources
 
-### Results
+- Code repository: [GitHub](https://github.com/aimonlabs/hallucination-detection-model)
 
-[More Information Needed]
+- Model weights: [HuggingFace](https://huggingface.co/AimonLabs/hallucination-detection-model/)
 
-#### Summary
+- Paper: [arXiv](https://arxiv.org/abs/2504.07069)
 
+- Demo: [Google Colab](https://colab.research.google.com/drive/1HclyB06twZVIxuK6AlyifRaf77vO5Yz#scrollTo=UVvBvBMWrDiv)
 
 
-## Model Examination [optional]
+## Uses
+
+### Direct Use
 
-<!-- Relevant interpretability work for the model goes here -->
+1. Automating Hallucination or Inaccuracy Evaluations
 
-[More Information Needed]
+2. Assisting humans evaluating LLM responses for Hallucinations
 
-## Environmental Impact
+3. Phrase, word or sentence-level identification of where Hallucinations lie
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+4. Selecting the best LLM with the least hallucinations for specific use cases
 
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+5. Automatic re-prompting for better LLM responses
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
 
-## Technical Specifications [optional]
+## Limitations
 
-### Model Architecture and Objective
+- Annotations of "common knowledge" may still contain subjective judgments
 
-[More Information Needed]
 
-### Compute Infrastructure
+## Technical Specifications 
 
-[More Information Needed]
+See paper for [more details](https://arxiv.org/abs/2504.07069)
 
-#### Hardware
 
-[More Information Needed]
+## Citation:
 
-#### Software
+@misc {hdm-2-3b,
 
-[More Information Needed]
+    author       = {Paudel, A. and Lyzhov, A. and AIMon Labs},
 
-## Citation [optional]
+    title        = {HDM-2-3B: Hallucination Detection Model for Enterprise LLMs}, [Bibek Paudel](mailto:bibek@aimon.ai) 
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+    year         = 2025,
 
-**BibTeX:**
+    url          = {<https://huggingface.co/aimonlabs/> ??????? [Preetam Joshi](mailto:preetam@aimon.ai)},
 
-[More Information Needed]
+    publisher    = {AIMon Labs, Inc.},
 
-**APA:**
+    eprint       = {2504.07069},
 
-[More Information Needed]
+    archivePrefix= {arXiv},
 
-## Glossary [optional]
+    primaryClass = {cs.CL},
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+    url          = {https\://arxiv.org/abs/2504.07069}
 
-[More Information Needed]
+}
 
-## More Information [optional]
 
-[More Information Needed]
+## Model Card Authors
 
-## Model Card Authors [optional]
+@bibekp, @alexlyzhov-aimon, @pjoshi30, @aimonp
 
-[More Information Needed]
 
 ## Model Card Contact
 
-[More Information Needed]
+<info@aimon.ai>, @aimonp, @pjoshi30