Update README.md

README.md

@@ -11,8 +11,7 @@ pipeline_tag: text-classification
 ---
 # QA-Evaluation-Metrics
 
-[![PyPI version qa-metrics](https://img.shields.io/pypi/v/qa-metrics.svg)](https://pypi.org/project/qa-metrics/) 
-
+[![PyPI version qa-metrics](https://img.shields.io/pypi/v/qa-metrics.svg)](https://pypi.org/project/qa-metrics/) [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/17b7vrZqH0Yun2AJaOXydYZxr3cw20Ga6?usp=sharing)
 
 QA-Evaluation-Metrics is a fast and lightweight Python package for evaluating question-answering models. It provides various basic metrics to assess the performance of QA models. Check out our paper [**PANDA**](https://arxiv.org/abs/2402.11161), a matching method going beyond token-level matching and is more efficient than LLM matchings but still retains competitive evaluation performance of transformer LLM models.
 
@@ -33,23 +32,29 @@ The python package currently provides four QA evaluation metrics.
 ```python
 from qa_metrics.em import em_match
 
-reference_answer = ["Charles , Prince of Wales"]
-candidate_answer = "Prince Charles"
+reference_answer = ["The Frog Prince", "The Princess and the Frog"]
+candidate_answer = "The movie \"The Princess and the Frog\" is loosely based off the Brother Grimm's \"Iron Henry\""
 match_result = em_match(reference_answer, candidate_answer)
 print("Exact Match: ", match_result)
+'''
+Exact Match:  False
+'''
 ```
 
 #### Transformer Match
-Our fine-tuned BERT model is on 🤗 [Huggingface](https://huggingface.co/Zongxia/answer_equivalence_bert?text=The+goal+of+life+is+%5BMASK%5D.). Our Package also supports downloading and matching directly. [distilroberta](https://huggingface.co/Zongxia/answer_equivalence_distilroberta), [distilbert](https://huggingface.co/Zongxia/answer_equivalence_distilbert), [roberta](https://huggingface.co/Zongxia/answer_equivalence_roberta), and [roberta-large](https://huggingface.co/Zongxia/answer_equivalence_roberta-large) are also supported now! 🔥🔥🔥
+Our fine-tuned BERT model is this repository. Our Package also supports downloading and matching directly. distilroberta, distilbert, and roberta are also supported now! 🔥🔥🔥
 
 ```python
 from qa_metrics.transformerMatcher import TransformerMatcher
 
-question = "who will take the throne after the queen dies"
+question = "Which movie is loosley based off the Brother Grimm's Iron Henry?"
 tm = TransformerMatcher("bert")
 scores = tm.get_scores(reference_answer, candidate_answer, question)
 match_result = tm.transformer_match(reference_answer, candidate_answer, question)
-print("Score: %s; CF Match: %s" % (scores, match_result))
+print("Score: %s; TM Match: %s" % (scores, match_result))
+'''
+Score: {'The Frog Prince': {'The movie "The Princess and the Frog" is loosely based off the Brother Grimm\'s "Iron Henry"': 0.88954514}, 'The Princess and the Frog': {'The movie "The Princess and the Frog" is loosely based off the Brother Grimm\'s "Iron Henry"': 0.9381995}}; TM Match: True
+'''
 ```
 
 #### F1 Score
@@ -61,17 +66,24 @@ print("F1 stats: ", f1_stats)
 
 match_result = f1_match(reference_answer, candidate_answer, threshold=0.5)
 print("F1 Match: ", match_result)
+'''
+F1 stats:  {'f1': 0.25, 'precision': 0.6666666666666666, 'recall': 0.15384615384615385}
+F1 Match:  False
+'''
 ```
 
 #### PANDA
 ```python
 from qa_metrics.cfm import CFMatcher
 
-question = "who will take the throne after the queen dies"
+question = "Which movie is loosley based off the Brother Grimm's Iron Henry?"
 cfm = CFMatcher()
 scores = cfm.get_scores(reference_answer, candidate_answer, question)
 match_result = cfm.cf_match(reference_answer, candidate_answer, question)
-print("Score: %s; bert Match: %s" % (scores, match_result))
+print("Score: %s; PD Match: %s" % (scores, match_result))
+'''
+Score: {'the frog prince': {'The movie "The Princess and the Frog" is loosely based off the Brother Grimm\'s "Iron Henry"': 0.7131625951317375}, 'the princess and the frog': {'The movie "The Princess and the Frog" is loosely based off the Brother Grimm\'s "Iron Henry"': 0.854451712151719}}; PD Match: True
+'''
 ```
 
 If you find this repo avialable, please cite our paper:
@@ -88,7 +100,7 @@ If you find this repo avialable, please cite our paper:
 
 
 ## Updates
-- [01/24/24] 🔥 The full paper is uploaded and can be accessed [here]([https://arxiv.org/abs/2310.14566](https://arxiv.org/abs/2401.13170)). The dataset is expanded and leaderboard is updated.
+- [01/24/24] 🔥 The full paper is uploaded and can be accessed [here](https://arxiv.org/abs/2402.11161). The dataset is expanded and leaderboard is updated.
 - Our Training Dataset is adapted and augmented from [Bulian et al](https://github.com/google-research-datasets/answer-equivalence-dataset). Our [dataset repo](https://github.com/zli12321/Answer_Equivalence_Dataset.git) includes the augmented training set and QA evaluation testing sets discussed in our paper. 
 - Now our model supports [distilroberta](https://huggingface.co/Zongxia/answer_equivalence_distilroberta), [distilbert](https://huggingface.co/Zongxia/answer_equivalence_distilbert), a smaller and more robust matching model than Bert!