readme

README.md

@@ -1,50 +1,56 @@
 ---
 title: mrr
-datasets:
--  
 tags:
 - evaluate
 - metric
-description: "TODO: add a description here"
+description: "This is the mean reciprocal rank (mrr) metric for retrieval systems.
+It is the average of the precision scores computer after each relevant document is got. You can refer to [here](https://amenra.github.io/ranx/metrics/#mean-average-precision)"
 sdk: gradio
 sdk_version: 3.19.1
 app_file: app.py
 pinned: false
 ---
 
-# Metric Card for mrr
-
-***Module Card Instructions:*** *Fill out the following subsections. Feel free to take a look at existing metric cards if you'd like examples.*
+# Metric Card for map
 
 ## Metric Description
-*Give a brief overview of this metric, including what task(s) it is usually used for, if any.*
+This is the mean average precision (map) metric for retrieval systems.
+It is the average of the precision scores computer after each relevant document is got. You can refer to [here](https://amenra.github.io/ranx/metrics/#mean-average-precision)
 
 ## How to Use
-*Give general statement of how to use the metric*
-
-*Provide simplest possible example for using the metric*
-
+```python
+>>> my_new_module = evaluate.load("mrr")
+>>> references= [json.dumps({"q_1":{"d_1":1, "d_2":2} }), 
+             json.dumps({"q_2":{"d_2":1, "d_3":2, "d_5":3}})] 
+>>> predictions = [json.dumps({"q_1": { "d_1": 0.8, "d_2": 0.9}}),
+         json.dumps({"q_2": {"d_2": 0.9, "d_1": 0.8, "d_5": 0.7, "d_3": 0.3}})]
+>>> results = my_new_module.compute(references=references, predictions=predictions)
+>>> print(results)
+{'mrr': 1.0}
+```
 ### Inputs
-*List all input arguments in the format below*
-- **input_field** *(type): Definition of input, with explanation if necessary. State any default value(s).*
+- **predictions:** a list of dictionaries where each dictionary consists of document relevancy scores produced by the model for a given query. One dictionary per query. The dictionaries should be converted to string. 
+- **references:** a lift of list of dictionaries where each dictionary consists of the relevant order for the documents for a given query in a sorted relevancy order. The dictionaries should be converted to string.
+- **k:**  an optional paramater whose default is None to calculate mrr@k
 
 ### Output Values
+- **mrr (`float`):** mean reciprocal rank. Minimum possible value is 0. Maximum possible value is 1.0
 
-*Explain what this metric outputs and provide an example of what the metric output looks like. Modules should return a dictionary with one or multiple key-value pairs, e.g. {"bleu" : 6.02}*
-
-*State the range of possible values that the metric's output can take, as well as what in that range is considered good. For example: "This metric can take on any value between 0 and 100, inclusive. Higher scores are better."*
-
-#### Values from Popular Papers
-*Give examples, preferrably with links to leaderboards or publications, to papers that have reported this metric, along with the values they have reported.*
-
-### Examples
-*Give code examples of the metric being used. Try to include examples that clear up any potential ambiguity left from the metric description above. If possible, provide a range of examples that show both typical and atypical results, as well as examples where a variety of input parameters are passed.*
 
 ## Limitations and Bias
 *Note any known limitations or biases that the metric has, with links and references if possible.*
 
 ## Citation
-*Cite the source where this metric was introduced.*
-
-## Further References
-*Add any useful further references.*
+```bibtex
+@inproceedings{ranx,
+  author       = {Elias Bassani},
+  title        = {ranx: {A} Blazing-Fast Python Library for Ranking Evaluation and Comparison},
+  booktitle    = {{ECIR} {(2)}},
+  series       = {Lecture Notes in Computer Science},
+  volume       = {13186},
+  pages        = {259--264},
+  publisher    = {Springer},
+  year         = {2022},
+  doi          = {10.1007/978-3-030-99739-7\_30}
+}
+```