updatre readme

README.md

@@ -31,27 +31,127 @@ title: mot-metrics
 >>> module = evaluate.load("SEA-AI/user-friendly-metrics")
 >>> res = module._calculate(b, max_iou=0.99, recognition_thresholds=[0.3, 0.5, 0.8])
 >>> print(res)
-{'Sentry_2022_11_PROACT_CELADON_7.5M_MOB_2022_11_25_12_12_39': {'volcanic-sweep-3_02_2023_N_LN1_ep288_TRACKER': {'idf1': 0.9543031226199543,
-   'idp': 0.9804381846635368,
-   'idr': 0.9295252225519288,
-   'recall': 0.9436201780415431,
-   'precision': 0.9953051643192489,
-   'num_unique_objects': 2,
-   'mostly_tracked': 1,
-   'partially_tracked': 0,
-   'mostly_lost': 1,
-   'num_false_positives': 6,
-   'num_misses': 76,
-   'num_switches': 1,
-   'num_fragmentations': 4,
-   'mota': 0.9384272997032641,
-   'motp': 0.5235835810268012,
-   'num_transfer': 0,
-   'num_ascend': 1,
-   'num_migrate': 0,
-   'recognition_0.3': 0.9230769230769231, 
-   'recognition_0.5': 0.8461538461538461, 
-   'recognition_0.8': 0.46153846153846156}}}
+```
+```
+global:
+    ahoy-IR-b2-whales__XAVIER-AGX-JP46_TRACKER:
+        all:
+            f1: 0.8262651742077881
+            fn: 2045.0
+            fp: 159.0
+            num_gt_ids: 13
+            precision: 0.9705555555555555
+            recall: 0.7193247323634367
+            recognition_0.3: 0.9230769230769231
+            recognition_0.5: 0.8461538461538461
+            recognition_0.8: 0.46153846153846156
+            recognized_0.3: 12
+            recognized_0.5: 11
+            recognized_0.8: 6
+            tp: 5241.0
+        area:
+            large:
+                f1: 0.4053050397877984
+                fn: 612.0
+                fp: 3872.0
+                num_gt_ids: 6
+                precision: 0.28296296296296297
+                recall: 0.7140186915887851
+                recognition_0.3: 0.8333333333333334
+                recognition_0.5: 0.8333333333333334
+                recognition_0.8: 0.3333333333333333
+                recognized_0.3: 5
+                recognized_0.5: 5
+                recognized_0.8: 2
+                tp: 1528.0
+            medium:
+                f1: 0.7398209644816635
+                fn: 1146.0
+                fp: 1557.0
+                num_gt_ids: 10
+                precision: 0.7116666666666667
+                recall: 0.7702946482260974
+                recognition_0.3: 1.0
+                recognition_0.5: 0.8
+                recognition_0.8: 0.6
+                recognized_0.3: 10
+                recognized_0.5: 8
+                recognized_0.8: 6
+                tp: 3843.0
+            small:
+                f1: 0.10373582388258838
+                fn: 285.0
+                fp: 5089.0
+                num_gt_ids: 6
+                precision: 0.05759259259259259
+                recall: 0.5218120805369127
+                recognition_0.3: 0.3333333333333333
+                recognition_0.5: 0.3333333333333333
+                recognition_0.8: 0.16666666666666666
+                recognized_0.3: 2
+                recognized_0.5: 2
+                recognized_0.8: 1
+                tp: 311.0
+per_sequence:
+    Sentry_2022_12_19_Romania_2022_12_19_17_09_34:
+        ahoy-IR-b2-whales__XAVIER-AGX-JP46_TRACKER:
+            all:
+                f1: 0.8262651742077881
+                fn: 2045.0
+                fp: 159.0
+                num_gt_ids: 13
+                precision: 0.9705555555555555
+                recall: 0.7193247323634367
+                recognition_0.3: 0.9230769230769231
+                recognition_0.5: 0.8461538461538461
+                recognition_0.8: 0.46153846153846156
+                recognized_0.3: 12
+                recognized_0.5: 11
+                recognized_0.8: 6
+                tp: 5241.0
+            area:
+                large:
+                    f1: 0.4053050397877984
+                    fn: 612.0
+                    fp: 3872.0
+                    num_gt_ids: 6
+                    precision: 0.28296296296296297
+                    recall: 0.7140186915887851
+                    recognition_0.3: 0.8333333333333334
+                    recognition_0.5: 0.8333333333333334
+                    recognition_0.8: 0.3333333333333333
+                    recognized_0.3: 5
+                    recognized_0.5: 5
+                    recognized_0.8: 2
+                    tp: 1528.0
+                medium:
+                    f1: 0.7398209644816635
+                    fn: 1146.0
+                    fp: 1557.0
+                    num_gt_ids: 10
+                    precision: 0.7116666666666667
+                    recall: 0.7702946482260974
+                    recognition_0.3: 1.0
+                    recognition_0.5: 0.8
+                    recognition_0.8: 0.6
+                    recognized_0.3: 10
+                    recognized_0.5: 8
+                    recognized_0.8: 6
+                    tp: 3843.0
+                small:
+                    f1: 0.10373582388258838
+                    fn: 285.0
+                    fp: 5089.0
+                    num_gt_ids: 6
+                    precision: 0.05759259259259259
+                    recall: 0.5218120805369127
+                    recognition_0.3: 0.3333333333333333
+                    recognition_0.5: 0.3333333333333333
+                    recognition_0.8: 0.16666666666666666
+                    recognized_0.3: 2
+                    recognized_0.5: 2
+                    recognized_0.8: 1
+                    tp: 311.0
 ```
 
 ## Metric Settings
@@ -64,22 +164,19 @@ The output is a dictionary containing the following metrics:
 
 | Name                 | Description                                                                        |
 | :------------------- | :--------------------------------------------------------------------------------- |
-| idf1                 | ID measures: global min-cost F1 score.                                             |
-| idp                  | ID measures: global min-cost precision.                                            |
-| idr                  | ID measures: global min-cost recall.                                               |
 | recall               | Number of detections over number of objects.                                       |
 | precision            | Number of detected objects over sum of detected and false positives.               |
-| num_unique_objects   | Total number of unique object ids encountered.                                     |
-| mostly_tracked       | Number of objects tracked for at least 80 percent of lifespan.                     |
-| partially_tracked    | Number of objects tracked between 20 and 80 percent of lifespan.                   |
-| mostly_lost          | Number of objects tracked less than 20 percent of lifespan.                        |
-| num_false_positives  | Total number of false positives (false-alarms).                                    |
-| num_misses           | Total number of misses.                                                            |
-| num_switches         | Total number of track switches.                                                    |
-| num_fragmentations   | Total number of switches from tracked to not tracked.                              |
-| mota                 | Multiple object tracker accuracy.                                                  |
-| motp                 | Multiple object tracker precision.                                                 |
-| recognition_0.3      | Recognition rate for a threshold of 0.3                                            |
+| f1   | F1 score                                     |
+| num_gt_ids       | Number of unique objects on the ground truth                     |
+| fn    | Number of false negatives                  |
+| fp          | Number of of false postives                        |
+| tp  | number of true positives                                    |
+| recognized_th           | Total number of unique objects on the ground truth that were seen more then th% of the times |
+| recognition_th           | Total number of unique objects on the ground truth that were seen more then th% of the times over the number of unique objects on the ground truth|
+
+## How it Works
+
+We levereage one of the internal variables of motmetrics ```MOTAccumulator``` class, ```events```, which keeps track of the detections hits and misses. These values are then processed via the ```track_ratios``` function which counts the ratio of assigned to total appearance count per unique object id. We then define the ```recognition``` function that counts how many objects have been seen more times then the desired threshold.
 
 ## Citations