| --- |
| title: README |
| emoji: 👀 |
| colorFrom: gray |
| colorTo: gray |
| sdk: static |
| pinned: false |
| --- |
| |
|
|
| <div align="center"> |
| <img src="https://cdn-uploads.huggingface.co/production/uploads/6567e9af88bfbc261a34673d/riHfcszflP6BzNls2Phrf.png" |
| width=400px |
| style="border-radius: 20px;" |
| /> |
| |
| </div> |
|
|
| <div align="center"> |
| <h1> Want to contribute? </h1> |
| </div> |
|
|
| - 🥙 For naming models, spaces, datasets, and metrics, employ [kebab-case](https://en.wiktionary.org/wiki/kebab_case). Use lowercase letters, except for acronyms, which may be capitalized. |
| - 🔒 Please mind about making the models, spaces and datasets public or private. Metrics can (and have to) be public. |
| - 🤐 Do not expose passwords or tokens, use [secrets](https://huggingface.co/docs/hub/spaces-overview#managing-secrets). |
|
|
| <div align="center"> |
| <h1> Looking for metrics? </h1> |
| </div> |
|
|
| - https://huggingface.co/spaces/SEA-AI/det-metrics |
| - Object detection metrics based on [`pycocotools`](https://github.com/cocodataset/cocoapi) and [torchmetrics' Mean Avergae Precision](https://lightning.ai/docs/torchmetrics/stable/detection/mean_average_precision.html). |
| - https://huggingface.co/spaces/SEA-AI/box-metrics |
| - Bounding box statistics, including IOU, BEP (bottom edge proximity), and others. |
| - https://huggingface.co/spaces/SEA-AI/horizon-metrics |
| - Comparing horizons in an image w.r.t their midpoint and slope errors |
| - https://huggingface.co/spaces/SEA-AI/mot-metrics |
| - Multi-object-tracking metrics using [`py-motmetrics`](https://github.com/cheind/py-motmetrics) |
| - https://huggingface.co/spaces/SEA-AI/panoptic-quality |
| - Evaluating panoptic models |
| |
| <details> |
| <summary>Proposed Metric Output Structure</summary> |
|
|
| Your metric should have the following structure: |
| - `_compute(references, predictions)` |
| - Calls a metric engine, defined for example in the `seametrics` package, or other |
| - `compute_from_payload(paylaod)` |
| - Call the `module.compute` method internally after converting payload -> references, predictions |
| - All the metric's parameters, such as `iou_threshold`, `area_ranges`, etc.. should be moved to the `__init__` method. |
| - Output should look like this: |
| ```json |
| { |
| "ahoy_IR_b2_engine_3_6_0_49_gd81d3b63_oversea": { |
| "overall": { |
| "all": { |
| "f1": 0.15967351103175614, |
| "fn": 2923.0, |
| "fp": 3666.0, |
| "num_gt_ids": 10, |
| "precision": 0.14585274930102515, |
| "recall": 0.1763877148492533, |
| "recognition_0.3": 0.1, |
| "recognition_0.5": 0.1, |
| "recognition_0.8": 0.1, |
| "recognized_0.3": 1, |
| "recognized_0.5": 1, |
| "recognized_0.8": 1, |
| "tp": 626.0 |
| } |
| }, |
| "per_sequence": { |
| "Sentry_2023_02_08_PROACT_CELADON_@6m_MOB_2023_02_08_12_51_49": { |
| "all": { |
| "f1": 0.15967351103175614, |
| "fn": 2923.0, |
| "fp": 3666.0, |
| "num_gt_ids": 10, |
| "precision": 0.14585274930102515, |
| "recall": 0.1763877148492533, |
| "recognition_0.3": 0.1, |
| "recognition_0.5": 0.1, |
| "recognition_0.8": 0.1, |
| "recognized_0.3": 1, |
| "recognized_0.5": 1, |
| "recognized_0.8": 1, |
| "tp": 626.0 |
| } |
| } |
| } |
| } |
| } |
| ``` |
|
|
| </details> |
