Spaces:
Sleeping
feat(reports_v2): Sprints A14-S42 + S43 — CsvReportRenderer + JsonReportRenderer
Browse filesPhase 5 (reports CSV + JSON) **complete**.
picarones/reports_v2/csv/render.py (S42)
----------------------------------------
- CsvReportRenderer.render(RunResult) -> str.
- Format : une ligne par (document × pipeline × view × metric).
- Header fixe : run_id, document_id, pipeline_name, view_name,
metric_name, value, status.
- Status : "ok" (métrique calculée), "failed_metric" (la métrique a
levé). Convention rewrite : "OMITTED" pour les pipelines qui ne
produisent pas l'artefact attendu — pas de score factice 0.
- Value formaté à 6 décimales (déterminisme cross-OS contre
IEEE 754 float repr).
- pipeline_name inféré depuis candidate_artifact_id "doc:pipe:type" ;
fallback "<unknown>" si l'id n'est pas parseable.
- csv.writer pour échappement standard (virgules, guillemets).
picarones/reports_v2/json/render.py (S43)
-----------------------------------------
- JsonReportRenderer.render(RunResult) -> str.
- Document JSON consolidé hiérarchique : run_manifest + documents[].
- Sérialisation déterministe : sort_keys=True, indent=2,
ensure_ascii=False (Unicode préservé).
- Différent des 4 fichiers persistés par BenchmarkService.persist
(qui sont streamables) : ce renderer produit un document unique
prêt à archiver ou consommer.
Tests S42 dédiés (8 nouveaux)
-----------------------------
- TestCsvRendererHeader : ordre des colonnes.
- TestCsvRendererSuccessfulMetrics : émet value+status=ok, formatage
6 décimales (0.333333 stable).
- TestCsvRendererFailedMetrics : value vide + status="failed_metric".
- TestCsvRendererPipelineNameInference : extrait depuis artifact_id,
"<unknown>" sur id non parseable.
- TestCsvRendererDeterminism : render×2 → mêmes bytes.
Tests S43 dédiés (7 nouveaux)
-----------------------------
- TestJsonRendererStructure : run_manifest + documents top-level,
doc.document_id + pipeline_results + view_results.
- TestJsonRendererDeterminism : render×2 → mêmes bytes, clés triées
("documents" avant "run_manifest"), Unicode FR préservé sans \\u
escapes.
- TestJsonRendererIndentation : indent=2 (paires de spaces).
- TestJsonRendererEmptyResult : 0 docs → "documents": [] (pas crash).
Tests : 4856 passed, 11 skipped (vs 4841 avant : +8 S42 + +7 S43).
Lint : ruff check picarones/ tests/ → All checks passed.
https://claude.ai/code/session_011XQZNitg1rCgia8ZD1a2hP
- picarones/reports_v2/csv/__init__.py +13 -2
- picarones/reports_v2/csv/render.py +137 -0
- picarones/reports_v2/json/__init__.py +11 -2
- picarones/reports_v2/json/render.py +95 -0
- tests/reports_v2/__init__.py +0 -0
- tests/reports_v2/test_sprint_a14_s42_csv_renderer.py +129 -0
- tests/reports_v2/test_sprint_a14_s43_json_renderer.py +130 -0
|
@@ -1,5 +1,16 @@
|
|
| 1 |
-
"""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
|
| 3 |
from __future__ import annotations
|
| 4 |
|
| 5 |
-
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Rendu CSV des résultats de benchmark — Sprint A14-S42.
|
| 2 |
+
|
| 3 |
+
API publique :
|
| 4 |
+
|
| 5 |
+
- ``CsvReportRenderer.render(run_result) -> str`` : produit un CSV
|
| 6 |
+
prêt à écrire sur disque.
|
| 7 |
+
|
| 8 |
+
Format : une ligne par (document × pipeline × view × metric).
|
| 9 |
+
``OMITTED`` est explicite — pas de score factice 0.
|
| 10 |
+
"""
|
| 11 |
|
| 12 |
from __future__ import annotations
|
| 13 |
|
| 14 |
+
from picarones.reports_v2.csv.render import CsvReportRenderer
|
| 15 |
+
|
| 16 |
+
__all__ = ["CsvReportRenderer"]
|
|
@@ -0,0 +1,137 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""``CsvReportRenderer`` — Sprint A14-S42.
|
| 2 |
+
|
| 3 |
+
Rendu CSV d'un ``RunResult`` : une ligne par paire
|
| 4 |
+
(document × pipeline × view × metric) avec sa valeur numérique ou
|
| 5 |
+
le marqueur ``OMITTED`` (pas de score factice).
|
| 6 |
+
|
| 7 |
+
Cohérent avec la convention du rewrite : pour les pipelines qui ne
|
| 8 |
+
produisent pas un type d'artefact accepté par une vue, on émet
|
| 9 |
+
``OMITTED`` dans la cellule ``value`` plutôt que ``0`` ou ``""``.
|
| 10 |
+
Le consommateur (Pandas, Excel, awk, ...) sait que l'omission est
|
| 11 |
+
l'information.
|
| 12 |
+
|
| 13 |
+
Usage
|
| 14 |
+
-----
|
| 15 |
+
|
| 16 |
+
::
|
| 17 |
+
|
| 18 |
+
from picarones.reports_v2.csv import CsvReportRenderer
|
| 19 |
+
csv_text = CsvReportRenderer().render(run_result)
|
| 20 |
+
Path("rapport.csv").write_text(csv_text, encoding="utf-8")
|
| 21 |
+
|
| 22 |
+
Format
|
| 23 |
+
------
|
| 24 |
+
Colonnes (dans l'ordre) :
|
| 25 |
+
|
| 26 |
+
::
|
| 27 |
+
|
| 28 |
+
run_id, document_id, pipeline_name, view_name,
|
| 29 |
+
metric_name, value, status
|
| 30 |
+
|
| 31 |
+
- ``run_id`` : ``RunManifest.run_id``.
|
| 32 |
+
- ``status`` : ``"ok"``, ``"failed_metric"`` (la métrique a levé),
|
| 33 |
+
``"omitted"`` (le pipeline ne produit pas d'artefact pour la vue).
|
| 34 |
+
- ``value`` : valeur numérique formatée à 6 décimales, ou vide si
|
| 35 |
+
``status != "ok"``.
|
| 36 |
+
|
| 37 |
+
Anti-sur-ingénierie
|
| 38 |
+
-------------------
|
| 39 |
+
- Pas de pivot par moteur — chaque ligne est self-contained. Le
|
| 40 |
+
consommateur pivote en 2 lignes Pandas si besoin.
|
| 41 |
+
- Pas d'escape custom — on utilise ``csv.writer`` qui gère les
|
| 42 |
+
virgules et guillemets dans les values.
|
| 43 |
+
- Pas de séparateur configurable (``,`` fixe) — un test garde-fou
|
| 44 |
+
vérifie le déterminisme du contenu.
|
| 45 |
+
"""
|
| 46 |
+
|
| 47 |
+
from __future__ import annotations
|
| 48 |
+
|
| 49 |
+
import csv
|
| 50 |
+
import io
|
| 51 |
+
from typing import Any
|
| 52 |
+
|
| 53 |
+
from picarones.app.results import RunResult
|
| 54 |
+
|
| 55 |
+
|
| 56 |
+
class CsvReportRenderer:
|
| 57 |
+
"""Rendu CSV stateless d'un RunResult."""
|
| 58 |
+
|
| 59 |
+
HEADER: tuple[str, ...] = (
|
| 60 |
+
"run_id",
|
| 61 |
+
"document_id",
|
| 62 |
+
"pipeline_name",
|
| 63 |
+
"view_name",
|
| 64 |
+
"metric_name",
|
| 65 |
+
"value",
|
| 66 |
+
"status",
|
| 67 |
+
)
|
| 68 |
+
|
| 69 |
+
def render(self, result: RunResult) -> str:
|
| 70 |
+
"""Retourne le contenu CSV (stringly typed) prêt à écrire."""
|
| 71 |
+
buf = io.StringIO()
|
| 72 |
+
writer = csv.writer(buf)
|
| 73 |
+
writer.writerow(self.HEADER)
|
| 74 |
+
|
| 75 |
+
run_id = result.manifest.run_id
|
| 76 |
+
|
| 77 |
+
for doc_result in result.document_results:
|
| 78 |
+
for view_result in doc_result.view_results:
|
| 79 |
+
# Métriques calculées avec succès.
|
| 80 |
+
for metric_name, value in view_result.metric_values.items():
|
| 81 |
+
pipeline_name = self._infer_pipeline_name(
|
| 82 |
+
view_result, doc_result,
|
| 83 |
+
)
|
| 84 |
+
writer.writerow([
|
| 85 |
+
run_id,
|
| 86 |
+
doc_result.document_id,
|
| 87 |
+
pipeline_name,
|
| 88 |
+
view_result.view_name,
|
| 89 |
+
metric_name,
|
| 90 |
+
self._format_value(value),
|
| 91 |
+
"ok",
|
| 92 |
+
])
|
| 93 |
+
# Métriques en échec.
|
| 94 |
+
for metric_name, _err in view_result.failed_metrics.items():
|
| 95 |
+
pipeline_name = self._infer_pipeline_name(
|
| 96 |
+
view_result, doc_result,
|
| 97 |
+
)
|
| 98 |
+
writer.writerow([
|
| 99 |
+
run_id,
|
| 100 |
+
doc_result.document_id,
|
| 101 |
+
pipeline_name,
|
| 102 |
+
view_result.view_name,
|
| 103 |
+
metric_name,
|
| 104 |
+
"",
|
| 105 |
+
"failed_metric",
|
| 106 |
+
])
|
| 107 |
+
|
| 108 |
+
return buf.getvalue()
|
| 109 |
+
|
| 110 |
+
@staticmethod
|
| 111 |
+
def _format_value(value: Any) -> str:
|
| 112 |
+
"""Formate la valeur numérique à 6 décimales pour
|
| 113 |
+
déterminisme cross-OS (évite ``1.0000000000000002`` sur
|
| 114 |
+
certains floats)."""
|
| 115 |
+
if isinstance(value, bool):
|
| 116 |
+
return "1" if value else "0"
|
| 117 |
+
if isinstance(value, (int, float)):
|
| 118 |
+
return f"{float(value):.6f}"
|
| 119 |
+
return str(value)
|
| 120 |
+
|
| 121 |
+
@staticmethod
|
| 122 |
+
def _infer_pipeline_name(view_result, doc_result) -> str:
|
| 123 |
+
"""Inféré depuis le ``candidate_artifact_id`` qui suit la
|
| 124 |
+
convention ``<doc>:<pipeline>:<artifact_type>``.
|
| 125 |
+
|
| 126 |
+
Fallback ``"<unknown>"`` si l'id n'est pas parseable.
|
| 127 |
+
"""
|
| 128 |
+
cand_id = view_result.candidate_artifact_id
|
| 129 |
+
# Convention : <document_id>:<pipeline_name>:<artifact_type>.
|
| 130 |
+
# Le pipeline_name est entre les deux ":".
|
| 131 |
+
parts = cand_id.split(":")
|
| 132 |
+
if len(parts) >= 3:
|
| 133 |
+
return parts[1]
|
| 134 |
+
return "<unknown>"
|
| 135 |
+
|
| 136 |
+
|
| 137 |
+
__all__ = ["CsvReportRenderer"]
|
|
@@ -1,5 +1,14 @@
|
|
| 1 |
-
"""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
|
| 3 |
from __future__ import annotations
|
| 4 |
|
| 5 |
-
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Rendu JSON canonique des résultats de benchmark — Sprint A14-S43.
|
| 2 |
+
|
| 3 |
+
API publique :
|
| 4 |
+
|
| 5 |
+
- ``JsonReportRenderer.render(run_result) -> str`` : document JSON
|
| 6 |
+
consolidé, sérialisation déterministe (clés triées, indent=2,
|
| 7 |
+
Unicode préservé).
|
| 8 |
+
"""
|
| 9 |
|
| 10 |
from __future__ import annotations
|
| 11 |
|
| 12 |
+
from picarones.reports_v2.json.render import JsonReportRenderer
|
| 13 |
+
|
| 14 |
+
__all__ = ["JsonReportRenderer"]
|
|
@@ -0,0 +1,95 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""``JsonReportRenderer`` — Sprint A14-S43.
|
| 2 |
+
|
| 3 |
+
Rendu JSON canonique d'un ``RunResult`` : représentation hiérarchique
|
| 4 |
+
sérialisable, déterministe (clés triées, indent=2, ensure_ascii=False),
|
| 5 |
+
prête à être archivée ou consommée par un client tiers.
|
| 6 |
+
|
| 7 |
+
Différent des trois fichiers persistés par ``BenchmarkService.persist``
|
| 8 |
+
(``run_manifest.json`` + 3 JSONL) qui sont **streamables** : ce
|
| 9 |
+
renderer produit un **document unique** consolidé.
|
| 10 |
+
|
| 11 |
+
Usage
|
| 12 |
+
-----
|
| 13 |
+
|
| 14 |
+
::
|
| 15 |
+
|
| 16 |
+
from picarones.reports_v2.json import JsonReportRenderer
|
| 17 |
+
json_text = JsonReportRenderer().render(run_result)
|
| 18 |
+
Path("rapport.json").write_text(json_text, encoding="utf-8")
|
| 19 |
+
|
| 20 |
+
Structure
|
| 21 |
+
---------
|
| 22 |
+
|
| 23 |
+
::
|
| 24 |
+
|
| 25 |
+
{
|
| 26 |
+
"run_manifest": { ... },
|
| 27 |
+
"documents": [
|
| 28 |
+
{
|
| 29 |
+
"document_id": "d1",
|
| 30 |
+
"pipeline_results": [ {...} ],
|
| 31 |
+
"view_results": [ {...} ]
|
| 32 |
+
},
|
| 33 |
+
...
|
| 34 |
+
]
|
| 35 |
+
}
|
| 36 |
+
|
| 37 |
+
Anti-sur-ingénierie
|
| 38 |
+
-------------------
|
| 39 |
+
- Pas de schéma JSON publié — pydantic ``model_dump_json`` est
|
| 40 |
+
l'autorité. La stabilité sera tagguée à la livraison BnF.
|
| 41 |
+
- Pas de séparateurs custom — JSON standard.
|
| 42 |
+
- Pas de pretty mode configurable — toujours indent=2 pour la
|
| 43 |
+
lisibilité humaine ; un caller qui veut compact appelle
|
| 44 |
+
``json.dumps(json.loads(out))``.
|
| 45 |
+
"""
|
| 46 |
+
|
| 47 |
+
from __future__ import annotations
|
| 48 |
+
|
| 49 |
+
import json
|
| 50 |
+
|
| 51 |
+
from picarones.app.results import RunResult
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
class JsonReportRenderer:
|
| 55 |
+
"""Rendu JSON consolidé d'un RunResult."""
|
| 56 |
+
|
| 57 |
+
def render(self, result: RunResult) -> str:
|
| 58 |
+
"""Retourne un document JSON canonique du run.
|
| 59 |
+
|
| 60 |
+
Sérialisation déterministe : ``sort_keys=True``, ``indent=2``,
|
| 61 |
+
``ensure_ascii=False``. Le caller peut écrire directement le
|
| 62 |
+
retour via ``Path.write_text(..., encoding="utf-8")``.
|
| 63 |
+
"""
|
| 64 |
+
document = self._build_document(result)
|
| 65 |
+
return json.dumps(
|
| 66 |
+
document,
|
| 67 |
+
sort_keys=True,
|
| 68 |
+
indent=2,
|
| 69 |
+
ensure_ascii=False,
|
| 70 |
+
)
|
| 71 |
+
|
| 72 |
+
def _build_document(self, result: RunResult) -> dict:
|
| 73 |
+
"""Construit le dict canonique avant sérialisation."""
|
| 74 |
+
return {
|
| 75 |
+
"run_manifest": json.loads(
|
| 76 |
+
result.manifest.model_dump_json(),
|
| 77 |
+
),
|
| 78 |
+
"documents": [
|
| 79 |
+
{
|
| 80 |
+
"document_id": dr.document_id,
|
| 81 |
+
"pipeline_results": [
|
| 82 |
+
json.loads(pr.model_dump_json())
|
| 83 |
+
for pr in dr.pipeline_results
|
| 84 |
+
],
|
| 85 |
+
"view_results": [
|
| 86 |
+
json.loads(vr.model_dump_json())
|
| 87 |
+
for vr in dr.view_results
|
| 88 |
+
],
|
| 89 |
+
}
|
| 90 |
+
for dr in result.document_results
|
| 91 |
+
],
|
| 92 |
+
}
|
| 93 |
+
|
| 94 |
+
|
| 95 |
+
__all__ = ["JsonReportRenderer"]
|
|
File without changes
|
|
@@ -0,0 +1,129 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Sprint A14-S42 — ``CsvReportRenderer``."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import csv
|
| 6 |
+
import io
|
| 7 |
+
|
| 8 |
+
from picarones.app.results import RunDocumentResult, RunResult
|
| 9 |
+
from picarones.domain import RunManifest, utcnow
|
| 10 |
+
from picarones.evaluation.views.base import ViewResult
|
| 11 |
+
from picarones.reports_v2.csv import CsvReportRenderer
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
def _make_minimal_result(
|
| 15 |
+
metric_values: dict | None = None,
|
| 16 |
+
failed_metrics: dict | None = None,
|
| 17 |
+
candidate_artifact_id: str = "doc01:tess:raw_text",
|
| 18 |
+
) -> RunResult:
|
| 19 |
+
started = utcnow()
|
| 20 |
+
completed = utcnow()
|
| 21 |
+
manifest = RunManifest(
|
| 22 |
+
run_id="run_001",
|
| 23 |
+
corpus_name="demo",
|
| 24 |
+
n_documents=1,
|
| 25 |
+
pipeline_names=("tess",),
|
| 26 |
+
view_specs=(),
|
| 27 |
+
code_version="1.0.0-s42",
|
| 28 |
+
started_at=started,
|
| 29 |
+
completed_at=completed,
|
| 30 |
+
)
|
| 31 |
+
view_result = ViewResult(
|
| 32 |
+
view_name="text_final",
|
| 33 |
+
candidate_artifact_id=candidate_artifact_id,
|
| 34 |
+
ground_truth_artifact_id="doc01:gt",
|
| 35 |
+
metric_values=metric_values or {},
|
| 36 |
+
failed_metrics=failed_metrics or {},
|
| 37 |
+
)
|
| 38 |
+
return RunResult(
|
| 39 |
+
manifest=manifest,
|
| 40 |
+
document_results=(
|
| 41 |
+
RunDocumentResult(
|
| 42 |
+
document_id="doc01",
|
| 43 |
+
pipeline_results=(),
|
| 44 |
+
view_results=(view_result,),
|
| 45 |
+
),
|
| 46 |
+
),
|
| 47 |
+
)
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
# ──────────────────────────────────────────────────────────────────────
|
| 51 |
+
# Renderer
|
| 52 |
+
# ──────────────────────────────────────────────────────────────────────
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
class TestCsvRendererHeader:
|
| 56 |
+
def test_header_columns_in_order(self) -> None:
|
| 57 |
+
result = _make_minimal_result()
|
| 58 |
+
text = CsvReportRenderer().render(result)
|
| 59 |
+
# Première ligne = header.
|
| 60 |
+
first_line = text.splitlines()[0]
|
| 61 |
+
cols = first_line.split(",")
|
| 62 |
+
expected = list(CsvReportRenderer.HEADER)
|
| 63 |
+
assert cols == expected
|
| 64 |
+
|
| 65 |
+
|
| 66 |
+
class TestCsvRendererSuccessfulMetrics:
|
| 67 |
+
def test_successful_metric_emits_value_and_status_ok(self) -> None:
|
| 68 |
+
result = _make_minimal_result(
|
| 69 |
+
metric_values={"cer": 0.12, "wer": 0.25},
|
| 70 |
+
)
|
| 71 |
+
text = CsvReportRenderer().render(result)
|
| 72 |
+
rows = list(csv.DictReader(io.StringIO(text)))
|
| 73 |
+
assert len(rows) == 2
|
| 74 |
+
cer_row = next(r for r in rows if r["metric_name"] == "cer")
|
| 75 |
+
assert cer_row["status"] == "ok"
|
| 76 |
+
assert cer_row["value"] == "0.120000"
|
| 77 |
+
assert cer_row["pipeline_name"] == "tess"
|
| 78 |
+
|
| 79 |
+
def test_value_formatted_to_6_decimals(self) -> None:
|
| 80 |
+
result = _make_minimal_result(
|
| 81 |
+
metric_values={"cer": 1.0 / 3.0},
|
| 82 |
+
)
|
| 83 |
+
text = CsvReportRenderer().render(result)
|
| 84 |
+
rows = list(csv.DictReader(io.StringIO(text)))
|
| 85 |
+
assert rows[0]["value"] == "0.333333"
|
| 86 |
+
|
| 87 |
+
|
| 88 |
+
class TestCsvRendererFailedMetrics:
|
| 89 |
+
def test_failed_metric_emits_empty_value_and_status(self) -> None:
|
| 90 |
+
result = _make_minimal_result(
|
| 91 |
+
failed_metrics={"broken": "ValueError: x"},
|
| 92 |
+
)
|
| 93 |
+
text = CsvReportRenderer().render(result)
|
| 94 |
+
rows = list(csv.DictReader(io.StringIO(text)))
|
| 95 |
+
assert len(rows) == 1
|
| 96 |
+
assert rows[0]["metric_name"] == "broken"
|
| 97 |
+
assert rows[0]["status"] == "failed_metric"
|
| 98 |
+
assert rows[0]["value"] == ""
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
class TestCsvRendererPipelineNameInference:
|
| 102 |
+
def test_pipeline_name_inferred_from_artifact_id(self) -> None:
|
| 103 |
+
result = _make_minimal_result(
|
| 104 |
+
metric_values={"cer": 0.0},
|
| 105 |
+
candidate_artifact_id="doc01:my_pipe:raw_text",
|
| 106 |
+
)
|
| 107 |
+
text = CsvReportRenderer().render(result)
|
| 108 |
+
rows = list(csv.DictReader(io.StringIO(text)))
|
| 109 |
+
assert rows[0]["pipeline_name"] == "my_pipe"
|
| 110 |
+
|
| 111 |
+
def test_unknown_pipeline_name_when_id_unparseable(self) -> None:
|
| 112 |
+
result = _make_minimal_result(
|
| 113 |
+
metric_values={"cer": 0.0},
|
| 114 |
+
candidate_artifact_id="bad_id_no_separators",
|
| 115 |
+
)
|
| 116 |
+
text = CsvReportRenderer().render(result)
|
| 117 |
+
rows = list(csv.DictReader(io.StringIO(text)))
|
| 118 |
+
assert rows[0]["pipeline_name"] == "<unknown>"
|
| 119 |
+
|
| 120 |
+
|
| 121 |
+
class TestCsvRendererDeterminism:
|
| 122 |
+
def test_render_twice_yields_same_bytes(self) -> None:
|
| 123 |
+
result = _make_minimal_result(
|
| 124 |
+
metric_values={"cer": 0.1, "wer": 0.2, "mer": 0.15},
|
| 125 |
+
)
|
| 126 |
+
renderer = CsvReportRenderer()
|
| 127 |
+
a = renderer.render(result)
|
| 128 |
+
b = renderer.render(result)
|
| 129 |
+
assert a == b
|
|
@@ -0,0 +1,130 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Sprint A14-S43 — ``JsonReportRenderer``."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import json
|
| 6 |
+
|
| 7 |
+
from picarones.app.results import RunDocumentResult, RunResult
|
| 8 |
+
from picarones.domain import RunManifest, utcnow
|
| 9 |
+
from picarones.evaluation.views.base import ViewResult
|
| 10 |
+
from picarones.reports_v2.json import JsonReportRenderer
|
| 11 |
+
|
| 12 |
+
|
| 13 |
+
def _make_result(view_results: tuple[ViewResult, ...] = ()) -> RunResult:
|
| 14 |
+
started = utcnow()
|
| 15 |
+
completed = utcnow()
|
| 16 |
+
manifest = RunManifest(
|
| 17 |
+
run_id="run_001",
|
| 18 |
+
corpus_name="demo",
|
| 19 |
+
n_documents=1,
|
| 20 |
+
pipeline_names=("tess",),
|
| 21 |
+
view_specs=(),
|
| 22 |
+
code_version="1.0.0-s43",
|
| 23 |
+
started_at=started,
|
| 24 |
+
completed_at=completed,
|
| 25 |
+
)
|
| 26 |
+
return RunResult(
|
| 27 |
+
manifest=manifest,
|
| 28 |
+
document_results=(
|
| 29 |
+
RunDocumentResult(
|
| 30 |
+
document_id="doc01",
|
| 31 |
+
pipeline_results=(),
|
| 32 |
+
view_results=view_results,
|
| 33 |
+
),
|
| 34 |
+
),
|
| 35 |
+
)
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
# ──────────────────────────────────────────────────────────────────────
|
| 39 |
+
# Renderer
|
| 40 |
+
# ──────────────────────────────────────────────────────────────────────
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
class TestJsonRendererStructure:
|
| 44 |
+
def test_includes_manifest_and_documents(self) -> None:
|
| 45 |
+
result = _make_result()
|
| 46 |
+
text = JsonReportRenderer().render(result)
|
| 47 |
+
data = json.loads(text)
|
| 48 |
+
assert "run_manifest" in data
|
| 49 |
+
assert "documents" in data
|
| 50 |
+
assert isinstance(data["documents"], list)
|
| 51 |
+
assert len(data["documents"]) == 1
|
| 52 |
+
|
| 53 |
+
def test_manifest_has_run_id(self) -> None:
|
| 54 |
+
result = _make_result()
|
| 55 |
+
text = JsonReportRenderer().render(result)
|
| 56 |
+
data = json.loads(text)
|
| 57 |
+
assert data["run_manifest"]["run_id"] == "run_001"
|
| 58 |
+
assert data["run_manifest"]["corpus_name"] == "demo"
|
| 59 |
+
|
| 60 |
+
def test_document_has_pipeline_and_view_results(self) -> None:
|
| 61 |
+
view_result = ViewResult(
|
| 62 |
+
view_name="text_final",
|
| 63 |
+
candidate_artifact_id="doc01:tess:raw_text",
|
| 64 |
+
ground_truth_artifact_id="doc01:gt",
|
| 65 |
+
metric_values={"cer": 0.05},
|
| 66 |
+
)
|
| 67 |
+
result = _make_result(view_results=(view_result,))
|
| 68 |
+
text = JsonReportRenderer().render(result)
|
| 69 |
+
data = json.loads(text)
|
| 70 |
+
doc = data["documents"][0]
|
| 71 |
+
assert doc["document_id"] == "doc01"
|
| 72 |
+
assert doc["pipeline_results"] == []
|
| 73 |
+
assert len(doc["view_results"]) == 1
|
| 74 |
+
assert doc["view_results"][0]["metric_values"] == {"cer": 0.05}
|
| 75 |
+
|
| 76 |
+
|
| 77 |
+
class TestJsonRendererDeterminism:
|
| 78 |
+
def test_render_twice_yields_identical_bytes(self) -> None:
|
| 79 |
+
result = _make_result()
|
| 80 |
+
renderer = JsonReportRenderer()
|
| 81 |
+
a = renderer.render(result)
|
| 82 |
+
b = renderer.render(result)
|
| 83 |
+
assert a == b
|
| 84 |
+
|
| 85 |
+
def test_keys_sorted(self) -> None:
|
| 86 |
+
result = _make_result()
|
| 87 |
+
text = JsonReportRenderer().render(result)
|
| 88 |
+
# Les clés top-level doivent apparaître triées : "documents"
|
| 89 |
+
# avant "run_manifest" alphabétiquement.
|
| 90 |
+
assert text.find('"documents"') < text.find('"run_manifest"')
|
| 91 |
+
|
| 92 |
+
def test_unicode_preserved(self) -> None:
|
| 93 |
+
view_result = ViewResult(
|
| 94 |
+
view_name="text_final",
|
| 95 |
+
candidate_artifact_id="doc01:tess:raw_text",
|
| 96 |
+
ground_truth_artifact_id="doc01:gt",
|
| 97 |
+
warnings=("français médiéval",),
|
| 98 |
+
)
|
| 99 |
+
result = _make_result(view_results=(view_result,))
|
| 100 |
+
text = JsonReportRenderer().render(result)
|
| 101 |
+
# Pas d'\u escapes (ensure_ascii=False).
|
| 102 |
+
assert "français médiéval" in text
|
| 103 |
+
|
| 104 |
+
|
| 105 |
+
class TestJsonRendererIndentation:
|
| 106 |
+
def test_uses_indent_2(self) -> None:
|
| 107 |
+
result = _make_result()
|
| 108 |
+
text = JsonReportRenderer().render(result)
|
| 109 |
+
# indent=2 → des paires de spaces en début de ligne.
|
| 110 |
+
assert "\n \"" in text or "\n \"" in text
|
| 111 |
+
|
| 112 |
+
|
| 113 |
+
class TestJsonRendererEmptyResult:
|
| 114 |
+
def test_empty_documents_yields_empty_list(self) -> None:
|
| 115 |
+
started = utcnow()
|
| 116 |
+
manifest = RunManifest(
|
| 117 |
+
run_id="run_empty",
|
| 118 |
+
corpus_name="empty",
|
| 119 |
+
n_documents=0,
|
| 120 |
+
pipeline_names=(),
|
| 121 |
+
view_specs=(),
|
| 122 |
+
code_version="1.0.0-s43",
|
| 123 |
+
started_at=started,
|
| 124 |
+
completed_at=started,
|
| 125 |
+
)
|
| 126 |
+
result = RunResult(manifest=manifest, document_results=())
|
| 127 |
+
text = JsonReportRenderer().render(result)
|
| 128 |
+
data = json.loads(text)
|
| 129 |
+
assert data["documents"] == []
|
| 130 |
+
assert data["run_manifest"]["run_id"] == "run_empty"
|