Spaces:
Running on Zero
fix: enforce semantic facets and consolidate overlapping spans
Browse filesImplemented HUDA-Net v39.0.1 semantic-facet contracts.
Changes:
- Added locked domain-neutral facets beside the central answer operator.
- Separated facet terms from mandatory subject coverage.
- Added temporal, spatial, financial, health, polarity, direction, stage,
and boundary facet families.
- Rejected opposite-dimension evidence regardless of higher retrieval scores.
- Added source-level and item-level facet validation for list synthesis.
- Prevented complementary set recovery from repairing facet mismatches.
- Added global-arbiter enforcement for every used source.
- Consolidated broad and contained temporal spans into one canonical list slot.
- Added Arabic and English adversarial facet regressions.
- Preserved all v39 central query-contract and earlier safety layers.
- DEPLOY_HUDANET_V39_0_1.md +72 -0
- app.py +70 -16
- hudanet_core/README.md +8 -2
- hudanet_core/__init__.py +1 -1
- hudanet_core/compatibility.py +23 -0
- hudanet_core/facet_contract.py +220 -0
- hudanet_core/global_arbiter.py +10 -1
- hudanet_core/pipeline.py +10 -0
- hudanet_core/query.py +7 -2
- hudanet_core/query_contract.py +14 -1
- hudanet_core/resources/ranking_config.json +29 -23
- hudanet_core/set_coverage.py +132 -2
- hudanet_core/tests/test_generic_pipeline.py +63 -0
- hudanet_core/types.py +5 -0
- hudanet_v39_0_1_facet_contract_regression.json +79 -0
- hudanet_v39_0_1_generic_tests.json +0 -0
- hudanet_v39_0_1_manifest.json +212 -0
- hudanet_v39_0_1_query_contract_regression.json +83 -0
- hudanet_v39_0_1_validation_report.json +33 -0
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Deploy HUDA-Net v39.0.1
|
| 2 |
+
|
| 3 |
+
Upload every file in this package to the Hugging Face Space while preserving the `hudanet_core` directory structure. Do not upload `app.py` alone.
|
| 4 |
+
|
| 5 |
+
## Required deployment steps
|
| 6 |
+
|
| 7 |
+
1. Replace the existing repository files with all files from the ZIP package.
|
| 8 |
+
2. Keep `hudanet_core/resources` and `hudanet_core/tests` in their original locations.
|
| 9 |
+
3. Commit the files directly to the active Space branch.
|
| 10 |
+
4. Run **Factory reboot** after the commit finishes.
|
| 11 |
+
5. Confirm that the lightweight preflight passes before the private Runtime download begins.
|
| 12 |
+
|
| 13 |
+
Expected pre-download line:
|
| 14 |
+
|
| 15 |
+
```text
|
| 16 |
+
โ
HUDA-Net v39.0.1 central-query-contract, semantic-facet-contract, global-arbiter, proposition, coverage-aware-set, list-canonicalization, contextual-sense, text-integrity, central-ruling-frame, relation-graph, issue-clustering, semantic-alignment, and compare-mode preflight passed before runtime download
|
| 17 |
+
```
|
| 18 |
+
|
| 19 |
+
Expected post-Runtime self-test line:
|
| 20 |
+
|
| 21 |
+
```text
|
| 22 |
+
โ
HUDA-Net v39.0.1 generic proposition self-test passed: 19 checks
|
| 23 |
+
```
|
| 24 |
+
|
| 25 |
+
## Main architectural change
|
| 26 |
+
|
| 27 |
+
v39.0.1 extends the central query contract with a locked semantic facet. An explicit qualifier such as temporal/spatial, financial/health, positive/negative, direct/indirect, initial/final, or internal/external is separated from the subject and validated independently.
|
| 28 |
+
|
| 29 |
+
The pipeline now:
|
| 30 |
+
|
| 31 |
+
- rejects evidence that answers the opposite facet even when its neural and lexical scores are higher;
|
| 32 |
+
- validates the facet of every list source and every list member;
|
| 33 |
+
- prevents complementary-set rescue from bypassing a facet mismatch;
|
| 34 |
+
- checks facet compatibility again in the global answer arbiter;
|
| 35 |
+
- consolidates overlapping temporal spans instead of displaying a broad period and its contained subperiod as contradictory peer items;
|
| 36 |
+
- preserves the central answer-operator contract, OCR integrity, contextual sense, list canonicalization, ruling frames, filter contracts, and compare mode.
|
| 37 |
+
|
| 38 |
+
No Hajj/Umrah place, month, question, answer, or scenario dictionary was added to the production core. The exact reported question appears only in the regression report.
|
| 39 |
+
|
| 40 |
+
## Suggested commit
|
| 41 |
+
|
| 42 |
+
### Title
|
| 43 |
+
|
| 44 |
+
```text
|
| 45 |
+
fix: enforce semantic facets and consolidate overlapping spans
|
| 46 |
+
```
|
| 47 |
+
|
| 48 |
+
### Description
|
| 49 |
+
|
| 50 |
+
```text
|
| 51 |
+
Implemented HUDA-Net v39.0.1 semantic-facet contracts.
|
| 52 |
+
|
| 53 |
+
Changes:
|
| 54 |
+
- Added locked domain-neutral facets beside the central answer operator.
|
| 55 |
+
- Separated facet terms from mandatory subject coverage.
|
| 56 |
+
- Added temporal, spatial, financial, health, polarity, direction, stage, and boundary facet families.
|
| 57 |
+
- Rejected opposite-dimension evidence regardless of higher retrieval scores.
|
| 58 |
+
- Added source-level and item-level facet validation for list synthesis.
|
| 59 |
+
- Prevented complementary set recovery from repairing facet mismatches.
|
| 60 |
+
- Added global-arbiter enforcement for every used source.
|
| 61 |
+
- Consolidated broad and contained temporal spans into one canonical list slot.
|
| 62 |
+
- Added Arabic and English adversarial facet regressions.
|
| 63 |
+
- Preserved all v39 central query-contract and earlier safety layers.
|
| 64 |
+
```
|
| 65 |
+
|
| 66 |
+
## Included reports
|
| 67 |
+
|
| 68 |
+
- `hudanet_v39_0_1_facet_contract_regression.json`
|
| 69 |
+
- `hudanet_v39_0_1_query_contract_regression.json`
|
| 70 |
+
- `hudanet_v39_0_1_generic_tests.json`
|
| 71 |
+
- `hudanet_v39_0_1_validation_report.json`
|
| 72 |
+
- `hudanet_v39_0_1_manifest.json`
|
|
@@ -1,6 +1,6 @@
|
|
| 1 |
# -*- coding: utf-8 -*-
|
| 2 |
"""
|
| 3 |
-
HUDA-Net Central Query Contract v39.0.
|
| 4 |
=============================
|
| 5 |
Deploy this file as app.py in a Hugging Face Space and add HF_TOKEN as a
|
| 6 |
read-only Space secret.
|
|
@@ -87,7 +87,7 @@ _emitted_ruling_filter_ids = {name for lang in ("ar", "en") for name, _ in RULIN
|
|
| 87 |
_missing_ruling_filter_ids = sorted(_emitted_ruling_filter_ids - set(_RULING_FILTER_DISPLAY_IDS))
|
| 88 |
if _missing_ruling_filter_ids:
|
| 89 |
raise RuntimeError(
|
| 90 |
-
"HUDA-Net v39.0.
|
| 91 |
+ json.dumps(_missing_ruling_filter_ids, ensure_ascii=False)
|
| 92 |
)
|
| 93 |
|
|
@@ -275,7 +275,7 @@ def _generic_proposition_preflight() -> None:
|
|
| 275 |
)
|
| 276 |
if not unrelated_rejected or not required_roles.issubset(roles):
|
| 277 |
raise RuntimeError(
|
| 278 |
-
"HUDA-Net v39.0.
|
| 279 |
+ json.dumps(
|
| 280 |
{
|
| 281 |
"unrelated_rejected": unrelated_rejected,
|
|
@@ -325,7 +325,7 @@ def _generic_proposition_preflight() -> None:
|
|
| 325 |
)
|
| 326 |
if not definition_ok:
|
| 327 |
raise RuntimeError(
|
| 328 |
-
"HUDA-Net v39.0.
|
| 329 |
+ json.dumps(
|
| 330 |
{
|
| 331 |
"answer": definition_result.answer,
|
|
@@ -382,7 +382,7 @@ def _generic_proposition_preflight() -> None:
|
|
| 382 |
)
|
| 383 |
if not relevant_passed or not distractor_rejected or not residue_pruned:
|
| 384 |
raise RuntimeError(
|
| 385 |
-
"HUDA-Net v39.0.
|
| 386 |
+ json.dumps(
|
| 387 |
{
|
| 388 |
"relevant_passed": relevant_passed,
|
|
@@ -456,7 +456,7 @@ def _generic_proposition_preflight() -> None:
|
|
| 456 |
)
|
| 457 |
if not principle_ok:
|
| 458 |
raise RuntimeError(
|
| 459 |
-
"HUDA-Net v39.0.
|
| 460 |
+ json.dumps(
|
| 461 |
{
|
| 462 |
"request_type": principle_result.query.primary_request_type,
|
|
@@ -538,7 +538,7 @@ def _generic_proposition_preflight() -> None:
|
|
| 538 |
)
|
| 539 |
if not sense_main_ok or not homograph_rejected or not ocr_rejected or not ruling_frame_ok:
|
| 540 |
raise RuntimeError(
|
| 541 |
-
"HUDA-Net v39.0.
|
| 542 |
+ json.dumps(
|
| 543 |
{
|
| 544 |
"sense_main_ok": sense_main_ok,
|
|
@@ -596,7 +596,7 @@ def _generic_proposition_preflight() -> None:
|
|
| 596 |
)
|
| 597 |
if not set_ok:
|
| 598 |
raise RuntimeError(
|
| 599 |
-
"HUDA-Net v39.0.
|
| 600 |
+ json.dumps({"answer": set_result.answer, "details": set_result.details}, ensure_ascii=False)
|
| 601 |
)
|
| 602 |
|
|
@@ -645,10 +645,64 @@ def _generic_proposition_preflight() -> None:
|
|
| 645 |
)
|
| 646 |
if not canonical_ok:
|
| 647 |
raise RuntimeError(
|
| 648 |
-
"HUDA-Net v39.0.
|
| 649 |
+ json.dumps({"answer": canonical_answer, "details": canonical_result.details}, ensure_ascii=False)
|
| 650 |
)
|
| 651 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 652 |
# Validate the compare-sources contract with coherent generic evidence before
|
| 653 |
# downloading the multi-gigabyte runtime. This catches audit-fixture regressions
|
| 654 |
# immediately and verifies real synthesis rather than bypassing the gate.
|
|
@@ -683,7 +737,7 @@ def _generic_proposition_preflight() -> None:
|
|
| 683 |
primary_ok = ("Book 1" in primary_only.answer) ^ ("Book 2" in primary_only.answer)
|
| 684 |
if not compare_ok or not primary_ok or compared.answer == primary_only.answer:
|
| 685 |
raise RuntimeError(
|
| 686 |
-
"HUDA-Net v39.0.
|
| 687 |
+ json.dumps(
|
| 688 |
{
|
| 689 |
"compare_ok": compare_ok,
|
|
@@ -695,7 +749,7 @@ def _generic_proposition_preflight() -> None:
|
|
| 695 |
)
|
| 696 |
)
|
| 697 |
print(
|
| 698 |
-
"โ
HUDA-Net v39.0.
|
| 699 |
"passed before runtime download"
|
| 700 |
)
|
| 701 |
|
|
@@ -712,7 +766,7 @@ def _zerogpu_registration_only():
|
|
| 712 |
|
| 713 |
_download_hudanet_private_datasets()
|
| 714 |
|
| 715 |
-
VERSION = "39.0.
|
| 716 |
CONFIG = {
|
| 717 |
"INPUT_ROOT": "/kaggle/input",
|
| 718 |
"WORK_ROOT": "/tmp/hudanet_v27",
|
|
@@ -3502,7 +3556,7 @@ import pandas as pd
|
|
| 3502 |
from scipy import sparse
|
| 3503 |
import joblib
|
| 3504 |
|
| 3505 |
-
UI_VERSION = "39.0.
|
| 3506 |
UI_CONFIG = {
|
| 3507 |
"INPUT_ROOT": "/kaggle/input",
|
| 3508 |
"RUNTIME_DATASET_SLUG": "hudanet-bilingual-certified-runtime",
|
|
@@ -4156,7 +4210,7 @@ for _lang, _labels in RULING_FILTER_LABELS.items():
|
|
| 4156 |
_extra_labels = sorted(set(_labels) - set(_RULING_FILTER_DISPLAY_IDS))
|
| 4157 |
if _missing_labels or _extra_labels:
|
| 4158 |
raise RuntimeError(
|
| 4159 |
-
f"HUDA-Net v39.0.
|
| 4160 |
+ json.dumps({"missing": _missing_labels, "extra": _extra_labels}, ensure_ascii=False)
|
| 4161 |
)
|
| 4162 |
RULING_FILTER_KEYS = _RULING_FILTER_DISPLAY_IDS
|
|
@@ -6051,8 +6105,8 @@ def validate_answer_quality_v36_4() -> dict:
|
|
| 6051 |
|
| 6052 |
failed=[item for item in checks if not item["passed"]]
|
| 6053 |
if failed:
|
| 6054 |
-
raise RuntimeError("HUDA-Net v39.0.
|
| 6055 |
-
print(f"โ
HUDA-Net v39.0.
|
| 6056 |
return {"passed":True,"tested":len(checks),"checks":checks}
|
| 6057 |
|
| 6058 |
def validate_specificity_guard_v33(engine: ProfessionalEvidenceEngine) -> dict:
|
|
|
|
| 1 |
# -*- coding: utf-8 -*-
|
| 2 |
"""
|
| 3 |
+
HUDA-Net Central Query Contract v39.0.1 Academic Integrated โ Gradio Stable
|
| 4 |
=============================
|
| 5 |
Deploy this file as app.py in a Hugging Face Space and add HF_TOKEN as a
|
| 6 |
read-only Space secret.
|
|
|
|
| 87 |
_missing_ruling_filter_ids = sorted(_emitted_ruling_filter_ids - set(_RULING_FILTER_DISPLAY_IDS))
|
| 88 |
if _missing_ruling_filter_ids:
|
| 89 |
raise RuntimeError(
|
| 90 |
+
"HUDA-Net v39.0.1 ruling-filter coverage preflight failed before runtime download: "
|
| 91 |
+ json.dumps(_missing_ruling_filter_ids, ensure_ascii=False)
|
| 92 |
)
|
| 93 |
|
|
|
|
| 275 |
)
|
| 276 |
if not unrelated_rejected or not required_roles.issubset(roles):
|
| 277 |
raise RuntimeError(
|
| 278 |
+
"HUDA-Net v39.0.1 preflight failed before runtime download: "
|
| 279 |
+ json.dumps(
|
| 280 |
{
|
| 281 |
"unrelated_rejected": unrelated_rejected,
|
|
|
|
| 325 |
)
|
| 326 |
if not definition_ok:
|
| 327 |
raise RuntimeError(
|
| 328 |
+
"HUDA-Net v39.0.1 central query-contract preflight failed before runtime download: "
|
| 329 |
+ json.dumps(
|
| 330 |
{
|
| 331 |
"answer": definition_result.answer,
|
|
|
|
| 382 |
)
|
| 383 |
if not relevant_passed or not distractor_rejected or not residue_pruned:
|
| 384 |
raise RuntimeError(
|
| 385 |
+
"HUDA-Net v39.0.1 semantic-alignment preflight failed before runtime download: "
|
| 386 |
+ json.dumps(
|
| 387 |
{
|
| 388 |
"relevant_passed": relevant_passed,
|
|
|
|
| 456 |
)
|
| 457 |
if not principle_ok:
|
| 458 |
raise RuntimeError(
|
| 459 |
+
"HUDA-Net v39.0.1 hierarchical principle preflight failed before runtime download: "
|
| 460 |
+ json.dumps(
|
| 461 |
{
|
| 462 |
"request_type": principle_result.query.primary_request_type,
|
|
|
|
| 538 |
)
|
| 539 |
if not sense_main_ok or not homograph_rejected or not ocr_rejected or not ruling_frame_ok:
|
| 540 |
raise RuntimeError(
|
| 541 |
+
"HUDA-Net v39.0.1 contextual-sense, integrity, and ruling-frame preflight failed before runtime download: "
|
| 542 |
+ json.dumps(
|
| 543 |
{
|
| 544 |
"sense_main_ok": sense_main_ok,
|
|
|
|
| 596 |
)
|
| 597 |
if not set_ok:
|
| 598 |
raise RuntimeError(
|
| 599 |
+
"HUDA-Net v39.0.1 coverage-aware set preflight failed before runtime download: "
|
| 600 |
+ json.dumps({"answer": set_result.answer, "details": set_result.details}, ensure_ascii=False)
|
| 601 |
)
|
| 602 |
|
|
|
|
| 645 |
)
|
| 646 |
if not canonical_ok:
|
| 647 |
raise RuntimeError(
|
| 648 |
+
"HUDA-Net v39.0.1 schema-constrained list canonicalization preflight failed before runtime download: "
|
| 649 |
+ json.dumps({"answer": canonical_answer, "details": canonical_result.details}, ensure_ascii=False)
|
| 650 |
)
|
| 651 |
|
| 652 |
+
# Validate the locked semantic-facet contract before mounting the heavy
|
| 653 |
+
# Runtime. The spatial distractor is intentionally stronger than the temporal
|
| 654 |
+
# sources, and a broad temporal span overlaps a more specific span.
|
| 655 |
+
facet_query = "ู
ุง ูู ุงูุญุฏูุฏ ุงูุฒู
ููุฉ ููุนู
ููุฉุ"
|
| 656 |
+
facet_sources = [
|
| 657 |
+
{
|
| 658 |
+
"record_id":"preflight-facet-time-specific", "book_id":"facet-time-specific", "book":"ูุชุงุจ ุงูุฒู
ู ุงูู
ูุตู",
|
| 659 |
+
"title":"ุงูุญุฏูุฏ ุงูุฒู
ููุฉ ููุนู
ููุฉ", "question":facet_query, "ruling":"ุชุญุฏูุฏ ุฒู
ูู",
|
| 660 |
+
"answer":"ุงูุญุฏูุฏ ุงูุฒู
ููุฉ ูู: - ุงูุดูุฑ ุงูุฃูู. - ุงูุดูุฑ ุงูุซุงูู. - ุฃูู ุนุดุฑุฉ ุฃูุงู
ู
ู ุงูุดูุฑ ุงูุซุงูุซ.",
|
| 661 |
+
"source_kind":"clean certified source", "direct_probability":0.88,
|
| 662 |
+
"score":0.88, "dense_score":0.93, "cross_encoder_score":0.80,
|
| 663 |
+
"bm25_score":0.88, "retriever_agreement":5,
|
| 664 |
+
},
|
| 665 |
+
{
|
| 666 |
+
"record_id":"preflight-facet-time-broad", "book_id":"facet-time-broad", "book":"ูุชุงุจ ุงูุฒู
ู ุงูู
ุฌู
ู",
|
| 667 |
+
"title":"ุงูุญุฏูุฏ ุงูุฒู
ููุฉ ููุนู
ููุฉ", "question":facet_query, "ruling":"ุชุญุฏูุฏ ุฒู
ูู",
|
| 668 |
+
"answer":"ุงูุญุฏูุฏ ุงูุฒู
ููุฉ ูู: - ุงูุดูุฑ ุงูุฃูู. - ุงูุดูุฑ ุงูุซุงูู. - ุงูุดูุฑ ุงูุซุงูุซ.",
|
| 669 |
+
"source_kind":"clean certified source", "direct_probability":0.84,
|
| 670 |
+
"score":0.84, "dense_score":0.92, "cross_encoder_score":0.78,
|
| 671 |
+
"bm25_score":0.84, "retriever_agreement":5,
|
| 672 |
+
},
|
| 673 |
+
{
|
| 674 |
+
"record_id":"preflight-facet-space-noise", "book_id":"facet-space", "book":"ูุชุงุจ ุงูู
ูุงู",
|
| 675 |
+
"title":"ุงูุญุฏูุฏ ุงูู
ูุงููุฉ ููุนู
ููุฉ", "question":"ู
ุง ูู ุงูุญุฏูุฏ ุงูู
ูุงููุฉ ููุนู
ููุฉุ", "ruling":"ุชุญุฏูุฏ ู
ูุงูู",
|
| 676 |
+
"answer":"ุงูุญุฏูุฏ ุงูู
ูุงููุฉ ูู: - ุงูู
ููุน ุฃูู. - ุงูู
ููุน ุจุงุก. - ุงูู
ููุน ุฌูู
. - ุงูู
ููุน ุฏุงู.",
|
| 677 |
+
"source_kind":"clean certified source", "direct_probability":0.99,
|
| 678 |
+
"score":0.99, "dense_score":0.98, "cross_encoder_score":0.94,
|
| 679 |
+
"bm25_score":0.97, "retriever_agreement":5,
|
| 680 |
+
},
|
| 681 |
+
]
|
| 682 |
+
facet_result = pipeline.resolve(facet_query, facet_sources, "ar")
|
| 683 |
+
facet_noise = next(
|
| 684 |
+
(item for item in facet_result.ranked if item.evidence.record_id == "preflight-facet-space-noise"),
|
| 685 |
+
None,
|
| 686 |
+
)
|
| 687 |
+
facet_ok = (
|
| 688 |
+
facet_result.query.facet_dimension == "time"
|
| 689 |
+
and facet_result.query.facet_locked
|
| 690 |
+
and "ุฒู
ู" not in facet_result.query.subject_terms
|
| 691 |
+
and facet_noise is not None
|
| 692 |
+
and not facet_noise.accepted
|
| 693 |
+
and "facet_dimension_mismatch" in facet_noise.hard_rejections
|
| 694 |
+
and "ุงูู
ููุน ุฃูู" not in facet_result.answer
|
| 695 |
+
and "ุฃูู ุนุดุฑุฉ ุฃูุงู
ู
ู ุงูุดูุฑ ุงูุซุงูุซ" in facet_result.answer
|
| 696 |
+
and facet_result.answer.count("ุงูุดูุฑ ุงูุซุงูุซ") == 1
|
| 697 |
+
and int((facet_result.details.get("set_coverage", {}) or {}).get("unique_answer_items", 0)) == 3
|
| 698 |
+
and bool((facet_result.details.get("global_arbitration", {}) or {}).get("passed"))
|
| 699 |
+
)
|
| 700 |
+
if not facet_ok:
|
| 701 |
+
raise RuntimeError(
|
| 702 |
+
"HUDA-Net v39.0.1 semantic-facet contract preflight failed before runtime download: "
|
| 703 |
+
+ json.dumps({"answer": facet_result.answer, "details": facet_result.details}, ensure_ascii=False)
|
| 704 |
+
)
|
| 705 |
+
|
| 706 |
# Validate the compare-sources contract with coherent generic evidence before
|
| 707 |
# downloading the multi-gigabyte runtime. This catches audit-fixture regressions
|
| 708 |
# immediately and verifies real synthesis rather than bypassing the gate.
|
|
|
|
| 737 |
primary_ok = ("Book 1" in primary_only.answer) ^ ("Book 2" in primary_only.answer)
|
| 738 |
if not compare_ok or not primary_ok or compared.answer == primary_only.answer:
|
| 739 |
raise RuntimeError(
|
| 740 |
+
"HUDA-Net v39.0.1 compare-mode preflight failed before runtime download: "
|
| 741 |
+ json.dumps(
|
| 742 |
{
|
| 743 |
"compare_ok": compare_ok,
|
|
|
|
| 749 |
)
|
| 750 |
)
|
| 751 |
print(
|
| 752 |
+
"โ
HUDA-Net v39.0.1 central-query-contract, semantic-facet-contract, global-arbiter, proposition, coverage-aware-set, list-canonicalization, contextual-sense, text-integrity, central-ruling-frame, relation-graph, issue-clustering, semantic-alignment, and compare-mode preflight "
|
| 753 |
"passed before runtime download"
|
| 754 |
)
|
| 755 |
|
|
|
|
| 766 |
|
| 767 |
_download_hudanet_private_datasets()
|
| 768 |
|
| 769 |
+
VERSION = "39.0.1"
|
| 770 |
CONFIG = {
|
| 771 |
"INPUT_ROOT": "/kaggle/input",
|
| 772 |
"WORK_ROOT": "/tmp/hudanet_v27",
|
|
|
|
| 3556 |
from scipy import sparse
|
| 3557 |
import joblib
|
| 3558 |
|
| 3559 |
+
UI_VERSION = "39.0.1"
|
| 3560 |
UI_CONFIG = {
|
| 3561 |
"INPUT_ROOT": "/kaggle/input",
|
| 3562 |
"RUNTIME_DATASET_SLUG": "hudanet-bilingual-certified-runtime",
|
|
|
|
| 4210 |
_extra_labels = sorted(set(_labels) - set(_RULING_FILTER_DISPLAY_IDS))
|
| 4211 |
if _missing_labels or _extra_labels:
|
| 4212 |
raise RuntimeError(
|
| 4213 |
+
f"HUDA-Net v39.0.1 ruling-filter label mismatch for {_lang}: "
|
| 4214 |
+ json.dumps({"missing": _missing_labels, "extra": _extra_labels}, ensure_ascii=False)
|
| 4215 |
)
|
| 4216 |
RULING_FILTER_KEYS = _RULING_FILTER_DISPLAY_IDS
|
|
|
|
| 6105 |
|
| 6106 |
failed=[item for item in checks if not item["passed"]]
|
| 6107 |
if failed:
|
| 6108 |
+
raise RuntimeError("HUDA-Net v39.0.1 generic proposition self-test failed: "+json.dumps(failed,ensure_ascii=False))
|
| 6109 |
+
print(f"โ
HUDA-Net v39.0.1 generic proposition self-test passed: {len(checks)} checks")
|
| 6110 |
return {"passed":True,"tested":len(checks),"checks":checks}
|
| 6111 |
|
| 6112 |
def validate_specificity_guard_v33(engine: ProfessionalEvidenceEngine) -> dict:
|
|
@@ -1,8 +1,8 @@
|
|
| 1 |
-
# HUDA-Net Generic Evidence Core v39.0.
|
| 2 |
|
| 3 |
## Central Query Contract architecture
|
| 4 |
|
| 5 |
-
v39 replaces competing local answer plans with one immutable query contract created before evidence-conditioned ranking.
|
| 6 |
|
| 7 |
The contract separates:
|
| 8 |
|
|
@@ -42,3 +42,9 @@ The v39 core retains the earlier generic layers for:
|
|
| 42 |
- multi-source comparison.
|
| 43 |
|
| 44 |
No topic-specific question, answer, entity alias table, or Hajj/Umrah scenario dictionary is stored in the production core.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# HUDA-Net Generic Evidence Core v39.0.1
|
| 2 |
|
| 3 |
## Central Query Contract architecture
|
| 4 |
|
| 5 |
+
v39.0.1 replaces competing local answer plans with one immutable query contract created before evidence-conditioned ranking.
|
| 6 |
|
| 7 |
The contract separates:
|
| 8 |
|
|
|
|
| 42 |
- multi-source comparison.
|
| 43 |
|
| 44 |
No topic-specific question, answer, entity alias table, or Hajj/Umrah scenario dictionary is stored in the production core.
|
| 45 |
+
|
| 46 |
+
## Semantic facet contract
|
| 47 |
+
|
| 48 |
+
v39.0.1 locks an explicit subject facet such as temporal/spatial, financial/health, positive/negative, direct/indirect, initial/final, or internal/external. Facet words are removed from mandatory subject coverage and checked separately against every source and list member.
|
| 49 |
+
|
| 50 |
+
The facet guard rejects a stronger source that answers the opposite dimension, computes list-cluster purity, and consolidates overlapping temporal spans so that a broad period and a more specific contained period do not appear as contradictory peer items.
|
|
@@ -1,5 +1,5 @@
|
|
| 1 |
"""HUDA-Net generic evidence resolution library."""
|
| 2 |
from .pipeline import GenericEvidencePipeline, ResolutionResult
|
| 3 |
|
| 4 |
-
__version__ = "39.0.
|
| 5 |
__all__ = ["GenericEvidencePipeline", "ResolutionResult", "__version__"]
|
|
|
|
| 1 |
"""HUDA-Net generic evidence resolution library."""
|
| 2 |
from .pipeline import GenericEvidencePipeline, ResolutionResult
|
| 3 |
|
| 4 |
+
__version__ = "39.0.1"
|
| 5 |
__all__ = ["GenericEvidencePipeline", "ResolutionResult", "__version__"]
|
|
@@ -7,6 +7,7 @@ from .contextual_sense import ContextualSenseAligner
|
|
| 7 |
from .centrality import ScopeCentralityEvaluator
|
| 8 |
from .text import TextProcessor
|
| 9 |
from .types import EvidenceFrame, QueryFrame, ScoredEvidence
|
|
|
|
| 10 |
|
| 11 |
|
| 12 |
class CompatibilityScorer:
|
|
@@ -22,6 +23,7 @@ class CompatibilityScorer:
|
|
| 22 |
self.compatibility = ranking.get("type_compatibility", {})
|
| 23 |
self.sense = ContextualSenseAligner(text)
|
| 24 |
self.centrality = ScopeCentralityEvaluator(text)
|
|
|
|
| 25 |
|
| 26 |
@staticmethod
|
| 27 |
def _bounded_score(value: object) -> float:
|
|
@@ -159,6 +161,11 @@ class CompatibilityScorer:
|
|
| 159 |
centrality = self.centrality.evaluate(
|
| 160 |
query, evidence, source, sense, type_match=type_match, directness=directness
|
| 161 |
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 162 |
|
| 163 |
source_quality = 0.68
|
| 164 |
source_kind = str(source.get("source_kind", "")).casefold()
|
|
@@ -177,6 +184,7 @@ class CompatibilityScorer:
|
|
| 177 |
+ 0.10 * sense.score
|
| 178 |
+ 0.07 * sense.joint_scope_coverage
|
| 179 |
+ 0.07 * evidence.integrity_score
|
|
|
|
| 180 |
)
|
| 181 |
contract_type_match = type_match if not query.contract_locked else (
|
| 182 |
1.0 if query.primary_request_type in evidence.answer_types else type_match
|
|
@@ -206,6 +214,10 @@ class CompatibilityScorer:
|
|
| 206 |
"linked_question_alignment": centrality.question_alignment,
|
| 207 |
"title_alignment": centrality.title_alignment,
|
| 208 |
"extra_scope_penalty": centrality.extra_scope_penalty,
|
|
|
|
|
|
|
|
|
|
|
|
|
| 209 |
}
|
| 210 |
score = sum(self.weights.get(name, 0.0) * value for name, value in metrics.items())
|
| 211 |
|
|
@@ -259,6 +271,13 @@ class CompatibilityScorer:
|
|
| 259 |
hard_rejections: List[str] = []
|
| 260 |
reasons: List[str] = []
|
| 261 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 262 |
if (
|
| 263 |
query.contract_locked
|
| 264 |
and query.primary_request_type not in evidence.answer_types
|
|
@@ -318,6 +337,8 @@ class CompatibilityScorer:
|
|
| 318 |
if evidence.answer_quality < self.thresholds.get("minimum_answer_quality", 0.48):
|
| 319 |
hard_rejections.append("low_quality_or_broken_answer")
|
| 320 |
|
|
|
|
|
|
|
| 321 |
if contract_type_match >= 0.80:
|
| 322 |
reasons.append("ููุน ุงูุฌูุงุจ ูุทุงุจู ุนูุฏ ุงูุณุคุงู ุงูู
ุฑูุฒู" if query.language == "ar" else "Answer type matches the central query contract")
|
| 323 |
if anchor_coverage >= 0.72:
|
|
@@ -365,6 +386,8 @@ class CompatibilityScorer:
|
|
| 365 |
"generic_metrics": {key: round(float(value), 4) for key, value in metrics.items()},
|
| 366 |
"generic_answer_types": list(evidence.answer_types),
|
| 367 |
"generic_outcomes": list(evidence.outcomes),
|
|
|
|
|
|
|
| 368 |
})
|
| 369 |
return ScoredEvidence(
|
| 370 |
source=source_copy,
|
|
|
|
| 7 |
from .centrality import ScopeCentralityEvaluator
|
| 8 |
from .text import TextProcessor
|
| 9 |
from .types import EvidenceFrame, QueryFrame, ScoredEvidence
|
| 10 |
+
from .facet_contract import FacetContractAnalyzer
|
| 11 |
|
| 12 |
|
| 13 |
class CompatibilityScorer:
|
|
|
|
| 23 |
self.compatibility = ranking.get("type_compatibility", {})
|
| 24 |
self.sense = ContextualSenseAligner(text)
|
| 25 |
self.centrality = ScopeCentralityEvaluator(text)
|
| 26 |
+
self.facets = FacetContractAnalyzer(text)
|
| 27 |
|
| 28 |
@staticmethod
|
| 29 |
def _bounded_score(value: object) -> float:
|
|
|
|
| 161 |
centrality = self.centrality.evaluate(
|
| 162 |
query, evidence, source, sense, type_match=type_match, directness=directness
|
| 163 |
)
|
| 164 |
+
facet = self.facets.align(
|
| 165 |
+
query.facet_dimension,
|
| 166 |
+
f"{evidence.metadata_text} {evidence.answer_text}",
|
| 167 |
+
query.language,
|
| 168 |
+
)
|
| 169 |
|
| 170 |
source_quality = 0.68
|
| 171 |
source_kind = str(source.get("source_kind", "")).casefold()
|
|
|
|
| 184 |
+ 0.10 * sense.score
|
| 185 |
+ 0.07 * sense.joint_scope_coverage
|
| 186 |
+ 0.07 * evidence.integrity_score
|
| 187 |
+
+ 0.10 * facet.score
|
| 188 |
)
|
| 189 |
contract_type_match = type_match if not query.contract_locked else (
|
| 190 |
1.0 if query.primary_request_type in evidence.answer_types else type_match
|
|
|
|
| 214 |
"linked_question_alignment": centrality.question_alignment,
|
| 215 |
"title_alignment": centrality.title_alignment,
|
| 216 |
"extra_scope_penalty": centrality.extra_scope_penalty,
|
| 217 |
+
"facet_alignment": facet.score,
|
| 218 |
+
"facet_requested_support": facet.requested_support,
|
| 219 |
+
"facet_conflicting_support": facet.conflicting_support,
|
| 220 |
+
"facet_explicit_conflict": 1.0 if facet.explicit_conflict else 0.0,
|
| 221 |
}
|
| 222 |
score = sum(self.weights.get(name, 0.0) * value for name, value in metrics.items())
|
| 223 |
|
|
|
|
| 271 |
hard_rejections: List[str] = []
|
| 272 |
reasons: List[str] = []
|
| 273 |
|
| 274 |
+
if (
|
| 275 |
+
query.facet_locked
|
| 276 |
+
and facet.conflicting_support >= self.thresholds.get("hard_facet_conflict", 0.62)
|
| 277 |
+
and facet.requested_support < self.thresholds.get("minimum_facet_support", 0.52)
|
| 278 |
+
):
|
| 279 |
+
hard_rejections.append("facet_dimension_mismatch")
|
| 280 |
+
|
| 281 |
if (
|
| 282 |
query.contract_locked
|
| 283 |
and query.primary_request_type not in evidence.answer_types
|
|
|
|
| 337 |
if evidence.answer_quality < self.thresholds.get("minimum_answer_quality", 0.48):
|
| 338 |
hard_rejections.append("low_quality_or_broken_answer")
|
| 339 |
|
| 340 |
+
if query.facet_locked and facet.score >= 0.72:
|
| 341 |
+
reasons.append("ุงูุจุนุฏ ุงูู
ููุฏ ูู ุงูุณุคุงู ู
ุทุงุจู ููุดุงูุฏ" if query.language == "ar" else "The requested facet matches the evidence")
|
| 342 |
if contract_type_match >= 0.80:
|
| 343 |
reasons.append("ููุน ุงูุฌูุงุจ ูุทุงุจู ุนูุฏ ุงูุณุคุงู ุงูู
ุฑูุฒู" if query.language == "ar" else "Answer type matches the central query contract")
|
| 344 |
if anchor_coverage >= 0.72:
|
|
|
|
| 386 |
"generic_metrics": {key: round(float(value), 4) for key, value in metrics.items()},
|
| 387 |
"generic_answer_types": list(evidence.answer_types),
|
| 388 |
"generic_outcomes": list(evidence.outcomes),
|
| 389 |
+
"generic_facet_dimension": query.facet_dimension,
|
| 390 |
+
"generic_facet_detected": list(facet.detected_dimensions),
|
| 391 |
})
|
| 392 |
return ScoredEvidence(
|
| 393 |
source=source_copy,
|
|
@@ -0,0 +1,220 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
import re
|
| 4 |
+
from dataclasses import dataclass
|
| 5 |
+
from typing import Dict, Mapping, Sequence, Tuple
|
| 6 |
+
|
| 7 |
+
from .text import TextProcessor
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
@dataclass(frozen=True)
|
| 11 |
+
class FacetContract:
|
| 12 |
+
"""An explicit semantic dimension attached to the requested subject.
|
| 13 |
+
|
| 14 |
+
Facets are domain-neutral qualifiers such as temporal/spatial, financial/health,
|
| 15 |
+
positive/negative, direct/indirect, or initial/final. They constrain which
|
| 16 |
+
evidence and list members may answer the question, but they are not themselves
|
| 17 |
+
part of the subject that must be repeated verbatim in every source.
|
| 18 |
+
"""
|
| 19 |
+
|
| 20 |
+
dimension: str = ""
|
| 21 |
+
surface_terms: Tuple[str, ...] = ()
|
| 22 |
+
locked: bool = False
|
| 23 |
+
confidence: float = 0.0
|
| 24 |
+
matched_pattern: str = ""
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
@dataclass(frozen=True)
|
| 28 |
+
class FacetAlignment:
|
| 29 |
+
score: float
|
| 30 |
+
requested_support: float
|
| 31 |
+
conflicting_support: float
|
| 32 |
+
detected_dimensions: Tuple[str, ...]
|
| 33 |
+
explicit_conflict: bool
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
class FacetContractAnalyzer:
|
| 37 |
+
"""Extract and validate domain-neutral semantic facets.
|
| 38 |
+
|
| 39 |
+
The patterns describe general dimensions rather than topic entities. No place,
|
| 40 |
+
month, person, ritual, product, or scenario names are stored here.
|
| 41 |
+
"""
|
| 42 |
+
|
| 43 |
+
_PATTERNS: Mapping[str, Mapping[str, Tuple[str, ...]]] = {
|
| 44 |
+
"time": {
|
| 45 |
+
"ar": (
|
| 46 |
+
r"\b(?:ุงู)?(?:ุฒู
ุงูู(?:ุฉ|ู)?|ุฒู
ูู(?:ุฉ|ู)?|ููุชู(?:ุฉ|ู)?|ุชูููุช(?:ูุฉ)?|ุฃููุงุช?|ุงููุงุช|ุฃุฒู
ูุฉ|ุงุฒู
ูุฉ)\b",
|
| 47 |
+
),
|
| 48 |
+
"en": (r"\b(?:temporal|time-based|time\s+related|chronological)\b",),
|
| 49 |
+
},
|
| 50 |
+
"location": {
|
| 51 |
+
"ar": (
|
| 52 |
+
r"\b(?:ุงู)?(?:ู
ูุงูู(?:ุฉ|ู)?|ู
ูุถุนู(?:ุฉ|ู)?|ุฌุบุฑุงูู(?:ุฉ|ู)?|ุฃู
ุงูู|ุงู
ุงูู|ู
ูุงูุน?)\b",
|
| 53 |
+
),
|
| 54 |
+
"en": (r"\b(?:spatial|location-based|geographical|geographic|regional)\b",),
|
| 55 |
+
},
|
| 56 |
+
"financial": {
|
| 57 |
+
"ar": (r"\b(?:ุงู)?(?:ู
ุงูู(?:ุฉ|ู)?|ููุฏู(?:ุฉ|ู)?|ุชู
ูููู(?:ุฉ|ู)?|ุงูุชุตุงุฏู(?:ุฉ|ู)?)\b",),
|
| 58 |
+
"en": (r"\b(?:financial|monetary|funding|economic)\b",),
|
| 59 |
+
},
|
| 60 |
+
"health": {
|
| 61 |
+
"ar": (r"\b(?:ุงู)?(?:ุตุญู(?:ุฉ|ู)?|ุทุจู(?:ุฉ|ู)?|ู
ุฑุถู(?:ุฉ|ู)?)\b",),
|
| 62 |
+
"en": (r"\b(?:health|medical|clinical)\b",),
|
| 63 |
+
},
|
| 64 |
+
"positive": {
|
| 65 |
+
"ar": (r"\b(?:ุงู)?(?:ุฅูุฌุงุจู(?:ุฉ|ู)?|ุงูุฌุงุจู(?:ุฉ|ู)?|ู
ูุฌุจ(?:ุฉ|ู)?)\b",),
|
| 66 |
+
"en": (r"\b(?:positive|beneficial|favorable)\b",),
|
| 67 |
+
},
|
| 68 |
+
"negative": {
|
| 69 |
+
"ar": (r"\b(?:ุงู)?(?:ุณูุจู(?:ุฉ|ู)?|ุณุงูุจ(?:ุฉ|ู)?|ุถุงุฑ(?:ุฉ|ู)?)\b",),
|
| 70 |
+
"en": (r"\b(?:negative|harmful|adverse)\b",),
|
| 71 |
+
},
|
| 72 |
+
"direct": {
|
| 73 |
+
"ar": (r"\b(?:ุงู)?(?:ู
ุจุงุดุฑ(?:ุฉ|ู)?)\b",),
|
| 74 |
+
"en": (r"\b(?:direct|immediate)\b",),
|
| 75 |
+
},
|
| 76 |
+
"indirect": {
|
| 77 |
+
"ar": (r"\b(?:ุงู)?(?:ุบูุฑ\s+ู
ุจุงุดุฑ(?:ุฉ|ู)?|ุบูุฑู
ุจุงุดุฑ(?:ุฉ|ู)?)\b",),
|
| 78 |
+
"en": (r"\b(?:indirect|mediated)\b",),
|
| 79 |
+
},
|
| 80 |
+
"initial": {
|
| 81 |
+
"ar": (r"\b(?:ุงู)?(?:ุฃููู(?:ุฉ|ู)?|ุงููู(?:ุฉ|ู)?|ุงุจุชุฏุงุฆู(?:ุฉ|ู)?|ุงูุฃููู|ุงูุงููู)\b",),
|
| 82 |
+
"en": (r"\b(?:initial|early|first-stage)\b",),
|
| 83 |
+
},
|
| 84 |
+
"final": {
|
| 85 |
+
"ar": (r"\b(?:ุงู)?(?:ููุงุฆู(?:ุฉ|ู)?|ุฎุชุงู
ู(?:ุฉ|ู)?|ุงูุฃุฎูุฑุฉ|ุงูุงุฎูุฑุฉ)\b",),
|
| 86 |
+
"en": (r"\b(?:final|late|last-stage|terminal)\b",),
|
| 87 |
+
},
|
| 88 |
+
"internal": {
|
| 89 |
+
"ar": (r"\b(?:ุงู)?(?:ุฏุงุฎูู(?:ุฉ|ู)?)\b",),
|
| 90 |
+
"en": (r"\b(?:internal|inside)\b",),
|
| 91 |
+
},
|
| 92 |
+
"external": {
|
| 93 |
+
"ar": (r"\b(?:ุงู)?(?:ุฎุงุฑุฌู(?:ุฉ|ู)?)\b",),
|
| 94 |
+
"en": (r"\b(?:external|outside)\b",),
|
| 95 |
+
},
|
| 96 |
+
}
|
| 97 |
+
|
| 98 |
+
# General evidence cues used only when the exact adjective is omitted.
|
| 99 |
+
_CUES: Mapping[str, Mapping[str, Tuple[str, ...]]] = {
|
| 100 |
+
"time": {
|
| 101 |
+
"ar": (
|
| 102 |
+
r"\b(?:ููุช|ุงููุงุช|ุฒู
ู|ู
ุฏุฉ|ุชุงุฑูุฎ|ู
ูุนุฏ|ุณุงุนุฉ|ุณุงุนุงุช|ููู
|ุงูุงู
|ูููุฉ|ููุงู|ุงุณุจูุน|ุงุณุงุจูุน|ุดูุฑ|ุงุดูุฑ|ุณูุฉ|ุณููุงุช|ุจุฏุงูุฉ|ููุงูุฉ|ูุจู|ุจุนุฏ)\b",
|
| 103 |
+
),
|
| 104 |
+
"en": (r"\b(?:time|date|duration|hour|day|night|week|month|year|before|after|start|end)\b",),
|
| 105 |
+
},
|
| 106 |
+
"location": {
|
| 107 |
+
"ar": (
|
| 108 |
+
r"\b(?:ู
ูุงู|ู
ููุน|ู
ูุถุน|ุฌูุฉ|ู
ูุทูุฉ|ุจูุฏ|ู
ุฏููุฉ|ูุฑูุฉ|ุทุฑูู|ู
ุณุงุฑ|ุญุฏูุฏ|ุงูู|ุฃูู)\b",
|
| 109 |
+
),
|
| 110 |
+
"en": (r"\b(?:place|location|site|region|country|city|village|route|road|boundary|residents)\b",),
|
| 111 |
+
},
|
| 112 |
+
"financial": {
|
| 113 |
+
"ar": (r"\b(?:ู
ุงู|ููุฏ|ุชู
ููู|ุชูููุฉ|ู
ุจูุบ|ุฏุฎู|ู
ุตุฑูู|ู
ูุฒุงููุฉ)\b",),
|
| 114 |
+
"en": (r"\b(?:money|funding|cost|amount|income|expense|budget)\b",),
|
| 115 |
+
},
|
| 116 |
+
"health": {
|
| 117 |
+
"ar": (r"\b(?:ุตุญุฉ|ู
ุฑุถ|ุนูุงุฌ|ุฏูุงุก|ุชุดุฎูุต|ุทุจูุจ|ู
ุฑูุถ)\b",),
|
| 118 |
+
"en": (r"\b(?:health|illness|treatment|medicine|diagnosis|doctor|patient)\b",),
|
| 119 |
+
},
|
| 120 |
+
"positive": {
|
| 121 |
+
"ar": (r"\b(?:ููุน|ูุงุฆุฏุฉ|ุชุญุณู|ูุฌุงุญ|ุฒูุงุฏุฉ)\b",),
|
| 122 |
+
"en": (r"\b(?:benefit|improvement|success|increase)\b",),
|
| 123 |
+
},
|
| 124 |
+
"negative": {
|
| 125 |
+
"ar": (r"\b(?:ุถุฑุฑ|ุฎุทุฑ|ูุดู|ููุต|ุชุฏููุฑ)\b",),
|
| 126 |
+
"en": (r"\b(?:harm|risk|failure|decrease|deterioration)\b",),
|
| 127 |
+
},
|
| 128 |
+
"direct": {"ar": (r"\b(?:ููุฑุง|ููุฑูุง|ู
ุจุงุดุฑุฉ)\b",), "en": (r"\b(?:immediately|directly)\b",)},
|
| 129 |
+
"indirect": {"ar": (r"\b(?:ูุงุณุทุฉ|ูุณูุท|ูุงุญูุง|ูุงุญููุง)\b",), "en": (r"\b(?:through|mediated|later)\b",)},
|
| 130 |
+
"initial": {"ar": (r"\b(?:ุจุฏุงูุฉ|ุงูู|ุฃูู|ุงุจุชุฏุงุก)\b",), "en": (r"\b(?:beginning|first|initially)\b",)},
|
| 131 |
+
"final": {"ar": (r"\b(?:ููุงูุฉ|ุงุฎุฑ|ุขุฎุฑ|ุฎุชุงู
)\b",), "en": (r"\b(?:end|last|finally)\b",)},
|
| 132 |
+
"internal": {"ar": (r"\b(?:ุฏุงุฎู|ุถู
ู)\b",), "en": (r"\b(?:inside|within)\b",)},
|
| 133 |
+
"external": {"ar": (r"\b(?:ุฎุงุฑุฌ|ุฎุงุฑุฌูุง|ุฎุงุฑุฌููุง)\b",), "en": (r"\b(?:outside|externally)\b",)},
|
| 134 |
+
}
|
| 135 |
+
|
| 136 |
+
_CONFLICTS: Mapping[str, Tuple[str, ...]] = {
|
| 137 |
+
"time": ("location",),
|
| 138 |
+
"location": ("time",),
|
| 139 |
+
"financial": ("health",),
|
| 140 |
+
"health": ("financial",),
|
| 141 |
+
"positive": ("negative",),
|
| 142 |
+
"negative": ("positive",),
|
| 143 |
+
"direct": ("indirect",),
|
| 144 |
+
"indirect": ("direct",),
|
| 145 |
+
"initial": ("final",),
|
| 146 |
+
"final": ("initial",),
|
| 147 |
+
"internal": ("external",),
|
| 148 |
+
"external": ("internal",),
|
| 149 |
+
}
|
| 150 |
+
|
| 151 |
+
def __init__(self, text: TextProcessor):
|
| 152 |
+
self.text = text
|
| 153 |
+
|
| 154 |
+
def extract(self, normalized: str, lang: str) -> Tuple[FacetContract, Tuple[Tuple[int, int, str], ...]]:
|
| 155 |
+
hits = []
|
| 156 |
+
for dimension, by_lang in self._PATTERNS.items():
|
| 157 |
+
for pattern in by_lang.get(lang, ()): # pragma: no branch
|
| 158 |
+
for match in re.finditer(pattern, normalized, re.I):
|
| 159 |
+
hits.append((match.start(), match.end(), match.group(0), dimension, pattern))
|
| 160 |
+
if not hits:
|
| 161 |
+
return FacetContract(), ()
|
| 162 |
+
hits.sort(key=lambda value: (value[0], -(value[1] - value[0])))
|
| 163 |
+
selected = hits[0]
|
| 164 |
+
terms = self.text.content_terms(selected[2], lang)
|
| 165 |
+
return (
|
| 166 |
+
FacetContract(
|
| 167 |
+
dimension=selected[3],
|
| 168 |
+
surface_terms=tuple(terms),
|
| 169 |
+
locked=True,
|
| 170 |
+
confidence=0.97,
|
| 171 |
+
matched_pattern=selected[4],
|
| 172 |
+
),
|
| 173 |
+
((selected[0], selected[1], selected[2]),),
|
| 174 |
+
)
|
| 175 |
+
|
| 176 |
+
def _support(self, dimension: str, text: str, lang: str) -> Tuple[float, bool]:
|
| 177 |
+
explicit = any(re.search(pattern, text, re.I) for pattern in self._PATTERNS.get(dimension, {}).get(lang, ()))
|
| 178 |
+
cue_hits = sum(1 for pattern in self._CUES.get(dimension, {}).get(lang, ()) if re.search(pattern, text, re.I))
|
| 179 |
+
if explicit:
|
| 180 |
+
return 1.0, True
|
| 181 |
+
if cue_hits:
|
| 182 |
+
return min(0.86, 0.58 + 0.12 * cue_hits), False
|
| 183 |
+
return 0.0, False
|
| 184 |
+
|
| 185 |
+
def align(self, dimension: str, value: str, lang: str) -> FacetAlignment:
|
| 186 |
+
if not dimension:
|
| 187 |
+
return FacetAlignment(1.0, 1.0, 0.0, (), False)
|
| 188 |
+
normalized = self.text.normalize(value, lang)
|
| 189 |
+
requested, requested_explicit = self._support(dimension, normalized, lang)
|
| 190 |
+
detected = []
|
| 191 |
+
conflict = 0.0
|
| 192 |
+
explicit_conflict = False
|
| 193 |
+
for other in self._CONFLICTS.get(dimension, ()):
|
| 194 |
+
support, explicit = self._support(other, normalized, lang)
|
| 195 |
+
if support > 0:
|
| 196 |
+
detected.append(other)
|
| 197 |
+
if support > conflict:
|
| 198 |
+
conflict = support
|
| 199 |
+
explicit_conflict = explicit
|
| 200 |
+
if requested > 0:
|
| 201 |
+
detected.insert(0, dimension)
|
| 202 |
+
if requested_explicit:
|
| 203 |
+
score = 1.0
|
| 204 |
+
elif requested > 0:
|
| 205 |
+
score = requested
|
| 206 |
+
elif conflict >= 0.80:
|
| 207 |
+
score = 0.0
|
| 208 |
+
elif conflict > 0:
|
| 209 |
+
score = 0.18
|
| 210 |
+
else:
|
| 211 |
+
score = 0.50
|
| 212 |
+
if conflict > requested and conflict >= 0.58:
|
| 213 |
+
score = min(score, 0.12)
|
| 214 |
+
return FacetAlignment(
|
| 215 |
+
score=max(0.0, min(1.0, score)),
|
| 216 |
+
requested_support=requested,
|
| 217 |
+
conflicting_support=conflict,
|
| 218 |
+
detected_dimensions=tuple(dict.fromkeys(detected)),
|
| 219 |
+
explicit_conflict=bool(explicit_conflict and conflict > requested),
|
| 220 |
+
)
|
|
@@ -38,7 +38,8 @@ class GlobalAnswerArbiter:
|
|
| 38 |
integrity = float(item.metrics.get("evidence_integrity", 0.0))
|
| 39 |
neural = float(item.metrics.get("neural_prior", 0.0))
|
| 40 |
linked = float(item.metrics.get("linked_question_alignment", 0.0))
|
| 41 |
-
|
|
|
|
| 42 |
if query.contract_locked and query.primary_request_type not in item.evidence.answer_types:
|
| 43 |
score *= 0.30
|
| 44 |
return max(0.0, min(1.0, score))
|
|
@@ -65,6 +66,14 @@ class GlobalAnswerArbiter:
|
|
| 65 |
|
| 66 |
if synthesis.used_record_ids and not direct_ids:
|
| 67 |
violations.append("no_used_source_directly_satisfies_query_contract")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 68 |
if query.contract_locked:
|
| 69 |
incompatible = [
|
| 70 |
item for item in used_items
|
|
|
|
| 38 |
integrity = float(item.metrics.get("evidence_integrity", 0.0))
|
| 39 |
neural = float(item.metrics.get("neural_prior", 0.0))
|
| 40 |
linked = float(item.metrics.get("linked_question_alignment", 0.0))
|
| 41 |
+
facet = float(item.metrics.get("facet_alignment", 1.0 if not query.facet_locked else 0.5))
|
| 42 |
+
score = 0.25 * type_match + 0.20 * directness + 0.14 * centrality + 0.11 * integrity + 0.10 * neural + 0.08 * linked + 0.12 * facet
|
| 43 |
if query.contract_locked and query.primary_request_type not in item.evidence.answer_types:
|
| 44 |
score *= 0.30
|
| 45 |
return max(0.0, min(1.0, score))
|
|
|
|
| 66 |
|
| 67 |
if synthesis.used_record_ids and not direct_ids:
|
| 68 |
violations.append("no_used_source_directly_satisfies_query_contract")
|
| 69 |
+
if query.facet_locked:
|
| 70 |
+
facet_incompatible = [
|
| 71 |
+
item for item in used_items
|
| 72 |
+
if float(item.metrics.get("facet_alignment", 0.5)) < 0.34
|
| 73 |
+
or float(item.metrics.get("facet_conflicting_support", 0.0)) >= 0.62
|
| 74 |
+
]
|
| 75 |
+
if facet_incompatible:
|
| 76 |
+
violations.append("locked_facet_dimension_violated")
|
| 77 |
if query.contract_locked:
|
| 78 |
incompatible = [
|
| 79 |
item for item in used_items
|
|
@@ -171,6 +171,11 @@ class GenericEvidencePipeline:
|
|
| 171 |
"scope_terms": list(query_frame.scope_terms),
|
| 172 |
"provenance_terms": list(query_frame.provenance_terms),
|
| 173 |
"pattern": query_frame.contract_pattern,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 174 |
},
|
| 175 |
"global_arbitration": {
|
| 176 |
"passed": arbitration.passed,
|
|
@@ -192,6 +197,8 @@ class GenericEvidencePipeline:
|
|
| 192 |
"term_weights": dict(query_frame.term_weights),
|
| 193 |
"operator_terms": list(query_frame.operator_terms),
|
| 194 |
"polarity": query_frame.polarity,
|
|
|
|
|
|
|
| 195 |
"accepted": len(accepted),
|
| 196 |
"rejected": len(rejected),
|
| 197 |
"consensus_state": consensus.state,
|
|
@@ -276,6 +283,7 @@ class GenericEvidencePipeline:
|
|
| 276 |
and float(item.metrics.get("joint_scope_coverage", 0.0)) >= float(self.resources.ranking.get("thresholds", {}).get("exact_joint_scope_coverage", 0.72))
|
| 277 |
and float(item.metrics.get("evidence_integrity", 0.0)) >= float(self.resources.ranking.get("thresholds", {}).get("exact_evidence_integrity", 0.58))
|
| 278 |
and float(item.metrics.get("scope_centrality", 0.0)) >= float(self.resources.ranking.get("thresholds", {}).get("exact_scope_centrality", 0.58))
|
|
|
|
| 279 |
)
|
| 280 |
if is_exact:
|
| 281 |
source["tier"] = "exact"
|
|
@@ -304,6 +312,8 @@ class GenericEvidencePipeline:
|
|
| 304 |
result["stats"]["generic_contract_locked"] = bool(resolution.query.contract_locked)
|
| 305 |
result["stats"]["generic_requested_shape"] = resolution.query.requested_shape
|
| 306 |
result["stats"]["generic_global_arbiter_passed"] = bool((resolution.details.get("global_arbitration", {}) or {}).get("passed"))
|
|
|
|
|
|
|
| 307 |
result["stats"]["exact_count"] = len(exact)
|
| 308 |
result["stats"]["related_count"] = len(related)
|
| 309 |
result["stats"]["distant_count"] = len(distant)
|
|
|
|
| 171 |
"scope_terms": list(query_frame.scope_terms),
|
| 172 |
"provenance_terms": list(query_frame.provenance_terms),
|
| 173 |
"pattern": query_frame.contract_pattern,
|
| 174 |
+
"facet_dimension": query_frame.facet_dimension,
|
| 175 |
+
"facet_terms": list(query_frame.facet_terms),
|
| 176 |
+
"facet_locked": query_frame.facet_locked,
|
| 177 |
+
"facet_confidence": query_frame.facet_confidence,
|
| 178 |
+
"facet_pattern": query_frame.facet_pattern,
|
| 179 |
},
|
| 180 |
"global_arbitration": {
|
| 181 |
"passed": arbitration.passed,
|
|
|
|
| 197 |
"term_weights": dict(query_frame.term_weights),
|
| 198 |
"operator_terms": list(query_frame.operator_terms),
|
| 199 |
"polarity": query_frame.polarity,
|
| 200 |
+
"facet_dimension": query_frame.facet_dimension,
|
| 201 |
+
"facet_terms": list(query_frame.facet_terms),
|
| 202 |
"accepted": len(accepted),
|
| 203 |
"rejected": len(rejected),
|
| 204 |
"consensus_state": consensus.state,
|
|
|
|
| 283 |
and float(item.metrics.get("joint_scope_coverage", 0.0)) >= float(self.resources.ranking.get("thresholds", {}).get("exact_joint_scope_coverage", 0.72))
|
| 284 |
and float(item.metrics.get("evidence_integrity", 0.0)) >= float(self.resources.ranking.get("thresholds", {}).get("exact_evidence_integrity", 0.58))
|
| 285 |
and float(item.metrics.get("scope_centrality", 0.0)) >= float(self.resources.ranking.get("thresholds", {}).get("exact_scope_centrality", 0.58))
|
| 286 |
+
and (not resolution.query.facet_locked or float(item.metrics.get("facet_alignment", 0.0)) >= 0.72)
|
| 287 |
)
|
| 288 |
if is_exact:
|
| 289 |
source["tier"] = "exact"
|
|
|
|
| 312 |
result["stats"]["generic_contract_locked"] = bool(resolution.query.contract_locked)
|
| 313 |
result["stats"]["generic_requested_shape"] = resolution.query.requested_shape
|
| 314 |
result["stats"]["generic_global_arbiter_passed"] = bool((resolution.details.get("global_arbitration", {}) or {}).get("passed"))
|
| 315 |
+
result["stats"]["generic_facet_dimension"] = resolution.query.facet_dimension
|
| 316 |
+
result["stats"]["generic_facet_locked"] = bool(resolution.query.facet_locked)
|
| 317 |
result["stats"]["exact_count"] = len(exact)
|
| 318 |
result["stats"]["related_count"] = len(related)
|
| 319 |
result["stats"]["distant_count"] = len(distant)
|
|
@@ -121,7 +121,7 @@ class QueryAnalyzer:
|
|
| 121 |
normalized = self.text.normalize(raw, lang)
|
| 122 |
contract = self.contract_builder.build(raw, lang)
|
| 123 |
relation_edges = self.relations.extract(raw, lang)
|
| 124 |
-
excluded_relation_terms = set(contract.scope_terms) | set(contract.provenance_terms)
|
| 125 |
if excluded_relation_terms:
|
| 126 |
relation_edges = tuple(
|
| 127 |
edge for edge in relation_edges
|
|
@@ -181,7 +181,7 @@ class QueryAnalyzer:
|
|
| 181 |
)
|
| 182 |
|
| 183 |
candidate_terms = [term for term in all_terms if not _is_operator_term(term)]
|
| 184 |
-
excluded_contract_terms = set(contract.scope_terms) | set(contract.provenance_terms) | set(contract.operator_terms)
|
| 185 |
candidate_terms = [term for term in candidate_terms if term not in excluded_contract_terms]
|
| 186 |
if contract.subject_terms:
|
| 187 |
candidate_terms = list(contract.subject_terms)
|
|
@@ -308,4 +308,9 @@ class QueryAnalyzer:
|
|
| 308 |
scope_terms=contract.scope_terms,
|
| 309 |
provenance_terms=contract.provenance_terms,
|
| 310 |
contract_pattern=contract.matched_pattern,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 311 |
)
|
|
|
|
| 121 |
normalized = self.text.normalize(raw, lang)
|
| 122 |
contract = self.contract_builder.build(raw, lang)
|
| 123 |
relation_edges = self.relations.extract(raw, lang)
|
| 124 |
+
excluded_relation_terms = set(contract.scope_terms) | set(contract.provenance_terms) | set(contract.facet_terms)
|
| 125 |
if excluded_relation_terms:
|
| 126 |
relation_edges = tuple(
|
| 127 |
edge for edge in relation_edges
|
|
|
|
| 181 |
)
|
| 182 |
|
| 183 |
candidate_terms = [term for term in all_terms if not _is_operator_term(term)]
|
| 184 |
+
excluded_contract_terms = set(contract.scope_terms) | set(contract.provenance_terms) | set(contract.operator_terms) | set(contract.facet_terms)
|
| 185 |
candidate_terms = [term for term in candidate_terms if term not in excluded_contract_terms]
|
| 186 |
if contract.subject_terms:
|
| 187 |
candidate_terms = list(contract.subject_terms)
|
|
|
|
| 308 |
scope_terms=contract.scope_terms,
|
| 309 |
provenance_terms=contract.provenance_terms,
|
| 310 |
contract_pattern=contract.matched_pattern,
|
| 311 |
+
facet_dimension=contract.facet_dimension,
|
| 312 |
+
facet_terms=contract.facet_terms,
|
| 313 |
+
facet_locked=contract.facet_locked,
|
| 314 |
+
facet_confidence=contract.facet_confidence,
|
| 315 |
+
facet_pattern=contract.facet_pattern,
|
| 316 |
)
|
|
@@ -5,6 +5,7 @@ from dataclasses import dataclass
|
|
| 5 |
from typing import Dict, Mapping, Sequence, Tuple
|
| 6 |
|
| 7 |
from .text import TextProcessor
|
|
|
|
| 8 |
|
| 9 |
|
| 10 |
@dataclass(frozen=True)
|
|
@@ -26,6 +27,11 @@ class QueryContract:
|
|
| 26 |
provenance_terms: Tuple[str, ...]
|
| 27 |
operator_terms: Tuple[str, ...]
|
| 28 |
matched_pattern: str = ""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
|
| 31 |
class QueryContractBuilder:
|
|
@@ -83,6 +89,7 @@ class QueryContractBuilder:
|
|
| 83 |
def __init__(self, text: TextProcessor, semantic: Mapping[str, object]):
|
| 84 |
self.text = text
|
| 85 |
self.semantic = semantic
|
|
|
|
| 86 |
|
| 87 |
def _extract_spans(self, normalized: str, patterns: Sequence[str]) -> Tuple[Tuple[int, int, str], ...]:
|
| 88 |
values = []
|
|
@@ -116,9 +123,10 @@ class QueryContractBuilder:
|
|
| 116 |
|
| 117 |
scope_patterns = self._AR_SCOPE_PATTERNS if lang == "ar" else self._EN_SCOPE_PATTERNS
|
| 118 |
provenance_patterns = self._AR_PROVENANCE_PATTERNS if lang == "ar" else self._EN_PROVENANCE_PATTERNS
|
|
|
|
| 119 |
scope_spans = self._extract_spans(normalized, scope_patterns)
|
| 120 |
provenance_spans = self._extract_spans(normalized, provenance_patterns)
|
| 121 |
-
removable = list(scope_spans) + list(provenance_spans)
|
| 122 |
|
| 123 |
operator = matched[0] if matched else ""
|
| 124 |
shape = matched[1] if matched else ""
|
|
@@ -148,4 +156,9 @@ class QueryContractBuilder:
|
|
| 148 |
provenance_terms=tuple(provenance_terms),
|
| 149 |
operator_terms=tuple(operator_terms),
|
| 150 |
matched_pattern=matched_pattern,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 151 |
)
|
|
|
|
| 5 |
from typing import Dict, Mapping, Sequence, Tuple
|
| 6 |
|
| 7 |
from .text import TextProcessor
|
| 8 |
+
from .facet_contract import FacetContractAnalyzer
|
| 9 |
|
| 10 |
|
| 11 |
@dataclass(frozen=True)
|
|
|
|
| 27 |
provenance_terms: Tuple[str, ...]
|
| 28 |
operator_terms: Tuple[str, ...]
|
| 29 |
matched_pattern: str = ""
|
| 30 |
+
facet_dimension: str = ""
|
| 31 |
+
facet_terms: Tuple[str, ...] = ()
|
| 32 |
+
facet_locked: bool = False
|
| 33 |
+
facet_confidence: float = 0.0
|
| 34 |
+
facet_pattern: str = ""
|
| 35 |
|
| 36 |
|
| 37 |
class QueryContractBuilder:
|
|
|
|
| 89 |
def __init__(self, text: TextProcessor, semantic: Mapping[str, object]):
|
| 90 |
self.text = text
|
| 91 |
self.semantic = semantic
|
| 92 |
+
self.facets = FacetContractAnalyzer(text)
|
| 93 |
|
| 94 |
def _extract_spans(self, normalized: str, patterns: Sequence[str]) -> Tuple[Tuple[int, int, str], ...]:
|
| 95 |
values = []
|
|
|
|
| 123 |
|
| 124 |
scope_patterns = self._AR_SCOPE_PATTERNS if lang == "ar" else self._EN_SCOPE_PATTERNS
|
| 125 |
provenance_patterns = self._AR_PROVENANCE_PATTERNS if lang == "ar" else self._EN_PROVENANCE_PATTERNS
|
| 126 |
+
facet, facet_spans = self.facets.extract(normalized, lang)
|
| 127 |
scope_spans = self._extract_spans(normalized, scope_patterns)
|
| 128 |
provenance_spans = self._extract_spans(normalized, provenance_patterns)
|
| 129 |
+
removable = list(scope_spans) + list(provenance_spans) + list(facet_spans)
|
| 130 |
|
| 131 |
operator = matched[0] if matched else ""
|
| 132 |
shape = matched[1] if matched else ""
|
|
|
|
| 156 |
provenance_terms=tuple(provenance_terms),
|
| 157 |
operator_terms=tuple(operator_terms),
|
| 158 |
matched_pattern=matched_pattern,
|
| 159 |
+
facet_dimension=facet.dimension,
|
| 160 |
+
facet_terms=facet.surface_terms,
|
| 161 |
+
facet_locked=facet.locked,
|
| 162 |
+
facet_confidence=facet.confidence,
|
| 163 |
+
facet_pattern=facet.matched_pattern,
|
| 164 |
)
|
|
@@ -1,26 +1,27 @@
|
|
| 1 |
{
|
| 2 |
-
"version": "2.
|
| 3 |
-
"description": "Domain-neutral
|
| 4 |
"weights": {
|
| 5 |
-
"request_type_match": 0.
|
| 6 |
-
"topic_match": 0.
|
| 7 |
-
"anchor_coverage": 0.
|
| 8 |
-
"query_focus_coverage": 0.
|
| 9 |
-
"semantic_mass_coverage": 0.
|
| 10 |
-
"critical_constraint_coverage": 0.
|
| 11 |
-
"polarity_alignment": 0.
|
| 12 |
-
"answer_directness": 0.
|
| 13 |
-
"evidence_completeness": 0.
|
| 14 |
-
"answer_quality": 0.
|
| 15 |
-
"source_quality": 0.
|
| 16 |
-
"neural_prior": 0.
|
| 17 |
-
"relation_alignment": 0.
|
| 18 |
-
"principle_strength": 0.
|
| 19 |
-
"answerability": 0.
|
| 20 |
-
"sense_alignment": 0.
|
| 21 |
-
"joint_scope_coverage": 0.
|
| 22 |
-
"evidence_integrity": 0.
|
| 23 |
-
"scope_centrality": 0.
|
|
|
|
| 24 |
},
|
| 25 |
"thresholds": {
|
| 26 |
"accept": 0.53,
|
|
@@ -62,7 +63,9 @@
|
|
| 62 |
"exact_joint_scope_coverage": 0.72,
|
| 63 |
"exact_evidence_integrity": 0.58,
|
| 64 |
"exact_scope_centrality": 0.58,
|
| 65 |
-
"minimum_clause_integrity": 0.48
|
|
|
|
|
|
|
| 66 |
},
|
| 67 |
"type_compatibility": {
|
| 68 |
"definition": {
|
|
@@ -383,6 +386,9 @@
|
|
| 383 |
"canonical_sentence_similarity": 0.84,
|
| 384 |
"canonical_label_length_slack": 1.25,
|
| 385 |
"canonical_attachment_floor": 0.84,
|
| 386 |
-
"canonical_description_coverage_floor": 0.6
|
|
|
|
|
|
|
|
|
|
| 387 |
}
|
| 388 |
}
|
|
|
|
| 1 |
{
|
| 2 |
+
"version": "2.6.0",
|
| 3 |
+
"description": "Domain-neutral central query contracts with locked answer operators, explicit semantic facets, typed list-member validation, temporal overlap consolidation, contextual sense, text integrity, ruling frames, issue clustering, and grounded synthesis.",
|
| 4 |
"weights": {
|
| 5 |
+
"request_type_match": 0.0782,
|
| 6 |
+
"topic_match": 0.0368,
|
| 7 |
+
"anchor_coverage": 0.046,
|
| 8 |
+
"query_focus_coverage": 0.0552,
|
| 9 |
+
"semantic_mass_coverage": 0.1012,
|
| 10 |
+
"critical_constraint_coverage": 0.0506,
|
| 11 |
+
"polarity_alignment": 0.0414,
|
| 12 |
+
"answer_directness": 0.046,
|
| 13 |
+
"evidence_completeness": 0.0138,
|
| 14 |
+
"answer_quality": 0.0276,
|
| 15 |
+
"source_quality": 0.0138,
|
| 16 |
+
"neural_prior": 0.0138,
|
| 17 |
+
"relation_alignment": 0.0644,
|
| 18 |
+
"principle_strength": 0.0184,
|
| 19 |
+
"answerability": 0.046,
|
| 20 |
+
"sense_alignment": 0.0782,
|
| 21 |
+
"joint_scope_coverage": 0.046,
|
| 22 |
+
"evidence_integrity": 0.0506,
|
| 23 |
+
"scope_centrality": 0.092,
|
| 24 |
+
"facet_alignment": 0.08
|
| 25 |
},
|
| 26 |
"thresholds": {
|
| 27 |
"accept": 0.53,
|
|
|
|
| 63 |
"exact_joint_scope_coverage": 0.72,
|
| 64 |
"exact_evidence_integrity": 0.58,
|
| 65 |
"exact_scope_centrality": 0.58,
|
| 66 |
+
"minimum_clause_integrity": 0.48,
|
| 67 |
+
"hard_facet_conflict": 0.62,
|
| 68 |
+
"minimum_facet_support": 0.52
|
| 69 |
},
|
| 70 |
"type_compatibility": {
|
| 71 |
"definition": {
|
|
|
|
| 386 |
"canonical_sentence_similarity": 0.84,
|
| 387 |
"canonical_label_length_slack": 1.25,
|
| 388 |
"canonical_attachment_floor": 0.84,
|
| 389 |
+
"canonical_description_coverage_floor": 0.6,
|
| 390 |
+
"hard_item_facet_conflict": 0.62,
|
| 391 |
+
"minimum_item_facet_support": 0.52,
|
| 392 |
+
"minimum_source_facet_alignment": 0.34
|
| 393 |
}
|
| 394 |
}
|
|
@@ -6,7 +6,8 @@ from typing import Dict, Iterable, List, Sequence, Set, Tuple
|
|
| 6 |
|
| 7 |
from .text import TextProcessor
|
| 8 |
from .text_integrity import TextIntegrityEvaluator
|
| 9 |
-
from .list_canonicalization import ListCanonicalizer
|
|
|
|
| 10 |
from .types import EvidenceFrame, Proposition, PropositionCluster, QueryFrame, ScoredEvidence
|
| 11 |
|
| 12 |
|
|
@@ -42,6 +43,7 @@ class SetCoveragePlanner:
|
|
| 42 |
self.integrity = TextIntegrityEvaluator(text, semantic)
|
| 43 |
self.config = ranking.get("set_coverage", {}) or {}
|
| 44 |
self.canonicalizer = ListCanonicalizer(text, self.config)
|
|
|
|
| 45 |
|
| 46 |
def _plural_interrogative(self, raw: str, lang: str) -> bool:
|
| 47 |
normalized = self.text.normalize(raw, lang)
|
|
@@ -294,6 +296,110 @@ class SetCoveragePlanner:
|
|
| 294 |
return True
|
| 295 |
return self.text.sentence_similarity(left_text, right_text, lang) >= 0.78
|
| 296 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 297 |
def plan(self, query: QueryFrame, selected: Sequence[ScoredEvidence]) -> SetCoveragePlan:
|
| 298 |
raw_candidates: List[Tuple[str, Proposition]] = []
|
| 299 |
rejected: List[str] = []
|
|
@@ -320,6 +426,19 @@ class SetCoveragePlanner:
|
|
| 320 |
if not terms or len(terms) > int(self.config.get("maximum_item_terms", 24)):
|
| 321 |
rejected.append(clean)
|
| 322 |
continue
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 323 |
proposition = Proposition(
|
| 324 |
text=clean if clean[-1:] in ".ุ?!" else clean + ".",
|
| 325 |
normalized=self.text.normalize(clean, query.language),
|
|
@@ -343,7 +462,18 @@ class SetCoveragePlanner:
|
|
| 343 |
raw_candidates.append((clean, proposition))
|
| 344 |
|
| 345 |
canonical_groups, canonical_rejected = self.canonicalizer.canonicalize(query, raw_candidates)
|
| 346 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 347 |
|
| 348 |
source_order: Dict[str, int] = {
|
| 349 |
item.evidence.record_id or str(id(item.source)): rank
|
|
|
|
| 6 |
|
| 7 |
from .text import TextProcessor
|
| 8 |
from .text_integrity import TextIntegrityEvaluator
|
| 9 |
+
from .list_canonicalization import CanonicalListGroup, ListCanonicalizer
|
| 10 |
+
from .facet_contract import FacetContractAnalyzer
|
| 11 |
from .types import EvidenceFrame, Proposition, PropositionCluster, QueryFrame, ScoredEvidence
|
| 12 |
|
| 13 |
|
|
|
|
| 43 |
self.integrity = TextIntegrityEvaluator(text, semantic)
|
| 44 |
self.config = ranking.get("set_coverage", {}) or {}
|
| 45 |
self.canonicalizer = ListCanonicalizer(text, self.config)
|
| 46 |
+
self.facets = FacetContractAnalyzer(text)
|
| 47 |
|
| 48 |
def _plural_interrogative(self, raw: str, lang: str) -> bool:
|
| 49 |
normalized = self.text.normalize(raw, lang)
|
|
|
|
| 296 |
return True
|
| 297 |
return self.text.sentence_similarity(left_text, right_text, lang) >= 0.78
|
| 298 |
|
| 299 |
+
def _temporal_base(self, value: str, lang: str) -> Tuple[str, ...]:
|
| 300 |
+
terms = list(self.text.content_terms(value, lang))
|
| 301 |
+
if lang == "ar":
|
| 302 |
+
glue = {
|
| 303 |
+
"ูู", "ู
ู", "ุงูู", "ุงูู", "ุฎูุงู", "ุงุซูุงุก", "ุฐู", "ุฐู", "ุฐุง",
|
| 304 |
+
"ุงูู", "ุงููู", "ุงุฎุฑ", "ููุงู", "ุจุฏุงู", "ูุงู
ู", "ุจุนุถ",
|
| 305 |
+
"ููู
", "ุงูุงู
", "ููู", "ููุงู", "ุงุณุจูุน", "ุงุณุงุจูุน", "ุดูุฑ", "ุงุดูุฑ", "ุณู", "ุณููุงุช", "ุณุงุน", "ุณุงุนุงุช", "ู
ุฏ",
|
| 306 |
+
"ุนุดุฑ", "ุซูุซ", "ูุตู", "ุฑุจุน",
|
| 307 |
+
}
|
| 308 |
+
else:
|
| 309 |
+
glue = {
|
| 310 |
+
"in", "of", "to", "during", "first", "last", "early", "late",
|
| 311 |
+
"full", "part", "day", "night", "week", "month", "year", "hour",
|
| 312 |
+
"ten", "third", "half", "quarter",
|
| 313 |
+
}
|
| 314 |
+
base = [term for term in terms if term not in glue and not term.isdigit()]
|
| 315 |
+
return tuple(base or terms[-1:])
|
| 316 |
+
|
| 317 |
+
def _temporal_specificity(self, value: str, lang: str) -> float:
|
| 318 |
+
normalized = self.text.normalize(value, lang)
|
| 319 |
+
if lang == "ar":
|
| 320 |
+
patterns = (
|
| 321 |
+
r"\b\d+\b", r"\b(?:ุงูู|ุงููู|ุงุฎุฑ|ููุงู|ุจุฏุงู|ุนุดุฑ|ุซูุซ|ูุตู|ุฑุจุน)\b",
|
| 322 |
+
r"\b(?:ููู
|ููู|ุงุณุจูุน|ุดูุฑ|ุณุงุน|ู
ุฏ)\b",
|
| 323 |
+
)
|
| 324 |
+
else:
|
| 325 |
+
patterns = (
|
| 326 |
+
r"\b\d+\b", r"\b(?:first|last|early|late|ten|third|half|quarter)\b",
|
| 327 |
+
r"\b(?:day|night|week|month|hour|duration)\b",
|
| 328 |
+
)
|
| 329 |
+
return min(1.0, 0.20 * len(self.text.content_terms(value, lang)) + 0.25 * sum(bool(re.search(p, normalized)) for p in patterns))
|
| 330 |
+
|
| 331 |
+
def _enrich_temporal_groups(
|
| 332 |
+
self,
|
| 333 |
+
query: QueryFrame,
|
| 334 |
+
groups: Sequence[CanonicalListGroup],
|
| 335 |
+
raw_candidates: Sequence[Tuple[str, Proposition]],
|
| 336 |
+
) -> Tuple[CanonicalListGroup, ...]:
|
| 337 |
+
if query.facet_dimension != "time":
|
| 338 |
+
return tuple(groups)
|
| 339 |
+
enriched: List[CanonicalListGroup] = []
|
| 340 |
+
for group in groups:
|
| 341 |
+
base = self._temporal_base(group.text, query.language)
|
| 342 |
+
best_text = group.text
|
| 343 |
+
best_specificity = self._temporal_specificity(group.text, query.language)
|
| 344 |
+
props = list(group.propositions)
|
| 345 |
+
for text, prop in raw_candidates:
|
| 346 |
+
candidate_base = self._temporal_base(text, query.language)
|
| 347 |
+
if not base or not candidate_base:
|
| 348 |
+
continue
|
| 349 |
+
forward = self.text.fuzzy_term_overlap(base, candidate_base)
|
| 350 |
+
backward = self.text.fuzzy_term_overlap(candidate_base, base)
|
| 351 |
+
if min(forward, backward) < 0.78:
|
| 352 |
+
continue
|
| 353 |
+
specificity = self._temporal_specificity(text, query.language)
|
| 354 |
+
if specificity > best_specificity:
|
| 355 |
+
best_text = text
|
| 356 |
+
best_specificity = specificity
|
| 357 |
+
if prop not in props:
|
| 358 |
+
props.append(prop)
|
| 359 |
+
enriched.append(CanonicalListGroup(
|
| 360 |
+
text=best_text,
|
| 361 |
+
identity=group.identity,
|
| 362 |
+
propositions=tuple(props),
|
| 363 |
+
score=max([group.score, *(prop.score for prop in props)]),
|
| 364 |
+
))
|
| 365 |
+
return tuple(enriched)
|
| 366 |
+
|
| 367 |
+
def _consolidate_facet_overlaps(
|
| 368 |
+
self, query: QueryFrame, groups: Sequence[CanonicalListGroup]
|
| 369 |
+
) -> Tuple[Tuple[CanonicalListGroup, ...], Tuple[str, ...]]:
|
| 370 |
+
if query.facet_dimension != "time" or len(groups) < 2:
|
| 371 |
+
return tuple(groups), ()
|
| 372 |
+
merged: List[CanonicalListGroup] = []
|
| 373 |
+
rejected: List[str] = []
|
| 374 |
+
for group in groups:
|
| 375 |
+
base = self._temporal_base(group.text, query.language)
|
| 376 |
+
target = None
|
| 377 |
+
for index, existing in enumerate(merged):
|
| 378 |
+
other = self._temporal_base(existing.text, query.language)
|
| 379 |
+
if not base or not other:
|
| 380 |
+
continue
|
| 381 |
+
forward = self.text.fuzzy_term_overlap(base, other)
|
| 382 |
+
backward = self.text.fuzzy_term_overlap(other, base)
|
| 383 |
+
if min(forward, backward) >= 0.78:
|
| 384 |
+
target = index
|
| 385 |
+
break
|
| 386 |
+
if target is None:
|
| 387 |
+
merged.append(group)
|
| 388 |
+
continue
|
| 389 |
+
existing = merged[target]
|
| 390 |
+
current_specificity = self._temporal_specificity(group.text, query.language)
|
| 391 |
+
existing_specificity = self._temporal_specificity(existing.text, query.language)
|
| 392 |
+
chosen = group if current_specificity > existing_specificity else existing
|
| 393 |
+
rejected.append(existing.text if chosen is group else group.text)
|
| 394 |
+
propositions = tuple(dict.fromkeys((*existing.propositions, *group.propositions)))
|
| 395 |
+
merged[target] = CanonicalListGroup(
|
| 396 |
+
text=chosen.text,
|
| 397 |
+
identity=chosen.identity,
|
| 398 |
+
propositions=propositions,
|
| 399 |
+
score=max(existing.score, group.score),
|
| 400 |
+
)
|
| 401 |
+
return tuple(merged), tuple(rejected)
|
| 402 |
+
|
| 403 |
def plan(self, query: QueryFrame, selected: Sequence[ScoredEvidence]) -> SetCoveragePlan:
|
| 404 |
raw_candidates: List[Tuple[str, Proposition]] = []
|
| 405 |
rejected: List[str] = []
|
|
|
|
| 426 |
if not terms or len(terms) > int(self.config.get("maximum_item_terms", 24)):
|
| 427 |
rejected.append(clean)
|
| 428 |
continue
|
| 429 |
+
if query.facet_locked:
|
| 430 |
+
facet = self.facets.align(
|
| 431 |
+
query.facet_dimension,
|
| 432 |
+
f"{item.evidence.metadata_text} {clean}",
|
| 433 |
+
query.language,
|
| 434 |
+
)
|
| 435 |
+
source_facet = float(item.metrics.get("facet_alignment", facet.score))
|
| 436 |
+
if (
|
| 437 |
+
facet.conflicting_support >= float(self.config.get("hard_item_facet_conflict", 0.62))
|
| 438 |
+
and facet.requested_support < float(self.config.get("minimum_item_facet_support", 0.52))
|
| 439 |
+
) or source_facet < float(self.config.get("minimum_source_facet_alignment", 0.34)):
|
| 440 |
+
rejected.append(clean)
|
| 441 |
+
continue
|
| 442 |
proposition = Proposition(
|
| 443 |
text=clean if clean[-1:] in ".ุ?!" else clean + ".",
|
| 444 |
normalized=self.text.normalize(clean, query.language),
|
|
|
|
| 462 |
raw_candidates.append((clean, proposition))
|
| 463 |
|
| 464 |
canonical_groups, canonical_rejected = self.canonicalizer.canonicalize(query, raw_candidates)
|
| 465 |
+
canonical_groups = self._enrich_temporal_groups(query, canonical_groups, raw_candidates)
|
| 466 |
+
used_surfaces = {self.text.normalize(group.text, query.language) for group in canonical_groups}
|
| 467 |
+
rejected.extend(
|
| 468 |
+
value for value in canonical_rejected
|
| 469 |
+
if self.text.normalize(value, query.language) not in used_surfaces
|
| 470 |
+
)
|
| 471 |
+
canonical_groups, facet_rejected = self._consolidate_facet_overlaps(query, canonical_groups)
|
| 472 |
+
final_surfaces = {self.text.normalize(group.text, query.language) for group in canonical_groups}
|
| 473 |
+
rejected.extend(
|
| 474 |
+
value for value in facet_rejected
|
| 475 |
+
if self.text.normalize(value, query.language) not in final_surfaces
|
| 476 |
+
)
|
| 477 |
|
| 478 |
source_order: Dict[str, int] = {
|
| 479 |
item.evidence.record_id or str(id(item.source)): rank
|
|
@@ -428,6 +428,69 @@ def run():
|
|
| 428 |
{"answer": matrix_result.answer, "details": matrix_result.details},
|
| 429 |
)
|
| 430 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 431 |
semantic = pipeline.resources.semantic
|
| 432 |
check("no domain anchor table", "domain_anchor_terms" not in semantic)
|
| 433 |
check("no entity alias table", "concept_aliases" not in semantic)
|
|
|
|
| 428 |
{"answer": matrix_result.answer, "details": matrix_result.details},
|
| 429 |
)
|
| 430 |
|
| 431 |
+
|
| 432 |
+
|
| 433 |
+
# v39.0.1 facet contract: the requested semantic dimension is locked beside
|
| 434 |
+
# the answer operator. A higher-scoring source about the opposite dimension
|
| 435 |
+
# must remain rejected, and overlapping temporal spans collapse to one slot.
|
| 436 |
+
facet_query = "ู
ุง ูู ุงูุญุฏูุฏ ุงูุฒู
ููุฉ ููุนู
ููุฉุ"
|
| 437 |
+
facet_sources = [
|
| 438 |
+
src("facet-time-specific", "ูุชุงุจ ุงูุฒู
ู ุงูู
ูุตู", "ุงูุญุฏูุฏ ุงูุฒู
ููุฉ ููุนู
ููุฉ", facet_query,
|
| 439 |
+
"ุชุญุฏูุฏ ุฒู
ูู", "ุงูุญุฏูุฏ ุงูุฒู
ููุฉ ูู: - ุงูุดูุฑ ุงูุฃูู. - ุงูุดูุฑ ุงูุซุงูู. - ุฃูู ุนุดุฑุฉ ุฃูุงู
ู
ู ุงูุดูุฑ ุงูุซุงูุซ.",
|
| 440 |
+
.88, .93, .80, direct_probability=.88, bm25_score=.88, retriever_agreement=5),
|
| 441 |
+
src("facet-time-broad", "ูุชุงุจ ุงูุฒู
ู ุงูู
ุฌู
ู", "ุงูุญุฏูุฏ ุงูุฒู
ููุฉ ููุนู
ููุฉ", facet_query,
|
| 442 |
+
"ุชุญุฏูุฏ ุฒู
ูู", "ุงูุญุฏูุฏ ุงูุฒู
ููุฉ ูู: - ุงูุดูุฑ ุงูุฃูู. - ุงูุดูุฑ ุงูุซุงูู. - ุงูุดูุฑ ุงูุซุงูุซ.",
|
| 443 |
+
.84, .92, .78, direct_probability=.84, bm25_score=.84, retriever_agreement=5),
|
| 444 |
+
src("facet-location-noise", "ูุชุงุจ ุงูู
ูุงู", "ุงูุญุฏูุฏ ุงูู
ูุงููุฉ ููุนู
ููุฉ", "ู
ุง ูู ุงูุญุฏูุฏ ุงูู
ูุงููุฉ ููุนู
ููุฉุ",
|
| 445 |
+
"ุชุญุฏูุฏ ู
ูุงูู", "ุงูุญุฏูุฏ ุงูู
ูุงููุฉ ูู: - ุงูู
ููุน ุฃูู. - ุงูู
ููุน ุจุงุก. - ุงูู
ููุน ุฌูู
. - ุงูู
ููุน ุฏุงู.",
|
| 446 |
+
.99, .98, .94, direct_probability=.99, bm25_score=.97, retriever_agreement=5),
|
| 447 |
+
]
|
| 448 |
+
facet_result = pipeline.resolve(facet_query, facet_sources, "ar")
|
| 449 |
+
check(
|
| 450 |
+
"explicit facet extracted and removed from subject",
|
| 451 |
+
facet_result.query.facet_dimension == "time"
|
| 452 |
+
and facet_result.query.facet_locked
|
| 453 |
+
and "ุฒู
ู" not in facet_result.query.subject_terms,
|
| 454 |
+
facet_result.details,
|
| 455 |
+
)
|
| 456 |
+
check(
|
| 457 |
+
"opposite facet rejected despite stronger retrieval",
|
| 458 |
+
any(item.evidence.record_id == "facet-location-noise" and not item.accepted and "facet_dimension_mismatch" in item.hard_rejections for item in facet_result.ranked)
|
| 459 |
+
and "ุงูู
ููุน ุฃูู" not in facet_result.answer
|
| 460 |
+
and "ูุชุงุจ ุงูู
ูุงู" not in facet_result.answer,
|
| 461 |
+
{"answer": facet_result.answer, "details": facet_result.details},
|
| 462 |
+
)
|
| 463 |
+
check(
|
| 464 |
+
"temporal overlap consolidated to one slot",
|
| 465 |
+
facet_result.details.get("set_coverage", {}).get("unique_answer_items") == 3
|
| 466 |
+
and facet_result.answer.count("ุงูุดูุฑ ุงูุซุงูุซ") == 1
|
| 467 |
+
and "ุฃูู ุนุดุฑุฉ ุฃูุงู
ู
ู ุงูุดูุฑ ุงูุซุงูุซ" in facet_result.answer,
|
| 468 |
+
facet_result.answer,
|
| 469 |
+
)
|
| 470 |
+
check(
|
| 471 |
+
"global arbiter enforces facet contract",
|
| 472 |
+
facet_result.details.get("global_arbitration", {}).get("passed") is True,
|
| 473 |
+
facet_result.details,
|
| 474 |
+
)
|
| 475 |
+
|
| 476 |
+
english_facet_query = "What are the temporal limits of the process?"
|
| 477 |
+
english_facet_sources = [
|
| 478 |
+
src("en-facet-time", "Time Source", "Temporal limits of the process", english_facet_query,
|
| 479 |
+
"temporal list", "- Month one: the first temporal period. - Month two: the second temporal period. - The first ten days of month three: the final temporal period.",
|
| 480 |
+
.87, .92, .79, direct_probability=.87, bm25_score=.87, retriever_agreement=5),
|
| 481 |
+
src("en-facet-space", "Space Source", "Spatial limits of the process", "What are the spatial limits of the process?",
|
| 482 |
+
"spatial list", "- Site Alpha: the first spatial location. - Site Beta: the second spatial location. - Site Gamma: the third spatial location.",
|
| 483 |
+
.99, .98, .93, direct_probability=.99, bm25_score=.96, retriever_agreement=5),
|
| 484 |
+
]
|
| 485 |
+
english_facet_result = pipeline.resolve(english_facet_query, english_facet_sources, "en")
|
| 486 |
+
check(
|
| 487 |
+
"facet contract is bilingual",
|
| 488 |
+
english_facet_result.query.facet_dimension == "time"
|
| 489 |
+
and "Site Alpha" not in english_facet_result.answer
|
| 490 |
+
and "Month one" in english_facet_result.answer,
|
| 491 |
+
english_facet_result.answer,
|
| 492 |
+
)
|
| 493 |
+
|
| 494 |
semantic = pipeline.resources.semantic
|
| 495 |
check("no domain anchor table", "domain_anchor_terms" not in semantic)
|
| 496 |
check("no entity alias table", "concept_aliases" not in semantic)
|
|
@@ -37,6 +37,11 @@ class QueryFrame:
|
|
| 37 |
scope_terms: Tuple[str, ...] = ()
|
| 38 |
provenance_terms: Tuple[str, ...] = ()
|
| 39 |
contract_pattern: str = ""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
|
| 41 |
|
| 42 |
@dataclass(frozen=True)
|
|
|
|
| 37 |
scope_terms: Tuple[str, ...] = ()
|
| 38 |
provenance_terms: Tuple[str, ...] = ()
|
| 39 |
contract_pattern: str = ""
|
| 40 |
+
facet_dimension: str = ""
|
| 41 |
+
facet_terms: Tuple[str, ...] = ()
|
| 42 |
+
facet_locked: bool = False
|
| 43 |
+
facet_confidence: float = 0.0
|
| 44 |
+
facet_pattern: str = ""
|
| 45 |
|
| 46 |
|
| 47 |
@dataclass(frozen=True)
|
|
@@ -0,0 +1,79 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"version": "39.0.1",
|
| 3 |
+
"passed": true,
|
| 4 |
+
"question": "ู
ุง ูู ุงูู
ูุงููุช ุงูุฒู
ุงููุฉ ููุญุฌุ",
|
| 5 |
+
"answer": "**ุงูุณุคุงู:** ู
ุง ูู ุงูู
ูุงููุช ุงูุฒู
ุงููุฉ ููุญุฌุ\n\n**ุงูุฅุฌุงุจุฉ:**\n1. ุดูุงู. _[ุงูุฑูุถ ุงูู
ุฑุจุน ุดุฑุญ ุฒุงุฏ ุงูู
ุณุชููุน (ุต 596)ุ ู
ูุงุณู ุงูุญุฌ ูุงูุนู
ุฑุฉ (ุต 21)]_\n2. ุฐู ุงููุนุฏุฉ. _[ุงูุฑูุถ ุงูู
ุฑุจุน ุดุฑุญ ุฒุงุฏ ุงูู
ุณุชููุน (ุต 596)ุ ู
ูุงุณู ุงูุญุฌ ูุงูุนู
ุฑุฉ (ุต 21)]_\n3. ุนุดุฑ ู
ู ุฐู ุงูุญุฌุฉ. _[ุงูุฑูุถ ุงูู
ุฑุจุน ุดุฑุญ ุฒุงุฏ ุงูู
ุณุชููุน (ุต 596)ุ ู
ูุงุณู ุงูุญุฌ ูุงูุนู
ุฑุฉ (ุต 21)]_\n\n**ุงูู
ุตุงุฏุฑ ุงูุชู ุจููู ุนูููุง ุงูุฌูุงุจ:**\n- ุงูุฑูุถ ุงูู
ุฑุจุน ุดุฑุญ ุฒุงุฏ ุงูู
ุณุชููุน (ุต 596)\n- ู
ูุงุณู ุงูุญุฌ ูุงูุนู
ุฑุฉ (ุต 21)",
|
| 6 |
+
"confidence": 0.819734,
|
| 7 |
+
"query_contract": {
|
| 8 |
+
"operator": "list",
|
| 9 |
+
"requested_shape": "list",
|
| 10 |
+
"locked": false,
|
| 11 |
+
"confidence": 0.76,
|
| 12 |
+
"scope_terms": [],
|
| 13 |
+
"provenance_terms": [],
|
| 14 |
+
"pattern": "^\\s*(?:ุงุฐูุฑ|ุนุฏุฏ|ู
ุง\\s+ูู|ู
ุงูู)\\b",
|
| 15 |
+
"facet_dimension": "time",
|
| 16 |
+
"facet_terms": [
|
| 17 |
+
"ุฒู
ุงู"
|
| 18 |
+
],
|
| 19 |
+
"facet_locked": true,
|
| 20 |
+
"facet_confidence": 0.97,
|
| 21 |
+
"facet_pattern": "\\b(?:ุงู)?(?:ุฒู
ุงูู(?:ุฉ|ู)?|ุฒู
ูู(?:ุฉ|ู)?|ููุชู(?:ุฉ|ู)?|ุชูููุช(?:ูุฉ)?|ุฃููุงุช?|ุงููุงุช|ุฃุฒู
ูุฉ|ุงุฒู
ูุฉ)\\b"
|
| 22 |
+
},
|
| 23 |
+
"global_arbitration": {
|
| 24 |
+
"passed": true,
|
| 25 |
+
"violations": [],
|
| 26 |
+
"direct_source_ids": [
|
| 27 |
+
"temporal-specific",
|
| 28 |
+
"temporal-broad"
|
| 29 |
+
],
|
| 30 |
+
"confidence_cap": 1.0
|
| 31 |
+
},
|
| 32 |
+
"set_coverage": {
|
| 33 |
+
"active": true,
|
| 34 |
+
"candidate_sources": 2,
|
| 35 |
+
"unique_answer_items": 3,
|
| 36 |
+
"used_books": 2
|
| 37 |
+
},
|
| 38 |
+
"checks": {
|
| 39 |
+
"operator_is_list": true,
|
| 40 |
+
"time_facet_locked": true,
|
| 41 |
+
"facet_removed_from_subject": true,
|
| 42 |
+
"spatial_source_rejected": true,
|
| 43 |
+
"spatial_items_not_rendered": true,
|
| 44 |
+
"three_temporal_slots": true,
|
| 45 |
+
"overlapping_span_consolidated": true,
|
| 46 |
+
"global_arbiter_passed": true
|
| 47 |
+
},
|
| 48 |
+
"ranked": [
|
| 49 |
+
{
|
| 50 |
+
"record_id": "temporal-specific",
|
| 51 |
+
"accepted": true,
|
| 52 |
+
"score": 1.0,
|
| 53 |
+
"rejections": [],
|
| 54 |
+
"facet_alignment": 1.0,
|
| 55 |
+
"facet_requested_support": 1.0,
|
| 56 |
+
"facet_conflicting_support": 0.0
|
| 57 |
+
},
|
| 58 |
+
{
|
| 59 |
+
"record_id": "temporal-broad",
|
| 60 |
+
"accepted": true,
|
| 61 |
+
"score": 0.827982,
|
| 62 |
+
"rejections": [],
|
| 63 |
+
"facet_alignment": 1.0,
|
| 64 |
+
"facet_requested_support": 1.0,
|
| 65 |
+
"facet_conflicting_support": 0.0
|
| 66 |
+
},
|
| 67 |
+
{
|
| 68 |
+
"record_id": "spatial-high",
|
| 69 |
+
"accepted": false,
|
| 70 |
+
"score": 0.916164,
|
| 71 |
+
"rejections": [
|
| 72 |
+
"facet_dimension_mismatch"
|
| 73 |
+
],
|
| 74 |
+
"facet_alignment": 0.0,
|
| 75 |
+
"facet_requested_support": 0.0,
|
| 76 |
+
"facet_conflicting_support": 1.0
|
| 77 |
+
}
|
| 78 |
+
]
|
| 79 |
+
}
|
|
The diff for this file is too large to render.
See raw diff
|
|
|
|
@@ -0,0 +1,212 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"version": "39.0.1",
|
| 3 |
+
"algorithm": "sha256",
|
| 4 |
+
"file_count_excluding_manifest": 41,
|
| 5 |
+
"files": [
|
| 6 |
+
{
|
| 7 |
+
"path": "DEPLOY_HUDANET_V39_0_1.md",
|
| 8 |
+
"bytes": 3299,
|
| 9 |
+
"sha256": "3f8cd40da7e8e277b2aa1444ecb51087bff5619a95760f337f44184cd27c2000"
|
| 10 |
+
},
|
| 11 |
+
{
|
| 12 |
+
"path": "app.py",
|
| 13 |
+
"bytes": 567454,
|
| 14 |
+
"sha256": "15e5c8f4caf5a5251448068f76d4537a4912de7359dae3f01ab258021c42b3a6"
|
| 15 |
+
},
|
| 16 |
+
{
|
| 17 |
+
"path": "hudanet_core/README.md",
|
| 18 |
+
"bytes": 2439,
|
| 19 |
+
"sha256": "c66001cf9174d9e7568d2f5285b0eca03091b5cf3b43c6e6b3fc806c88cacd58"
|
| 20 |
+
},
|
| 21 |
+
{
|
| 22 |
+
"path": "hudanet_core/__init__.py",
|
| 23 |
+
"bytes": 213,
|
| 24 |
+
"sha256": "ff9b35abb727302a9297f875ded5794d36d7f8a34918ae8aef0384cef3ad5735"
|
| 25 |
+
},
|
| 26 |
+
{
|
| 27 |
+
"path": "hudanet_core/answerability.py",
|
| 28 |
+
"bytes": 1023,
|
| 29 |
+
"sha256": "3af690623128621b6be5e9eddf03672172b9cb6fa70ea107ceed60f40a74209c"
|
| 30 |
+
},
|
| 31 |
+
{
|
| 32 |
+
"path": "hudanet_core/centrality.py",
|
| 33 |
+
"bytes": 2230,
|
| 34 |
+
"sha256": "e68fc0bdb9f787996930b927861affb645e95ac040a8857605cbfee9930984cf"
|
| 35 |
+
},
|
| 36 |
+
{
|
| 37 |
+
"path": "hudanet_core/compatibility.py",
|
| 38 |
+
"bytes": 21886,
|
| 39 |
+
"sha256": "784540d1884a81fefda57d3a72dc11c2927ffaf2f89ec5034014061b616d8f44"
|
| 40 |
+
},
|
| 41 |
+
{
|
| 42 |
+
"path": "hudanet_core/consensus.py",
|
| 43 |
+
"bytes": 7950,
|
| 44 |
+
"sha256": "c03a2c3cd290adfb0b0a92729c39616f2c5e1b8cb2c265104f66fcf94a8d5db9"
|
| 45 |
+
},
|
| 46 |
+
{
|
| 47 |
+
"path": "hudanet_core/contextual_sense.py",
|
| 48 |
+
"bytes": 6633,
|
| 49 |
+
"sha256": "1d2a44e785288d9df8c66811ff0e48b728887d1c3bdfeb5869e08a63fdafa38d"
|
| 50 |
+
},
|
| 51 |
+
{
|
| 52 |
+
"path": "hudanet_core/evidence.py",
|
| 53 |
+
"bytes": 10861,
|
| 54 |
+
"sha256": "659c3cc5a2580f8634bc40fa627f11d4df3a5edf82c9ae9cf785408e3fb2f86f"
|
| 55 |
+
},
|
| 56 |
+
{
|
| 57 |
+
"path": "hudanet_core/facet_contract.py",
|
| 58 |
+
"bytes": 9478,
|
| 59 |
+
"sha256": "1ea22d2faf83d8bd6a80938d6a4e46fe48e905b421d99a6e47245fc6cd20c71c"
|
| 60 |
+
},
|
| 61 |
+
{
|
| 62 |
+
"path": "hudanet_core/global_arbiter.py",
|
| 63 |
+
"bytes": 5148,
|
| 64 |
+
"sha256": "cf26015d166f738ddb625114aa81269729ddcbd88ea2289838dda64430607215"
|
| 65 |
+
},
|
| 66 |
+
{
|
| 67 |
+
"path": "hudanet_core/grounding.py",
|
| 68 |
+
"bytes": 1108,
|
| 69 |
+
"sha256": "7abe6dd19b685ae687ee1242dd9301306efbe2ecffc160aa26011ea5f206f74a"
|
| 70 |
+
},
|
| 71 |
+
{
|
| 72 |
+
"path": "hudanet_core/intent_repair.py",
|
| 73 |
+
"bytes": 5487,
|
| 74 |
+
"sha256": "d3a3390da753fa6d60ffb14ef33638f75d2cc7e91f302d7f68d818773cae18d9"
|
| 75 |
+
},
|
| 76 |
+
{
|
| 77 |
+
"path": "hudanet_core/issue_clustering.py",
|
| 78 |
+
"bytes": 5548,
|
| 79 |
+
"sha256": "b76e3ebd47cc4360d854d32960884912e759ead1d477b58558ee252dd9f19348"
|
| 80 |
+
},
|
| 81 |
+
{
|
| 82 |
+
"path": "hudanet_core/list_canonicalization.py",
|
| 83 |
+
"bytes": 20939,
|
| 84 |
+
"sha256": "1cff8a44f7aee7f7df3088bd542b1a924d741436b7bb22d7d19eb86fad2805a6"
|
| 85 |
+
},
|
| 86 |
+
{
|
| 87 |
+
"path": "hudanet_core/pipeline.py",
|
| 88 |
+
"bytes": 19674,
|
| 89 |
+
"sha256": "4785f489bd9526a5c21db00eda81fc3c8d2d46be6f8f84cadf3851cddaf74fba"
|
| 90 |
+
},
|
| 91 |
+
{
|
| 92 |
+
"path": "hudanet_core/propositions.py",
|
| 93 |
+
"bytes": 29258,
|
| 94 |
+
"sha256": "a3feeb76459c56a847c1d71562a740ff94cd956e3d8d30b7567e1ed829c6385c"
|
| 95 |
+
},
|
| 96 |
+
{
|
| 97 |
+
"path": "hudanet_core/query.py",
|
| 98 |
+
"bytes": 15496,
|
| 99 |
+
"sha256": "2c4a1e0870eebac51dcbadf616b1a1ddedfe98dc7803481926c2616f7b1ea44d"
|
| 100 |
+
},
|
| 101 |
+
{
|
| 102 |
+
"path": "hudanet_core/query_contract.py",
|
| 103 |
+
"bytes": 9032,
|
| 104 |
+
"sha256": "1fcbde9b07402945c9e3cb0779744f236a45b9d3c13686b1d64ed3aaaa6854e6"
|
| 105 |
+
},
|
| 106 |
+
{
|
| 107 |
+
"path": "hudanet_core/relation_graph.py",
|
| 108 |
+
"bytes": 4902,
|
| 109 |
+
"sha256": "b1713307bd5a66d97fe4521512bb7ab92db22b724f05e6e3ca286d562fbbf420"
|
| 110 |
+
},
|
| 111 |
+
{
|
| 112 |
+
"path": "hudanet_core/rendering.py",
|
| 113 |
+
"bytes": 6922,
|
| 114 |
+
"sha256": "7303e3e1156ab9cbb1a3129e48e3c64535d5fd775d9b5e978ea0236ac383f6fd"
|
| 115 |
+
},
|
| 116 |
+
{
|
| 117 |
+
"path": "hudanet_core/resources/answer_templates.json",
|
| 118 |
+
"bytes": 5149,
|
| 119 |
+
"sha256": "d4efe1f130a8179f957b5150c1befc75a5bb859cc7f5d2e2db965a7b4f4a5a1b"
|
| 120 |
+
},
|
| 121 |
+
{
|
| 122 |
+
"path": "hudanet_core/resources/evidence_schema.json",
|
| 123 |
+
"bytes": 732,
|
| 124 |
+
"sha256": "522d171dd2b6371a3d9b1d01e9395ab12dcbccbbd96e546dfa965a05c77888a4"
|
| 125 |
+
},
|
| 126 |
+
{
|
| 127 |
+
"path": "hudanet_core/resources/ranking_config.json",
|
| 128 |
+
"bytes": 9887,
|
| 129 |
+
"sha256": "0c59ea40367c45d8df8910ad5572c9581df46eff16a6364718399a289612a802"
|
| 130 |
+
},
|
| 131 |
+
{
|
| 132 |
+
"path": "hudanet_core/resources/semantic_rules.json",
|
| 133 |
+
"bytes": 35055,
|
| 134 |
+
"sha256": "8425062a5aa9f16cf49f7854b3463e61fbf8e9a49d0de3f52f88a2b560586210"
|
| 135 |
+
},
|
| 136 |
+
{
|
| 137 |
+
"path": "hudanet_core/resources.py",
|
| 138 |
+
"bytes": 2268,
|
| 139 |
+
"sha256": "d043d3df9da9ec9c43b9a269f395ba2754f08945a80f98ff6b6516fad3087e1d"
|
| 140 |
+
},
|
| 141 |
+
{
|
| 142 |
+
"path": "hudanet_core/roles.py",
|
| 143 |
+
"bytes": 959,
|
| 144 |
+
"sha256": "d0e0a4b5fb7bb58f44cbbc62f9c0ffcb22628f2bc8811117919c4be7f1a38936"
|
| 145 |
+
},
|
| 146 |
+
{
|
| 147 |
+
"path": "hudanet_core/ruling_frame.py",
|
| 148 |
+
"bytes": 11249,
|
| 149 |
+
"sha256": "de6c700b7cce1c4507ccf9eb13f5716a9dbc7ffcf7f83b4ca9df3768672de18f"
|
| 150 |
+
},
|
| 151 |
+
{
|
| 152 |
+
"path": "hudanet_core/set_coverage.py",
|
| 153 |
+
"bytes": 27777,
|
| 154 |
+
"sha256": "ec91e55e5b2ab0b7dabe6dd6367579dceb829bf90226b44c3846998bd6e96699"
|
| 155 |
+
},
|
| 156 |
+
{
|
| 157 |
+
"path": "hudanet_core/synthesis.py",
|
| 158 |
+
"bytes": 20771,
|
| 159 |
+
"sha256": "36de2cdba5443bf9ef1ce983407bad528e187fd8a66115c67c35b985f15f5a2c"
|
| 160 |
+
},
|
| 161 |
+
{
|
| 162 |
+
"path": "hudanet_core/tests/test_generic_pipeline.py",
|
| 163 |
+
"bytes": 37431,
|
| 164 |
+
"sha256": "7796e34f99577e87054d7c5228e5f15956e729a3be7042161b396775cc0991a2"
|
| 165 |
+
},
|
| 166 |
+
{
|
| 167 |
+
"path": "hudanet_core/text.py",
|
| 168 |
+
"bytes": 12005,
|
| 169 |
+
"sha256": "ca4805236af4e883cad820ccdd629d31f77a63a673605b0579a806e1d9a73b84"
|
| 170 |
+
},
|
| 171 |
+
{
|
| 172 |
+
"path": "hudanet_core/text_integrity.py",
|
| 173 |
+
"bytes": 4942,
|
| 174 |
+
"sha256": "1faa64d233350750389479d19c46bafd63f475f66431554126ba888e8456844b"
|
| 175 |
+
},
|
| 176 |
+
{
|
| 177 |
+
"path": "hudanet_core/types.py",
|
| 178 |
+
"bytes": 4238,
|
| 179 |
+
"sha256": "79b989d7f10f2a55b0992921ca6283609fbeac30937ff648501ed849647dbc3c"
|
| 180 |
+
},
|
| 181 |
+
{
|
| 182 |
+
"path": "hudanet_dialects.json",
|
| 183 |
+
"bytes": 61047,
|
| 184 |
+
"sha256": "9d73324aa9a53cef401b1b155691c5c594504ef5a89618709856d8ff86032413"
|
| 185 |
+
},
|
| 186 |
+
{
|
| 187 |
+
"path": "hudanet_retrieval_rules.json",
|
| 188 |
+
"bytes": 201,
|
| 189 |
+
"sha256": "e5c5f7e6af72c062262419bb7384b8206013522710306d3fa12387ac108534fb"
|
| 190 |
+
},
|
| 191 |
+
{
|
| 192 |
+
"path": "hudanet_v39_0_1_facet_contract_regression.json",
|
| 193 |
+
"bytes": 2748,
|
| 194 |
+
"sha256": "a1eed448b06446840d8906025d17020b3625a4b3f4283c1cf2c8f6dda39ff434"
|
| 195 |
+
},
|
| 196 |
+
{
|
| 197 |
+
"path": "hudanet_v39_0_1_generic_tests.json",
|
| 198 |
+
"bytes": 148013,
|
| 199 |
+
"sha256": "b7dc82e37152cf8b1a7d322ab1f370f3e099904736ea7953926e7e32a5a53615"
|
| 200 |
+
},
|
| 201 |
+
{
|
| 202 |
+
"path": "hudanet_v39_0_1_query_contract_regression.json",
|
| 203 |
+
"bytes": 2209,
|
| 204 |
+
"sha256": "603f5068d2b07b915c107313102d7710f2b7f572c0c31b9c775b9d0b3be97cec"
|
| 205 |
+
},
|
| 206 |
+
{
|
| 207 |
+
"path": "hudanet_v39_0_1_validation_report.json",
|
| 208 |
+
"bytes": 2723,
|
| 209 |
+
"sha256": "b1f0853e0cfa16623f65e1bcdad00e60de751630d3b06601fbd292f3d5bd7eaf"
|
| 210 |
+
}
|
| 211 |
+
]
|
| 212 |
+
}
|
|
@@ -0,0 +1,83 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"version": "39.0.1",
|
| 3 |
+
"passed": true,
|
| 4 |
+
"question": "ู
ุง ูู ู
ูููู
ุงูุฅุญุฑุงู
ูู ุงูููู ุงูุฅุณูุงู
ูุ",
|
| 5 |
+
"answer": "**ุงูุณุคุงู:** ู
ุง ูู ู
ูููู
ุงูุฅุญุฑุงู
ูู ุงูููู ุงูุฅุณูุงู
ูุ\n\n**ุงูุฅุฌุงุจุฉ:** ุงูุฅุญุฑุงู
ูู ููุฉ ุงููุณูุ ููุง ููุนูุฏ ุฅูุง ุจููุฉ\n\n**ุงูู
ุตุงุฏุฑ ุงูุชู ุจููู ุนูููุง ุงูุฌูุงุจ:**\n- ุงูู
ุบูู",
|
| 6 |
+
"confidence": 0.900979,
|
| 7 |
+
"query_contract": {
|
| 8 |
+
"operator": "definition",
|
| 9 |
+
"requested_shape": "single_definition",
|
| 10 |
+
"locked": true,
|
| 11 |
+
"confidence": 0.94,
|
| 12 |
+
"scope_terms": [
|
| 13 |
+
"ููู",
|
| 14 |
+
"ุงุณูุงู
"
|
| 15 |
+
],
|
| 16 |
+
"provenance_terms": [],
|
| 17 |
+
"pattern": "(?:^|\\s)(?:ู
ุง\\s+ูู|ู
ุงูู|ู
ุง\\s+ูู|ู
ุงูู|ู
ุง)\\s+(?:ู
ูููู
|ุชุนุฑูู|ู
ุนูู|ุงูู
ูุตูุฏ\\s+ุจ?)\\b",
|
| 18 |
+
"facet_dimension": "",
|
| 19 |
+
"facet_terms": [],
|
| 20 |
+
"facet_locked": false,
|
| 21 |
+
"facet_confidence": 0.0,
|
| 22 |
+
"facet_pattern": ""
|
| 23 |
+
},
|
| 24 |
+
"global_arbitration": {
|
| 25 |
+
"passed": true,
|
| 26 |
+
"violations": [],
|
| 27 |
+
"direct_source_ids": [
|
| 28 |
+
"definition-2"
|
| 29 |
+
],
|
| 30 |
+
"confidence_cap": 1.0
|
| 31 |
+
},
|
| 32 |
+
"checks": {
|
| 33 |
+
"operator_definition": true,
|
| 34 |
+
"contract_locked": true,
|
| 35 |
+
"single_definition_shape": true,
|
| 36 |
+
"scope_separated": true,
|
| 37 |
+
"definition_rendered": true,
|
| 38 |
+
"list_distractor_not_rendered": true,
|
| 39 |
+
"global_arbiter_passed": true
|
| 40 |
+
},
|
| 41 |
+
"ranked": [
|
| 42 |
+
{
|
| 43 |
+
"record_id": "definition-2",
|
| 44 |
+
"accepted": true,
|
| 45 |
+
"score": 0.961313,
|
| 46 |
+
"answer_types": [
|
| 47 |
+
"definition",
|
| 48 |
+
"conditions",
|
| 49 |
+
"exception",
|
| 50 |
+
"ruling"
|
| 51 |
+
],
|
| 52 |
+
"rejections": [],
|
| 53 |
+
"contract_type_match": 1.0
|
| 54 |
+
},
|
| 55 |
+
{
|
| 56 |
+
"record_id": "definition-1",
|
| 57 |
+
"accepted": true,
|
| 58 |
+
"score": 0.951863,
|
| 59 |
+
"answer_types": [
|
| 60 |
+
"definition",
|
| 61 |
+
"location",
|
| 62 |
+
"ruling"
|
| 63 |
+
],
|
| 64 |
+
"rejections": [],
|
| 65 |
+
"contract_type_match": 1.0
|
| 66 |
+
},
|
| 67 |
+
{
|
| 68 |
+
"record_id": "list-distractor",
|
| 69 |
+
"accepted": false,
|
| 70 |
+
"score": 0.764951,
|
| 71 |
+
"answer_types": [
|
| 72 |
+
"amount",
|
| 73 |
+
"list",
|
| 74 |
+
"ruling"
|
| 75 |
+
],
|
| 76 |
+
"rejections": [
|
| 77 |
+
"answer_contract_mismatch",
|
| 78 |
+
"request_type_mismatch"
|
| 79 |
+
],
|
| 80 |
+
"contract_type_match": 0.0
|
| 81 |
+
}
|
| 82 |
+
]
|
| 83 |
+
}
|
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"version": "39.0.1",
|
| 3 |
+
"passed": true,
|
| 4 |
+
"architecture": "central_query_contract_with_locked_semantic_facets_and_global_answer_arbiter",
|
| 5 |
+
"python_files_compiled": 29,
|
| 6 |
+
"compile_failures": [],
|
| 7 |
+
"json_files_validated_before_manifest": 9,
|
| 8 |
+
"json_failures": [],
|
| 9 |
+
"semantic_regex_checked": 352,
|
| 10 |
+
"regex_failures": [],
|
| 11 |
+
"generic_tests_passed": 66,
|
| 12 |
+
"generic_tests_total": 66,
|
| 13 |
+
"facet_contract_regression_passed": true,
|
| 14 |
+
"facet_contract_checks_passed": 8,
|
| 15 |
+
"facet_contract_checks_total": 8,
|
| 16 |
+
"query_contract_regression_passed": true,
|
| 17 |
+
"query_contract_checks_passed": 7,
|
| 18 |
+
"query_contract_checks_total": 7,
|
| 19 |
+
"lightweight_app_preflight_executed": true,
|
| 20 |
+
"lightweight_app_preflight_passed": true,
|
| 21 |
+
"production_topic_specific_hits": [],
|
| 22 |
+
"zip_extraction_verified": true,
|
| 23 |
+
"post_extraction_generic_tests_passed": true,
|
| 24 |
+
"manifest_verified_after_extraction": true,
|
| 25 |
+
"runtime_full_download_executed": false,
|
| 26 |
+
"runtime_note": "The private multi-gigabyte Hugging Face runtime was not downloaded in this container. The real app preflight was executed from app.py with the download call stopped immediately afterward; compilation, resources, generic tests, target regressions, package extraction, and post-extraction tests were executed.",
|
| 27 |
+
"verification_diagnostics": {
|
| 28 |
+
"missing": [],
|
| 29 |
+
"mismatch": [],
|
| 30 |
+
"test_stdout_tail": " \"source_record_ids\": [\n \"facet-time-broad\",\n \"facet-time-specific\"\n ],\n \"source_books\": [\n \"ูุชุงุจ ุงูุฒู
ู ุงูู
ุฌู
ู\",\n \"ูุชุงุจ ุงูุฒู
ู ุงูู
ูุตู\"\n ],\n \"source_pages\": [],\n \"role\": \"statement\"\n }\n ],\n \"rejected_fragments\": [],\n \"answerability_state\": \"grounded\",\n \"set_coverage\": {\n \"active\": true,\n \"candidate_sources\": 2,\n \"unique_answer_items\": 3,\n \"used_books\": 2\n }\n }\n },\n {\n \"name\": \"facet contract is bilingual\",\n \"passed\": true,\n \"value\": \"**Question:** What are the temporal limits of the process?\\n\\n**Answer:**\\n1. Month one: the first temporal period. _[Time Source]_\\n2. Month two: the second temporal period. _[Time Source]_\\n\\n**Sources used for the answer:**\\n- Time Source\"\n },\n {\n \"name\": \"no domain anchor table\",\n \"passed\": true,\n \"value\": null\n },\n {\n \"name\": \"no entity alias table\",\n \"passed\": true,\n \"value\": null\n },\n {\n \"name\": \"no named intent table\",\n \"passed\": true,\n \"value\": null\n }\n ]\n}\n",
|
| 31 |
+
"test_stderr": ""
|
| 32 |
+
}
|
| 33 |
+
}
|