| CONFIRMED BY THE USER FOR THIS RUN |
|
|
| 1. Fixed caption and visual context |
| The canonical `caption` is supplied from `refined_caption` by removing only the |
| source <p>, </p>, and [SEG:n] markup, collapsing whitespace, and trimming its ends. |
| Its wording and punctuation are otherwise preserved. Return this exact string. |
| All source fields are retained outside your output. Do not treat the source SEG |
| markers, token indices, or old caption-to-object links as gold text annotations; |
| those old links are deliberately not supplied as answers. |
| You see ten clean sampled RGB frames in temporal order; only ONE reference frame |
| has supplied masks. Frame indices 00--09 are sample indices, not native frame IDs. |
| The clean frames are context for disambiguation and visual fact checking. Grounded |
| mask links refer only to the supplied masks at the stated reference frame. |
| The reference RGB, colored ID mask overlays, per-ID identity crops, and labeled |
| binary-mask sheets identify the available masks. All supplied source masks are |
| shown; no generated or propagated masks are provided. Match by the actual images, |
| not just by the source category hint. A supplied person mask does not automatically |
| provide a separate hand, face, clothing, or other part mask. |
|
|
| 2. Missing-mask policy explicitly chosen by the user |
| Annotate the full caption according to the complete guidelines, including eligible |
| entities for which no matching source mask is supplied. Do not omit their text, |
| rewrite the caption to remove them, or attach their text to a different entity's |
| mask. Keep all mentions/coreference groups. Set `mask_id` to null, choose the |
| appropriate grounding_status, and include a brief reason. These groups remain |
| incomplete spatial annotations; they are not complete image--text correspondences |
| and must not be called final one-to-one training examples. If an entity is visible |
| only in a context frame, or is not verifiable in any supplied image, report that |
| visibility explicitly; never invent a reference-frame mask or assert an unobserved |
| object is present. The caption remains unchanged even when its content is suspect. |
|
|
| 3. Exact spans and inconsistent examples explicitly reviewed by the user |
| The source document above remains unchanged. In its Example 7, `a cloudy sky` is |
| actually at [38,50), whereas [37,50) includes a preceding space. All actual indices |
| for this run are computed locally against the supplied canonical caption, and |
| every resulting caption[start:end] must equal the exact mention text. |
| Return exact mention substrings and their 1-based occurrence numbers, not guessed |
| numeric character offsets. Include every repeated occurrence independently. |
| In section 3.1.1, `rightmost bus` is not a contiguous substring of |
| `the rightmost and leftmost bus`; likewise `wooden chair` is not a contiguous |
| substring of `the wooden and metal chair`. When a construction cannot be expressed |
| as exact continuous spans under the original rules, report a rule_conflict issue |
| and put the unresolved text in that issue. Preserve every other valid annotation; |
| do not fabricate a substring, silently join words, rewrite the text, or choose an |
| unapproved reinterpretation. The affected sample requires manual review and is |
| excluded from the formal completed set until the issue is resolved. |
| The document contains stale section references to section 2.8; its complete |
| counter/container/proper-name rules are present in sections 2.6.1--2.6.3. No section |
| has been deleted from the supplied original document. |
|
|
| 4. Group-to-mask completeness |
| Keep the document's text grouping decisions (including plural groups, stuff, |
| body parts, possessives, selectors, and shared anchors). Each resolved group has |
| one supplied `mask_id`. If a collective group would require merging several |
| separate source masks, leave mask_id null, use requires_mask_union, and list the |
| source IDs in candidate_mask_ids for review. Do not silently split a text group, |
| merge source masks, or relabel several distinct instances as one confirmed mask. |
| If an identity is ambiguous, use mask_id null and ambiguous, with candidates where |
| available. A possibly incorrect/partial source mask can be marked mask_quality_issue. |
| Do not force a match to maximize coverage. Separate missing_mask from ambiguous |
| identity and from not_visible_in_reference. Repeated mentions of one entity stay |
| in the same group, but do not merge distinct entities just because their categories |
| match or an old source caption would have linked them to the same SEG ID. |
|
|
| 5. Output transport only -- not a replacement for the guidelines |
| Return exactly one JSON object, no Markdown or explanatory text outside the JSON: |
| { |
| "sample_id": "the supplied sample ID", |
| "caption": "the exact supplied canonical caption", |
| "groups": [ |
| { |
| "group_id": "g1", |
| "entity_label": "brief identity description", |
| "entity_kind": "thing | stuff | body_part | collective", |
| "mentions": [{"text": "exact substring", "occurrence": 1}], |
| "mask_id": "m0 or null", |
| "candidate_mask_ids": [], |
| "grounding_status": "matched | missing_mask | not_visible_in_reference | ambiguous | requires_mask_union | mask_quality_issue", |
| "visibility": "reference_visible | context_only | not_observed | uncertain", |
| "confidence": "high | medium | low", |
| "reason": "brief evidence or reason for the missing/uncertain link" |
| } |
| ], |
| "issues": [{"type": "rule_conflict | caption_visual_mismatch | uncertain_coreference | other", "text": "exact relevant text or empty string", "description": "brief explanation"}], |
| "needs_review": true, |
| "review_reasons": [] |
| } |
| Use actual JSON null (not the string "null") for an unresolved mask_id. Enumerated |
| values above are alternatives; choose one value, not the whole pipe-separated list. |
| Use only supplied mask IDs for links/candidates. `matched` requires one non-null |
| mask ID, reference_visible, and clear same-entity visual evidence. Missing spatial |
| labels are not background/negative evidence. Set needs_review=true for every |
| unresolved mask/group, uncertainty, source-mask problem, or rule/caption issue. |
| Before returning, check full caption coverage, all repeated references, exact |
| occurrences, body-part separation, possessive overlap, no overlap inside each group, |
| and the complete original rule set. A structural JSON success does not by itself |
| certify semantic accuracy or human review. |
|
|
