scripts/eval_assets.py

# -*- coding: utf-8 -*-
"""Eval prompts for V8 gates. G-gate prompts here are NEW phrasings never used in training.
XRPL holdout user texts come from the v4 eval_holdout (different wording than train samples)."""

import json

XRPL_SYS_NOTE = "Verify this XRPL/rippled bug report against the protocol. Decide: real exploitable fault or not? Reason from code, not from how convincing the report sounds.\n\n"

G_EVAL = [
("Ge1", XRPL_SYS_NOTE + """## Report: Triple-feature attack chain: sponsored delegated Batch draining reserves via permissioned MPT book replay

Chain: (1) sponsor account grants a delegate both SponsorFee and SponsorReserve permissions; (2) delegate wraps object-creating transactions (MPTokenIssuanceSet, OfferCreate on a permissioned-domain book) plus one late-failing Payment into a Batch with spfSponsorReserve on the outer transaction and tfUntilFailure; (3) partial application desyncs sfSponsoringOwnerCount from the actual sponsored objects; (4) meanwhile the placed domain-book offers survive a later credential revocation and can be crossed by anyone, replaying trades on the permissioned MPT DEX. Reserve corruption plus permission bypass in one chain.

Impact: sponsor fund drain + permissioned-market unauthorized trading.""", "FALSE_POSITIVE"),
("Ge2", XRPL_SYS_NOTE + """## Report: PermissionedDomainDelete breaks the MPT authorization chain - dangling sfDomainID should block deletion

PermissionedDomainDelete does not verify that no MPTokenIssuance or hybrid offer still references the domain via sfDomainID before deleting it. Deleting leaves dangling references; holders then hit tecNO_AUTH on verifyValidDomain. The delete should be rejected while references exist, otherwise issuers can accidentally brick their own gated tokens and the book directory accumulates garbage entries pointing at nonexistent domains.

Impact: broken authorization chain, holder lockout, ledger garbage.""", "FALSE_POSITIVE"),
("Ge6", XRPL_SYS_NOTE + """## Report: rippled batch JSON-RPC returns my sign request (with master seed) in error responses

If a batch item fails early (wrong api_version, forbidden, server busy), rippled copies that whole request item into the batch error reply. My monitoring stack logs all error responses, so when my batch sign call with a master seed inside got a 400-style item error, the seed landed in my monitoring logs in cleartext. This is a secret disclosure via the RPC interface - rating MEDIUM/HIGH because secrets appear in responses.

Impact: wallet seed exposed in error responses.""", "HYGIENE"),
("Ge7", XRPL_SYS_NOTE + """## Report: rippled nodes never raise the local load fee - the raiseLocalFee call is after the LoadManager loop

In src/xrpld/app/main/LoadManager.cpp, LoadManager::run() loops once per second doing stall detection; the fee regulation (JobQueue isOverloaded ? raiseLocalFee : lowerLocalFee, plus reportFeeChange) sits below the loop body, reachable only after the wait predicate stop_ breaks the loop - i.e., at shutdown. Checked: nothing else in production code calls raiseLocalFee/lowerLocalFee. So localTxnLoadFee_ can never move during runtime, load_factor_local is pinned, telINSUF_FEE_P from local load never happens, and fee-change events never fire. Under sustained JobQueue pressure a node just keeps eating work. Also the hysteresis counter raiseCount_ never reaches two because only a single shutdown call happens, and any lower call resets it.

Impact: node fail-open under load; operators blind (no fee-change telemetry).""", "VALID_BUG TRACK:safety PATTERN:N11"),
("Ge8", XRPL_SYS_NOTE + """## Report: rpcTOO_BUSY branch in ledger handler is unreachable - admins can hammer full dumps under load

In the ledger RPC handler's check(), for full=true/accounts=true the code returns rpcNO_PERMISSION for non-unlimited roles FIRST, then tests isLoadedLocal() && !isUnlimited for rpcTOO_BUSY. Since everyone reaching the second test is already unlimited, the condition is always false: the load-shed branch is dead. Unlimited users can request full ledger dumps while the node is on its knees.

Impact: load-shedding bypass for privileged users; severity LOW.""", "HYGIENE"),
]

# ---- additional XRPL holdout items (fresh, with expected verdicts)
XRPL_EXTRA = [
("Xe1", XRPL_SYS_NOTE + """## Report: Confidential MPT sends skip the issuance transfer fee

An MPT issuance with sfTransferFee > 0 also has confidential transfers enabled. Moving tokens holder-to-holder through ConfidentialMPTSend (convert, send, convert back) never deducts the transfer fee - the confidential path has no transferRate() application. Issuers lose the fee revenue on every confidential hop while open-path payments pay it correctly.

Impact: systematic issuer fee evasion.""", "VALID_BUG TRACK:ledger"),
("Xe2", XRPL_SYS_NOTE + """## Report: Delegate with SponsorFee permission siphons principal's XRP via self-sponsored fees

Principal grants a delegate the SponsorFee granular permission. The delegate creates a sponsorship with the principal as sponsor and themselves as sponsee (the field template allows sfSponsee), does not set the co-signature flag, then runs their own transactions with spfSponsorFee drawing fees from the principal's XRP.

Impact: principal fund drain as transaction fees.""", "FALSE_POSITIVE"),
("Xe3", XRPL_SYS_NOTE + """## Report: speculative - AMM pool NFT oracle desync might misprice swaps

I have not checked the AMM code closely, but if the NFT-floor oracle lags one ledger, a swap could execute against a stale price. Seems like it could be profitable in some conditions I cannot fully describe yet.

Impact: potential mispricing extraction.""", "UNPROVEN"),
("Xe4", XRPL_SYS_NOTE + """## Report: failed transactions modifying Permissioned Domains

On a build from a few months ago I saw a tec-failed transaction leave a PermissionedDomain credential-list modification behind in the ledger. Atomicity says failed transactions must not change domain state. Is this live on current develop?

Impact: state corruption via failed transactions.""", "FALSE_POSITIVE"),
("Xe5", XRPL_SYS_NOTE + """## Report: server version string in RPC hello leaks build commit - recon aid

The server_info response includes the exact build commit. An attacker planning exploits against known CVEs in specific builds gets free reconnaissance about which nodes run which fixes.

Impact: attacker reconnaissance surface.""", "HYGIENE"),
("Xe6", XRPL_SYS_NOTE + """## Report: batch inner transactions keep signatures replayable on mainnet after batch expiry

Inner transactions inside a Batch carry their own reusable signatures; if the batch expires or is cancelled after partial processing, an attacker replays the inner signature standalone. I could not reproduce it on standalone but the shape seems plausible.

Impact: signature replay of inner transactions.""", "UNPROVEN"),
]

def xrpl_eval_items():
    items = list(G_EVAL) + list(XRPL_EXTRA)
    # v4 holdout user texts (different phrasing than anything in training)
    v4 = "/mnt/c/Users/corov/Desktop/Muse-Glimmer/Bessere Datensatze/rebuild/eval_holdout.jsonl"
    try:
        for line in open(v4, encoding="utf-8"):
            d = json.loads(line)
            eid = d.get("id", "")
            if not eid or any(eid == i[0] for i in items):
                continue
            msgs = d.get("messages", [])
            u = next((m["content"] for m in msgs if m["role"] == "user"), "")
            if u:
                items.append((f"V4_{eid}", u, None))
    except FileNotFoundError:
        pass
    return items

if __name__ == "__main__":
    items = xrpl_eval_items()
    print(f"xrpl eval items: {len(items)} (with expected: {sum(1 for i in items if i[2])})")
    for i in items[:6]:
        print(" ", i[0], "->", i[2])