adminspec commited on
Commit
3e94469
·
verified ·
1 Parent(s): ce2b49e

Release v0.1.1 real-patch evaluation tooling

Browse files

Add local exact-execution baseline tooling, source, contract, and accurate development-result documentation. Existing controller candidates, runtime, and examples are unchanged.

CHANGELOG.md CHANGED
@@ -1,5 +1,11 @@
1
  # Changelog
2
 
 
 
 
 
 
 
3
  ## 0.1.0 — 29 September 2026
4
 
5
  First public experimental controller package. Retains all four original September 15–16 candidates without selecting a winner or changing weights. Includes the negative development result, a smaller reviewed source closure, Apache 2.0 attribution, and a retained synthetic H1 installation/replay example.
 
1
  # Changelog
2
 
3
+ ## 0.1.1 — 1 October 2026
4
+
5
+ Adds a separate `evaluation/real-patch/` tooling component within the same Laxmi release: an exact visible-source baseline wheel, source, versioned input contract and design-only private API draft. One exposed development case is scored correctly by exact execution; five other historical cases remain oracle-only. No learned real-patch model, blind result, hosted API, Linux exact-execution qualification or case dataset is released.
6
+
7
+ All four controller candidate weights, metadata, runtime wheel/source ZIP and retained examples are byte-identical to v0.1.0. The controller experiment's failed repair and scientific scope are unchanged.
8
+
9
  ## 0.1.0 — 29 September 2026
10
 
11
  First public experimental controller package. Retains all four original September 15–16 candidates without selecting a winner or changing weights. Includes the negative development result, a smaller reviewed source closure, Apache 2.0 attribution, and a retained synthetic H1 installation/replay example.
CITATION.cff CHANGED
@@ -13,5 +13,5 @@ abstract: >-
13
  consequences from observed state, history, actions and proposed code changes.
14
  Controlled experiments and reproducibility infrastructure do not establish
15
  general software-engineering capability or production suitability.
16
- version: "0.1.0"
17
- date-released: 2026-09-29
 
13
  consequences from observed state, history, actions and proposed code changes.
14
  Controlled experiments and reproducibility infrastructure do not establish
15
  general software-engineering capability or production suitability.
16
+ version: "0.1.1"
17
+ date-released: 2026-10-01
README.md CHANGED
@@ -6,6 +6,7 @@ tags:
6
  - experimental
7
  - negative-results
8
  - custom-model
 
9
  ---
10
 
11
  # Laxmi — experimental software-state prediction
@@ -14,11 +15,11 @@ tags:
14
 
15
  Created and maintained by **Navneet Prabhakar**, creator and sole contributor to Laxmi. Third-party libraries retain their authorship and licenses.
16
 
17
- Laxmi investigates whether observed software state, history, an action and a proposed code change can predict what happens next. This first release makes a bounded controller experiment inspectable and runnable. It is not a chat model, a general code-review assistant or an autonomous software engineer.
18
 
19
- > **Version 0.1.0 · Experimental research release**
20
  >
21
- > The consumer works on the verified installation scope. The latest model repair failed its improvement test. All four candidates are retained; none is promoted as an accepted model.
22
 
23
  ## The experiment
24
 
@@ -47,14 +48,18 @@ Readout normalization did not improve the task. These are exposed development re
47
 
48
  The packaged example checks installation and replay. It must not be presented as a fresh benchmark. See [verification.json](verification.json) for the exact smoke scope and [BENCHMARK.md](BENCHMARK.md) for the next comparison design.
49
 
 
 
 
 
50
  ## Download and run
51
 
52
- **Verified platform: Apple Silicon macOS, Python 3.12.11.** The wheel is macOS ARM64 tagged. Other Python 3.12 patch builds and platforms have not been verified. The source archive is provided for inspection and experimentation; it does not establish Linux support. No GPU is required for the example.
53
 
54
  With the Hugging Face `hf` CLI installed:
55
 
56
  ```bash
57
- hf download Executespec/laxmi-controller-experimental --revision v0.1.0 --local-dir laxmi-release
58
  cd laxmi-release
59
  shasum -a 256 -c SHA256SUMS
60
  python3.12 -m venv .venv
@@ -66,12 +71,15 @@ Expected completion: `17-original: all six operations passed`. The output direct
66
 
67
  The wheel declares exact dependency versions, also recorded in `requirements-macos-arm64-py312.txt`. Dependencies are installed separately; they are not bundled. Version pins are not a dependency artifact hash lock. The source package includes the runtime source and license files. No optimizer checkpoint is shipped.
68
 
 
 
69
  ## Package layout
70
 
71
  | Path | Contents |
72
  |---|---|
73
  | `candidates/17-original/`, `17-normalized/`, `43-original/`, `43-normalized/` | Original retained safetensors weights, configuration, manifests and matching consumer metadata |
74
  | `runtime/` | Reviewed runtime wheel and source ZIP |
 
75
  | `examples/` | One retained synthetic observation, proposal comparison and outcome template |
76
  | `verify_release.py` | Installation/replay check using only this package |
77
  | `release.json`, `SHA256SUMS` | Candidate identity, runtime identity and file integrity |
@@ -90,6 +98,7 @@ Weights, configuration, readout metadata and runtime must match. Original and no
90
  - Longer rollouts may contain unresolved/pruned probability mass; it is not silently renormalized away.
91
  - Supported consumer horizons are 1, 2, 4 and 8; this release's installation smoke covers H1 only.
92
  - The learned probabilities are not proven calibrated and do not authorize external actions.
 
93
 
94
  ## Benchmark and iteration
95
 
 
6
  - experimental
7
  - negative-results
8
  - custom-model
9
+ - real-patch-evaluation
10
  ---
11
 
12
  # Laxmi — experimental software-state prediction
 
15
 
16
  Created and maintained by **Navneet Prabhakar**, creator and sole contributor to Laxmi. Third-party libraries retain their authorship and licenses.
17
 
18
+ Laxmi investigates whether observed software state, history, an action and a proposed code change can predict what happens next. This release keeps the bounded controller experiment inspectable and adds local real-patch evaluation tooling to the **same research project**. It is not a chat model, a general code-review assistant or an autonomous software engineer.
19
 
20
+ > **Version 0.1.1 · Experimental research release**
21
  >
22
+ > Controller weights and runtime are unchanged from v0.1.0. The latest model repair failed its improvement test. The new real-patch exact-execution tool is a baseline, **not a trained model**. None of the four controller candidates is promoted as an accepted model.
23
 
24
  ## The experiment
25
 
 
48
 
49
  The packaged example checks installation and replay. It must not be presented as a fresh benchmark. See [verification.json](verification.json) for the exact smoke scope and [BENCHMARK.md](BENCHMARK.md) for the next comparison design.
50
 
51
+ ### Real-patch development screen
52
+
53
+ This release also contains [local real-patch evaluation tooling](evaluation/real-patch/README.md). Six exposed historical changes were inventoried; only one currently has a valid forecast packet, and five have oracle-only scenarios excluded from forecast accuracy. On the one scored development case, the **same-information exact-execution baseline** matched both branch observations and the patch effect. The scenario was selected after its outcome was known. There are no blind/acceptance-eligible cases, no learned real-patch model scored, and no released case data. The result shows that this fully runnable case offers no measured headroom over exact execution; it does not validate generalization.
54
+
55
  ## Download and run
56
 
57
+ **Controller runtime verified platform: Apple Silicon macOS, Python 3.12.11.** That wheel is macOS ARM64 tagged. Other Python 3.12 patch builds and platforms have not been verified for the controller runtime. The source archive is provided for inspection and experimentation; it does not establish Linux controller support. No GPU is required for the example.
58
 
59
  With the Hugging Face `hf` CLI installed:
60
 
61
  ```bash
62
+ hf download Executespec/laxmi-controller-experimental --revision v0.1.1 --local-dir laxmi-release
63
  cd laxmi-release
64
  shasum -a 256 -c SHA256SUMS
65
  python3.12 -m venv .venv
 
71
 
72
  The wheel declares exact dependency versions, also recorded in `requirements-macos-arm64-py312.txt`. Dependencies are installed separately; they are not bundled. Version pins are not a dependency artifact hash lock. The source package includes the runtime source and license files. No optimizer checkpoint is shipped.
73
 
74
+ The separate real-patch package has its own [local installation instructions](evaluation/real-patch/README.md). It installed and **validated** its input in an offline Linux ARM64 container, but Linux exact execution and any hosted API remain unqualified. Its OpenAPI file is a design draft only.
75
+
76
  ## Package layout
77
 
78
  | Path | Contents |
79
  |---|---|
80
  | `candidates/17-original/`, `17-normalized/`, `43-original/`, `43-normalized/` | Original retained safetensors weights, configuration, manifests and matching consumer metadata |
81
  | `runtime/` | Reviewed runtime wheel and source ZIP |
82
+ | `evaluation/real-patch/` | Local exact-baseline evaluator wheel, source, contract and design-only API draft; no model weights or case data |
83
  | `examples/` | One retained synthetic observation, proposal comparison and outcome template |
84
  | `verify_release.py` | Installation/replay check using only this package |
85
  | `release.json`, `SHA256SUMS` | Candidate identity, runtime identity and file integrity |
 
98
  - Longer rollouts may contain unresolved/pruned probability mass; it is not silently renormalized away.
99
  - Supported consumer horizons are 1, 2, 4 and 8; this release's installation smoke covers H1 only.
100
  - The learned probabilities are not proven calibrated and do not authorize external actions.
101
+ - The real-patch exact comparator is a baseline, not learned prediction. Only one exposed development forecast was scored; no public API or Linux exact-execution qualification exists.
102
 
103
  ## Benchmark and iteration
104
 
RELEASE_SCOPE.md CHANGED
@@ -9,3 +9,9 @@ The runtime closure consists of original Laxmi source files extracted byte-for-b
9
  Dependencies such as PyTorch, NumPy, safetensors and jsonschema are referenced as installation requirements and are not bundled. Their licenses and copyright holders remain applicable; Laxmi does not claim their authorship. The source ZIP contains the package manifest, Python sources, required original configuration/schema, build files and Laxmi license/notice files.
10
 
11
  Repository-wide datasets and other artifacts outside this distribution receive no new redistribution grant from this release. Scientific model qualification remains open despite publication. License selection and distribution permission do not establish model accuracy or independence of evaluation.
 
 
 
 
 
 
 
9
  Dependencies such as PyTorch, NumPy, safetensors and jsonschema are referenced as installation requirements and are not bundled. Their licenses and copyright holders remain applicable; Laxmi does not claim their authorship. The source ZIP contains the package manifest, Python sources, required original configuration/schema, build files and Laxmi license/notice files.
10
 
11
  Repository-wide datasets and other artifacts outside this distribution receive no new redistribution grant from this release. Scientific model qualification remains open despite publication. License selection and distribution permission do not establish model accuracy or independence of evaluation.
12
+
13
+ ## v0.1.1 extension
14
+
15
+ The `evaluation/real-patch/` directory adds original Laxmi source, schema, documentation and a local wheel built from the source at commit `63cbecbad1bd8832d9c4fae65e4cfad27c57fa76`. These authored files are included under Apache 2.0, with exact file bytes enumerated by SHA256SUMS. The component contains no third-party repository code, real-patch case inputs, oracles, learned weights, corpus, private operational data or credentials. The public six-case counts and one-case exact-baseline result are summaries of exposed development evidence, not a distributed benchmark dataset.
16
+
17
+ The existing four controller candidates, matching runtime artifacts and synthetic example bytes are unchanged from v0.1.0. The new OpenAPI file describes a possible private service; no HTTP service is deployed or licensed as a maintained public endpoint. The wheel's Linux install/validation receipt does not establish Linux exact execution. Dependencies remain external with their own terms.
SHA256SUMS CHANGED
@@ -1,11 +1,11 @@
1
  78080e2966ee7383e24c55bf5ddd9e895fc80bdf9d2bb69de996e3d0b18f8cfb BENCHMARK.md
2
- 15d87975bbf97b56e3377d82f57e76e210d156ddcf26d34184e34990af8e6c7f CHANGELOG.md
3
- d5de9a7a59d6618ca67413f9a5991dfb8785fb2ac1c82de9b8be4e99a407337a CITATION.cff
4
  cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 LICENSE
5
  3a66b62901c108c4f8946247ff7a773f73ac3bcec0b8a34286eb921164e97d5f LICENSE_SCOPE.md
6
  3baa95979dcbe90444a854d04bbb6546c799e2077d9a69cfa34014a5c6512950 NOTICE
7
- 6a64cd7750ed0b39dbb3558ce1a74a41f509ab680ae3991b13c2bd0d5e101810 README.md
8
- 73b3b48727cd9b100cef2e20a9eb5feaf4d2ac8859bbb1b31d8e66bf5aae0d59 RELEASE_SCOPE.md
9
  225a71bb67d619e70f1aec0313d2c5791d0a94ac3942297ad12ba284269cddd8 candidates/17-normalized/consumer.json
10
  05478c5c7bf33bce2df7f7b1ffdffaa11dab3d6220558b1c2bb4d8432a5230e7 candidates/17-normalized/inference/config.json
11
  e12fd2cc04ad23ae6b1c8937652a56fc08cac040e56a3f411e3388da6b4eb6b5 candidates/17-normalized/inference/manifest.json
@@ -23,10 +23,23 @@ b01a8a82dc6c7881b439006d6e24a60a40e59a4416ff9719ad34611c205af87c candidates/43-
23
  d78f08c415e769d3e2676927e709c22d535a32ea5c79c6252ce0575af4abbcfd candidates/43-original/inference/manifest.json
24
  604d14c55256a7dd3c8f56072eda97fd7517912bd4d965d416d76040428261cb candidates/43-original/inference/model.safetensors
25
  ec1b954a497b276bb1806fbfcabde6e363f495f9dda60b1e686339a68bddd52d download-verification.json
 
 
 
 
 
 
 
 
 
 
 
 
 
26
  0d98c572630d61128ca07c4f92939597704ba8964d3f40f9d04bf60bab8f761f examples/compare.json
27
  2032524a11a31ff92634369060ff5a41adc6bc993161d6eb47a25a4839756ad1 examples/input.json
28
  b208c06bd5829a5b18521711433338fdf2500ddcf312658c9d0b788b0fc0873c examples/outcome-template.json
29
- d2552789b1d10160c7fd0446e6c5a4833f1816954e235b2edc2fc579e669ee9f release.json
30
  b3bb0efc5b695baf36d6959e650baaaf4dd5bd32c0d6189a355b05185d45ac80 requirements-macos-arm64-py312.txt
31
  67517f7c692d81580248e994c61829ddfdf2f9ef29358dd71a6b17eb953ef99f runtime/laxmi-readout-runtime-capsule-0.1.0-source.zip
32
  47d03da88fa9bd9c1b4573ac4a0d5cb36c2cd1d106d687b68aa2968b713ee9b3 runtime/laxmi_readout_runtime_capsule-0.1.0-py3-none-macosx_11_0_arm64.whl
 
1
  78080e2966ee7383e24c55bf5ddd9e895fc80bdf9d2bb69de996e3d0b18f8cfb BENCHMARK.md
2
+ f6e111f7762997f2a1310f08707a5af146615dab7b6f076d803dc92a517f1196 CHANGELOG.md
3
+ f42243d8ec7031d491961ebcbe70b70c9d410b1964f2080c73c4ab03bc9b01d8 CITATION.cff
4
  cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 LICENSE
5
  3a66b62901c108c4f8946247ff7a773f73ac3bcec0b8a34286eb921164e97d5f LICENSE_SCOPE.md
6
  3baa95979dcbe90444a854d04bbb6546c799e2077d9a69cfa34014a5c6512950 NOTICE
7
+ af5cdcd4819c7997c61a1f19f1dbff5a4b254547ae6395f88964e0b0e6d5c24a README.md
8
+ f7dbdd9349019149fc54ae8bcf3edf442d5b53a62bf0905b57b88a1e3379a48c RELEASE_SCOPE.md
9
  225a71bb67d619e70f1aec0313d2c5791d0a94ac3942297ad12ba284269cddd8 candidates/17-normalized/consumer.json
10
  05478c5c7bf33bce2df7f7b1ffdffaa11dab3d6220558b1c2bb4d8432a5230e7 candidates/17-normalized/inference/config.json
11
  e12fd2cc04ad23ae6b1c8937652a56fc08cac040e56a3f411e3388da6b4eb6b5 candidates/17-normalized/inference/manifest.json
 
23
  d78f08c415e769d3e2676927e709c22d535a32ea5c79c6252ce0575af4abbcfd candidates/43-original/inference/manifest.json
24
  604d14c55256a7dd3c8f56072eda97fd7517912bd4d965d416d76040428261cb candidates/43-original/inference/model.safetensors
25
  ec1b954a497b276bb1806fbfcabde6e363f495f9dda60b1e686339a68bddd52d download-verification.json
26
+ c9693a57855ca9af6ff2f0ab681d3f7902b164dccc93602e6d58d22e8eecda0e evaluation/real-patch/README.md
27
+ a36e136b730b9b941af303ca23dcdaaff0268b3e3dfafab4122730ee45d8da0e evaluation/real-patch/runtime/laxmi_real_patch-0.1.0.dev1-py3-none-any.whl
28
+ cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 evaluation/real-patch/source/LICENSE
29
+ 3a66b62901c108c4f8946247ff7a773f73ac3bcec0b8a34286eb921164e97d5f evaluation/real-patch/source/LICENSE_SCOPE.md
30
+ 3baa95979dcbe90444a854d04bbb6546c799e2077d9a69cfa34014a5c6512950 evaluation/real-patch/source/NOTICE
31
+ 77aa1eb34fb78016565afcce6a27bbe18561bea3652544ac50107e5c770b45cd evaluation/real-patch/source/code_review/contracts/real_patch_api_v1/openapi.yaml
32
+ 06a5bb4ca539bcb95ff97f58d2a003107b03a5af2a0b2fe21643d0c75590e1c0 evaluation/real-patch/source/code_review/contracts/real_patch_prediction_v1/README.md
33
+ ddd2f1a6f908334d4111971a14810fd5393d1735a00a39c3e745dd9d0f3d018c evaluation/real-patch/source/code_review/contracts/real_patch_prediction_v1/forecast_request.schema.json
34
+ 23b973a4fdd00e148254723cbcada5be02c65e80818cba6b1f0f3dbd42cc6a55 evaluation/real-patch/source/code_review/contracts/real_patch_prediction_v1/validator.py
35
+ ef8e9e728670c20a75dfc50a26649a073d4d04943791dc02a3b4a3e73af9f3e7 evaluation/real-patch/source/code_review/packaging/real_patch/api.py
36
+ a9a3cf605b0f34de4f0dcde1a9378682a3fd9149dc112152c3cf418aa2a9df4e evaluation/real-patch/source/code_review/scripts/build_real_patch_local_package_v1.py
37
+ 8538931b8044a98b1f245aaba4ec281b71a10e30944a46baf6ec34a4c659d9c8 evaluation/real-patch/source/code_review/scripts/run_real_patch_baseline_gate_v2.py
38
+ 6344b680488213d222cca267f47ed54dcfecb0b1b4990033b4785d808ee54e3a evaluation/real-patch/source/code_review/scripts/run_real_patch_visible_exact_v1.py
39
  0d98c572630d61128ca07c4f92939597704ba8964d3f40f9d04bf60bab8f761f examples/compare.json
40
  2032524a11a31ff92634369060ff5a41adc6bc993161d6eb47a25a4839756ad1 examples/input.json
41
  b208c06bd5829a5b18521711433338fdf2500ddcf312658c9d0b788b0fc0873c examples/outcome-template.json
42
+ 0e02068ddc38d1f32f79c2003649695cf807857843784ebba179884e64ae8a69 release.json
43
  b3bb0efc5b695baf36d6959e650baaaf4dd5bd32c0d6189a355b05185d45ac80 requirements-macos-arm64-py312.txt
44
  67517f7c692d81580248e994c61829ddfdf2f9ef29358dd71a6b17eb953ef99f runtime/laxmi-readout-runtime-capsule-0.1.0-source.zip
45
  47d03da88fa9bd9c1b4573ac4a0d5cb36c2cd1d106d687b68aa2968b713ee9b3 runtime/laxmi_readout_runtime_capsule-0.1.0-py3-none-macosx_11_0_arm64.whl
evaluation/real-patch/README.md ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Real-patch development evaluation tooling
2
+
3
+ This directory is part of **Laxmi experimental release v0.1.1**, maintained by Navneet Prabhakar. It is a tooling extension to the same Laxmi research project, not another model. The controller model weights and runtime remain byte-identical to v0.1.0. This directory contains **no learned real-patch weights**, user source, candidate data or oracle records.
4
+
5
+ The local package validates a paired pre-execution forecast request, executes the unchanged and proposed-patch branches from the **same visible source, dependency and scenario bytes**, seals the prediction, and scores only through a separately invoked oracle operation. Exact execution is a baseline. Its success must not be credited to a learned world model.
6
+
7
+ ## Current evaluation
8
+
9
+ Six exposed historical changes are inventoried. Only one has a forecast packet under the current contract; the other five have oracle-only scenarios mostly adapted from post-change tests and are excluded from forecast accuracy. On that one exposed development case, exact execution matched both branch observations and the change effect. There are zero blind/acceptance-eligible cases and no learned real-patch candidate scored. The selected scenario was projected after the outcome was known. No evaluation case data is distributed in this release.
10
+
11
+ ## Local installation
12
+
13
+ From the downloaded release root in a fresh Python 3.12 environment:
14
+
15
+ ```bash
16
+ python3.12 -m venv .venv-real-patch
17
+ .venv-real-patch/bin/python -m pip install evaluation/real-patch/runtime/laxmi_real_patch-0.1.0.dev1-py3-none-any.whl
18
+ .venv-real-patch/bin/laxmi-real-patch validate --request /path/to/forecast_request.json
19
+ ```
20
+
21
+ The [request contract](source/code_review/contracts/real_patch_prediction_v1/README.md) and [schema](source/code_review/contracts/real_patch_prediction_v1/forecast_request.schema.json) specify model-visible input. A request must have independently reviewed source/scenario provenance; passing schema validation cannot establish truth of self-declared timestamps or acceptance eligibility. The installed interface explicitly reports `custody_verified=false`, `acceptance_eligible=false` and `development_only=true`.
22
+
23
+ `forecast` requires a **new** output directory and a suitable local Docker runtime with `python:3.12-slim` resolving to image ID `sha256:78387bc3881b8273120a12ebe6c1ab22b018ccc2c9adf565ae1ac9b536e184ea`:
24
+
25
+ ```bash
26
+ .venv-real-patch/bin/laxmi-real-patch forecast --request /path/to/forecast_request.json --output-dir ./forecast-001
27
+ ```
28
+
29
+ The comparator does not read oracles. A separately authorized evaluator can later score a sealed prediction:
30
+
31
+ ```bash
32
+ .venv-real-patch/bin/laxmi-real-patch score --request /path/to/forecast_request.json --oracle /path/to/gate-oracle.json --output-dir ./forecast-001
33
+ ```
34
+
35
+ The wheel SHA-256 is `a36e136b730b9b941af303ca23dcdaaff0268b3e3dfafab4122730ee45d8da0e`. A fresh macOS ARM64 Python 3.12 install reproduced the original prediction and score receipt bytes. Offline Linux ARM64 installation and **validation** returned the same request hash; Linux exact execution was not qualified. The wheel tag `py3-none-any` is packaging metadata, not a blanket portability claim. The exact comparator executes supplied scenario code in an isolated local Docker container; do not expose this CLI as a public code-execution service.
36
+
37
+ The [OpenAPI document](source/code_review/contracts/real_patch_api_v1/openapi.yaml) is **design-only**. There is no hosted API, authenticated job service or production deployment. Source files and the builder are provided under [`source/`](source/) for inspection; build the wheel from that directory with `python code_review/scripts/build_real_patch_local_package_v1.py --output-dir /new/output`. The root [LICENSE](../../LICENSE), [NOTICE](../../NOTICE), [RELEASE_SCOPE.md](../../RELEASE_SCOPE.md) and [SHA256SUMS](../../SHA256SUMS) govern this release.
evaluation/real-patch/runtime/laxmi_real_patch-0.1.0.dev1-py3-none-any.whl ADDED
Binary file (23.1 kB). View file
 
evaluation/real-patch/source/LICENSE ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ Apache License
3
+ Version 2.0, January 2004
4
+ http://www.apache.org/licenses/
5
+
6
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7
+
8
+ 1. Definitions.
9
+
10
+ "License" shall mean the terms and conditions for use, reproduction,
11
+ and distribution as defined by Sections 1 through 9 of this document.
12
+
13
+ "Licensor" shall mean the copyright owner or entity authorized by
14
+ the copyright owner that is granting the License.
15
+
16
+ "Legal Entity" shall mean the union of the acting entity and all
17
+ other entities that control, are controlled by, or are under common
18
+ control with that entity. For the purposes of this definition,
19
+ "control" means (i) the power, direct or indirect, to cause the
20
+ direction or management of such entity, whether by contract or
21
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
22
+ outstanding shares, or (iii) beneficial ownership of such entity.
23
+
24
+ "You" (or "Your") shall mean an individual or Legal Entity
25
+ exercising permissions granted by this License.
26
+
27
+ "Source" form shall mean the preferred form for making modifications,
28
+ including but not limited to software source code, documentation
29
+ source, and configuration files.
30
+
31
+ "Object" form shall mean any form resulting from mechanical
32
+ transformation or translation of a Source form, including but
33
+ not limited to compiled object code, generated documentation,
34
+ and conversions to other media types.
35
+
36
+ "Work" shall mean the work of authorship, whether in Source or
37
+ Object form, made available under the License, as indicated by a
38
+ copyright notice that is included in or attached to the work
39
+ (an example is provided in the Appendix below).
40
+
41
+ "Derivative Works" shall mean any work, whether in Source or Object
42
+ form, that is based on (or derived from) the Work and for which the
43
+ editorial revisions, annotations, elaborations, or other modifications
44
+ represent, as a whole, an original work of authorship. For the purposes
45
+ of this License, Derivative Works shall not include works that remain
46
+ separable from, or merely link (or bind by name) to the interfaces of,
47
+ the Work and Derivative Works thereof.
48
+
49
+ "Contribution" shall mean any work of authorship, including
50
+ the original version of the Work and any modifications or additions
51
+ to that Work or Derivative Works thereof, that is intentionally
52
+ submitted to Licensor for inclusion in the Work by the copyright owner
53
+ or by an individual or Legal Entity authorized to submit on behalf of
54
+ the copyright owner. For the purposes of this definition, "submitted"
55
+ means any form of electronic, verbal, or written communication sent
56
+ to the Licensor or its representatives, including but not limited to
57
+ communication on electronic mailing lists, source code control systems,
58
+ and issue tracking systems that are managed by, or on behalf of, the
59
+ Licensor for the purpose of discussing and improving the Work, but
60
+ excluding communication that is conspicuously marked or otherwise
61
+ designated in writing by the copyright owner as "Not a Contribution."
62
+
63
+ "Contributor" shall mean Licensor and any individual or Legal Entity
64
+ on behalf of whom a Contribution has been received by Licensor and
65
+ subsequently incorporated within the Work.
66
+
67
+ 2. Grant of Copyright License. Subject to the terms and conditions of
68
+ this License, each Contributor hereby grants to You a perpetual,
69
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70
+ copyright license to reproduce, prepare Derivative Works of,
71
+ publicly display, publicly perform, sublicense, and distribute the
72
+ Work and such Derivative Works in Source or Object form.
73
+
74
+ 3. Grant of Patent License. Subject to the terms and conditions of
75
+ this License, each Contributor hereby grants to You a perpetual,
76
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77
+ (except as stated in this section) patent license to make, have made,
78
+ use, offer to sell, sell, import, and otherwise transfer the Work,
79
+ where such license applies only to those patent claims licensable
80
+ by such Contributor that are necessarily infringed by their
81
+ Contribution(s) alone or by combination of their Contribution(s)
82
+ with the Work to which such Contribution(s) was submitted. If You
83
+ institute patent litigation against any entity (including a
84
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
85
+ or a Contribution incorporated within the Work constitutes direct
86
+ or contributory patent infringement, then any patent licenses
87
+ granted to You under this License for that Work shall terminate
88
+ as of the date such litigation is filed.
89
+
90
+ 4. Redistribution. You may reproduce and distribute copies of the
91
+ Work or Derivative Works thereof in any medium, with or without
92
+ modifications, and in Source or Object form, provided that You
93
+ meet the following conditions:
94
+
95
+ (a) You must give any other recipients of the Work or
96
+ Derivative Works a copy of this License; and
97
+
98
+ (b) You must cause any modified files to carry prominent notices
99
+ stating that You changed the files; and
100
+
101
+ (c) You must retain, in the Source form of any Derivative Works
102
+ that You distribute, all copyright, patent, trademark, and
103
+ attribution notices from the Source form of the Work,
104
+ excluding those notices that do not pertain to any part of
105
+ the Derivative Works; and
106
+
107
+ (d) If the Work includes a "NOTICE" text file as part of its
108
+ distribution, then any Derivative Works that You distribute must
109
+ include a readable copy of the attribution notices contained
110
+ within such NOTICE file, excluding those notices that do not
111
+ pertain to any part of the Derivative Works, in at least one
112
+ of the following places: within a NOTICE text file distributed
113
+ as part of the Derivative Works; within the Source form or
114
+ documentation, if provided along with the Derivative Works; or,
115
+ within a display generated by the Derivative Works, if and
116
+ wherever such third-party notices normally appear. The contents
117
+ of the NOTICE file are for informational purposes only and
118
+ do not modify the License. You may add Your own attribution
119
+ notices within Derivative Works that You distribute, alongside
120
+ or as an addendum to the NOTICE text from the Work, provided
121
+ that such additional attribution notices cannot be construed
122
+ as modifying the License.
123
+
124
+ You may add Your own copyright statement to Your modifications and
125
+ may provide additional or different license terms and conditions
126
+ for use, reproduction, or distribution of Your modifications, or
127
+ for any such Derivative Works as a whole, provided Your use,
128
+ reproduction, and distribution of the Work otherwise complies with
129
+ the conditions stated in this License.
130
+
131
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
132
+ any Contribution intentionally submitted for inclusion in the Work
133
+ by You to the Licensor shall be under the terms and conditions of
134
+ this License, without any additional terms or conditions.
135
+ Notwithstanding the above, nothing herein shall supersede or modify
136
+ the terms of any separate license agreement you may have executed
137
+ with Licensor regarding such Contributions.
138
+
139
+ 6. Trademarks. This License does not grant permission to use the trade
140
+ names, trademarks, service marks, or product names of the Licensor,
141
+ except as required for reasonable and customary use in describing the
142
+ origin of the Work and reproducing the content of the NOTICE file.
143
+
144
+ 7. Disclaimer of Warranty. Unless required by applicable law or
145
+ agreed to in writing, Licensor provides the Work (and each
146
+ Contributor provides its Contributions) on an "AS IS" BASIS,
147
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148
+ implied, including, without limitation, any warranties or conditions
149
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150
+ PARTICULAR PURPOSE. You are solely responsible for determining the
151
+ appropriateness of using or redistributing the Work and assume any
152
+ risks associated with Your exercise of permissions under this License.
153
+
154
+ 8. Limitation of Liability. In no event and under no legal theory,
155
+ whether in tort (including negligence), contract, or otherwise,
156
+ unless required by applicable law (such as deliberate and grossly
157
+ negligent acts) or agreed to in writing, shall any Contributor be
158
+ liable to You for damages, including any direct, indirect, special,
159
+ incidental, or consequential damages of any character arising as a
160
+ result of this License or out of the use or inability to use the
161
+ Work (including but not limited to damages for loss of goodwill,
162
+ work stoppage, computer failure or malfunction, or any and all
163
+ other commercial damages or losses), even if such Contributor
164
+ has been advised of the possibility of such damages.
165
+
166
+ 9. Accepting Warranty or Additional Liability. While redistributing
167
+ the Work or Derivative Works thereof, You may choose to offer,
168
+ and charge a fee for, acceptance of support, warranty, indemnity,
169
+ or other liability obligations and/or rights consistent with this
170
+ License. However, in accepting such obligations, You may act only
171
+ on Your own behalf and on Your sole responsibility, not on behalf
172
+ of any other Contributor, and only if You agree to indemnify,
173
+ defend, and hold each Contributor harmless for any liability
174
+ incurred by, or claims asserted against, such Contributor by reason
175
+ of your accepting any such warranty or additional liability.
176
+
177
+ END OF TERMS AND CONDITIONS
178
+
179
+ APPENDIX: How to apply the Apache License to your work.
180
+
181
+ To apply the Apache License to your work, attach the following
182
+ boilerplate notice, with the fields enclosed by brackets "[]"
183
+ replaced with your own identifying information. (Don't include
184
+ the brackets!) The text should be enclosed in the appropriate
185
+ comment syntax for the file format. We also recommend that a
186
+ file or class name and description of purpose be included on the
187
+ same "printed page" as the copyright notice for easier
188
+ identification within third-party archives.
189
+
190
+ Copyright [yyyy] [name of copyright owner]
191
+
192
+ Licensed under the Apache License, Version 2.0 (the "License");
193
+ you may not use this file except in compliance with the License.
194
+ You may obtain a copy of the License at
195
+
196
+ http://www.apache.org/licenses/LICENSE-2.0
197
+
198
+ Unless required by applicable law or agreed to in writing, software
199
+ distributed under the License is distributed on an "AS IS" BASIS,
200
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201
+ See the License for the specific language governing permissions and
202
+ limitations under the License.
evaluation/real-patch/source/LICENSE_SCOPE.md ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Licensing scope
2
+
3
+ Copyright 2026 Navneet Prabhakar.
4
+
5
+ Navneet Prabhakar's original Laxmi source code, documentation, configuration,
6
+ schemas and authored test fixtures are licensed under Apache License 2.0,
7
+ unless an individual file explicitly states different terms. LICENSE contains
8
+ the unmodified license text; NOTICE records project attribution. Future accepted
9
+ contributions follow the license's contribution terms unless separately agreed.
10
+
11
+ Apache 2.0 is also the selected license for original Laxmi model weights shipped
12
+ in an explicitly identified release after provenance and redistribution review.
13
+ Every model release must identify its exact weights and applicable terms. The
14
+ presence of a local checkpoint does not alone establish its release eligibility.
15
+
16
+ The repository license does not relicense third-party source, dependencies,
17
+ models, tokenizers, copied documents or other third-party material. Their existing
18
+ terms and required notices remain applicable. Incorporating or linking a dependency
19
+ does not claim its authorship or change its license.
20
+
21
+ Acquired corpora, datasets, repository clones, training/evaluation records,
22
+ operational archives and generated evidence under data/, results/, archive/ and
23
+ the corresponding code_review/ directories are not granted redistribution rights
24
+ by this repository license alone. Individual released datasets or artifact bundles
25
+ require an explicit scope manifest and their applicable terms. Original source or
26
+ documentation in those directories may be expressly included in such a manifest.
27
+ Private inputs, credentials and operational records are not public release assets.
28
+
29
+ ## Attribution and citation
30
+
31
+ Redistributors must comply with Apache 2.0's license and applicable notice
32
+ preservation requirements. The NOTICE file provides Laxmi's attribution. It adds
33
+ no requirement to display a badge, advertise the author or obtain permission for
34
+ permitted uses. CITATION.cff makes a requested research citation convenient;
35
+ citation is not an additional condition of the license.
36
+
37
+ Navneet Prabhakar remains Laxmi's original creator. Contributor lists should
38
+ reflect later accepted contributions if the project gains other contributors.
evaluation/real-patch/source/NOTICE ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Laxmi
2
+ Copyright 2026 Navneet Prabhakar
3
+
4
+ Created and maintained by Navneet Prabhakar.
5
+ Project: https://github.com/navneetprabhakar/laxmi
6
+
7
+ Laxmi's original contributions are licensed under the Apache License,
8
+ Version 2.0. See LICENSE and LICENSE_SCOPE.md for applicable scope.
9
+
10
+ Third-party libraries, models and other materials retain their respective
11
+ copyrights, licenses and attribution requirements. This notice does not
12
+ claim authorship of those materials or imply their authors' endorsement.
evaluation/real-patch/source/code_review/contracts/real_patch_api_v1/openapi.yaml ADDED
@@ -0,0 +1,251 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ openapi: 3.1.0
2
+ info:
3
+ title: Laxmi real-patch private API
4
+ version: 0.1.0-design
5
+ description: >-
6
+ Design-only asynchronous transport over the real-patch forecast contract.
7
+ No hosted implementation, learned real-patch model, or public endpoint exists.
8
+ x-implementation-status: DESIGN_ONLY
9
+ paths:
10
+ /v1/real-patch/requests:validate:
11
+ post:
12
+ operationId: validateRealPatchRequest
13
+ summary: Validate model-visible input without execution or oracle access
14
+ security:
15
+ - bearerAuth: []
16
+ requestBody:
17
+ required: true
18
+ content:
19
+ application/json:
20
+ schema:
21
+ $ref: ../real_patch_prediction_v1/forecast_request.schema.json
22
+ responses:
23
+ '200':
24
+ description: Structural admission only; not proof of external provenance
25
+ content:
26
+ application/json:
27
+ schema:
28
+ $ref: '#/components/schemas/ValidationResult'
29
+ '422':
30
+ $ref: '#/components/responses/Refusal'
31
+ /v1/real-patch/forecast-jobs:
32
+ post:
33
+ operationId: submitRealPatchForecast
34
+ summary: Submit a sealed-before-outcome forecast job
35
+ security:
36
+ - bearerAuth: []
37
+ parameters:
38
+ - name: Idempotency-Key
39
+ in: header
40
+ required: true
41
+ schema:
42
+ type: string
43
+ minLength: 16
44
+ maxLength: 128
45
+ requestBody:
46
+ required: true
47
+ content:
48
+ application/json:
49
+ schema:
50
+ type: object
51
+ additionalProperties: false
52
+ required: [request, engine_id, engine_version]
53
+ properties:
54
+ request:
55
+ $ref: ../real_patch_prediction_v1/forecast_request.schema.json
56
+ engine_id:
57
+ type: string
58
+ enum: [EXACT_VISIBLE_SOURCE, STATIC_RULES, RETRIEVAL, LEARNED]
59
+ engine_version:
60
+ type: string
61
+ responses:
62
+ '202':
63
+ description: Create-only job accepted; no prediction is implied
64
+ content:
65
+ application/json:
66
+ schema:
67
+ $ref: '#/components/schemas/JobStatus'
68
+ '409':
69
+ $ref: '#/components/responses/Refusal'
70
+ '422':
71
+ $ref: '#/components/responses/Refusal'
72
+ /v1/real-patch/forecast-jobs/{id}:
73
+ get:
74
+ operationId: getRealPatchForecastJob
75
+ security:
76
+ - bearerAuth: []
77
+ parameters:
78
+ - $ref: '#/components/parameters/JobId'
79
+ responses:
80
+ '200':
81
+ description: Job state, including unsupported and failed states
82
+ content:
83
+ application/json:
84
+ schema:
85
+ $ref: '#/components/schemas/JobStatus'
86
+ /v1/real-patch/forecast-jobs/{id}/prediction:
87
+ get:
88
+ operationId: getSealedRealPatchPrediction
89
+ security:
90
+ - bearerAuth: []
91
+ parameters:
92
+ - $ref: '#/components/parameters/JobId'
93
+ responses:
94
+ '200':
95
+ description: Prediction receipt without oracle or score
96
+ content:
97
+ application/json:
98
+ schema:
99
+ $ref: '#/components/schemas/PredictionResult'
100
+ '409':
101
+ $ref: '#/components/responses/Refusal'
102
+ /v1/real-patch/outcomes:
103
+ post:
104
+ operationId: attachRealPatchOutcome
105
+ summary: Privileged evaluator attaches observed outcome after forecast seal
106
+ security:
107
+ - evaluatorAuth: []
108
+ requestBody:
109
+ required: true
110
+ content:
111
+ application/json:
112
+ schema:
113
+ type: object
114
+ additionalProperties: false
115
+ required: [job_id, oracle_sha256, oracle_ref]
116
+ properties:
117
+ job_id:
118
+ type: string
119
+ oracle_sha256:
120
+ $ref: '#/components/schemas/Sha256'
121
+ oracle_ref:
122
+ type: string
123
+ description: Evaluator-only immutable artifact reference
124
+ responses:
125
+ '201':
126
+ description: Outcome attached immutably after sealed prediction
127
+ '409':
128
+ $ref: '#/components/responses/Refusal'
129
+ /v1/real-patch/evaluations/{id}:
130
+ get:
131
+ operationId: getRealPatchEvaluation
132
+ summary: Retrieve an evaluator-only score and eligibility receipt
133
+ security:
134
+ - evaluatorAuth: []
135
+ parameters:
136
+ - $ref: '#/components/parameters/JobId'
137
+ responses:
138
+ '200':
139
+ description: Scored result with the attempted-case denominator
140
+ content:
141
+ application/json:
142
+ schema:
143
+ $ref: '#/components/schemas/EvaluationResult'
144
+ components:
145
+ securitySchemes:
146
+ bearerAuth:
147
+ type: http
148
+ scheme: bearer
149
+ evaluatorAuth:
150
+ type: http
151
+ scheme: bearer
152
+ description: Separate evaluator principal with outcome-storage access
153
+ parameters:
154
+ JobId:
155
+ name: id
156
+ in: path
157
+ required: true
158
+ schema:
159
+ type: string
160
+ schemas:
161
+ Sha256:
162
+ type: string
163
+ pattern: '^[0-9a-f]{64}$'
164
+ ValidationResult:
165
+ type: object
166
+ additionalProperties: false
167
+ required: [status, input_sha256, custody_verified, acceptance_eligible]
168
+ properties:
169
+ status:
170
+ const: ADMITTED
171
+ input_sha256:
172
+ $ref: '#/components/schemas/Sha256'
173
+ custody_verified:
174
+ const: false
175
+ acceptance_eligible:
176
+ const: false
177
+ JobStatus:
178
+ type: object
179
+ additionalProperties: false
180
+ required: [job_id, status, input_sha256, engine_id, engine_version]
181
+ properties:
182
+ job_id:
183
+ type: string
184
+ status:
185
+ enum: [QUEUED, RUNNING, PREDICTED, UNSUPPORTED, FAILED]
186
+ input_sha256:
187
+ $ref: '#/components/schemas/Sha256'
188
+ engine_id:
189
+ type: string
190
+ engine_version:
191
+ type: string
192
+ reason_code:
193
+ type: string
194
+ PredictionResult:
195
+ type: object
196
+ additionalProperties: false
197
+ required: [job_id, status, input_sha256, engine_id, method_sha256, information_tier, observations, seal_sha256]
198
+ properties:
199
+ job_id:
200
+ type: string
201
+ status:
202
+ const: PREDICTED
203
+ input_sha256:
204
+ $ref: '#/components/schemas/Sha256'
205
+ engine_id:
206
+ type: string
207
+ method_sha256:
208
+ $ref: '#/components/schemas/Sha256'
209
+ information_tier:
210
+ enum: [SAME_INFORMATION, PRIVILEGED_CEILING]
211
+ observations:
212
+ type: object
213
+ description: Exactly the declared branch fields from the validated request
214
+ seal_sha256:
215
+ $ref: '#/components/schemas/Sha256'
216
+ EvaluationResult:
217
+ type: object
218
+ additionalProperties: false
219
+ required: [evaluation_id, evaluation_split, dataset_sha256, candidate_count, scored_case_count, unsupported_case_count, receipt_sha256]
220
+ properties:
221
+ evaluation_id:
222
+ type: string
223
+ evaluation_split:
224
+ enum: [DEVELOPMENT_EXPOSED, ACCEPTANCE_CANDIDATE]
225
+ dataset_sha256:
226
+ $ref: '#/components/schemas/Sha256'
227
+ candidate_count:
228
+ type: integer
229
+ minimum: 1
230
+ scored_case_count:
231
+ type: integer
232
+ minimum: 0
233
+ unsupported_case_count:
234
+ type: integer
235
+ minimum: 0
236
+ receipt_sha256:
237
+ $ref: '#/components/schemas/Sha256'
238
+ responses:
239
+ Refusal:
240
+ description: Request refused without a prediction
241
+ content:
242
+ application/json:
243
+ schema:
244
+ type: object
245
+ additionalProperties: false
246
+ required: [error, reason_code]
247
+ properties:
248
+ error:
249
+ type: string
250
+ reason_code:
251
+ type: string
evaluation/real-patch/source/code_review/contracts/real_patch_prediction_v1/README.md ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Real-patch forecast input v1
2
+
3
+ `forecast_request.schema.json` defines the **entire model-visible request** for a
4
+ pre-execution forecast. `validator.py` additionally checks content hashes,
5
+ chronology, source/patch lineage, duplicate identities and patch file roles.
6
+ All consumers receive the same canonical request bytes; `shared_input.input_sha256`
7
+ is SHA-256 of canonical JSON excluding `shared_input` itself. Canonical JSON uses
8
+ sorted keys, compact separators, UTF-8, and no non-finite numbers. A comparator
9
+ may report `unsupported`, but may not substitute more informative material.
10
+
11
+ The request contains an opaque repository alias and a hash of the visible source
12
+ file manifest. Explicit repository URL, Git commit, issue locator, selection notes,
13
+ PR metadata, oracle code/results and any future test metadata live in a **separate custody
14
+ record** that must never be passed to a forecast consumer. Pinned source bytes
15
+ may themselves contain public repository identifiers; opaque metadata does not
16
+ remove this contamination risk. The visible proposed
17
+ diff may change production files only; tests and fixtures are rejected. Scenario
18
+ input is structured as entrypoint plus JSON arguments, with an artifact hash and
19
+ availability timestamp. The same sanitized Python `program` bytes are visible
20
+ to every comparator and bind the exact runner; an AST check rejects assertions
21
+ and obvious expected-value variables. There is no field for issue prose,
22
+ expected values, diagnosis, PR title/body, or post-patch result. Pre-patch history can
23
+ include observed outcomes on the pinned base source only.
24
+
25
+ The schema cannot prove that a timestamp is true, that a source slice really
26
+ comes from the claimed immutable snapshot, or that a string argument encodes no
27
+ answer. An assembler must verify external artifact timestamps, version history,
28
+ licenses, patch provenance and scenario selection against the custody record
29
+ before signing a case. A reviewer must inspect each assembled packet for semantic
30
+ leakage. Historical public code may still be memorized by a pretrained model;
31
+ opaque IDs reduce direct lookup, but do not establish contamination freedom.
32
+
33
+ `target` requires paired `NO_CHANGE` and `PROPOSED_PATCH` branches under the
34
+ same invocation and bounded completion horizon. It declares observable fields
35
+ and the categorical vocabulary, never their values. The sealed oracle and
36
+ predictor outputs are separate files. This contract
37
+ does not itself establish that a learned predictor beats exact execution, static
38
+ rules or retrieval.
evaluation/real-patch/source/code_review/contracts/real_patch_prediction_v1/forecast_request.schema.json ADDED
@@ -0,0 +1,137 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "$schema": "https://json-schema.org/draft/2020-12/schema",
3
+ "$id": "https://laxmi.local/contracts/real-patch-prediction/v1/forecast-request.schema.json",
4
+ "title": "Real patch pre-execution forecast request v1",
5
+ "description": "Model-visible information only. Oracle, PR-added tests, expected assertions, diagnoses and post-patch outcomes belong in separate sealed records.",
6
+ "type": "object",
7
+ "additionalProperties": false,
8
+ "required": ["schema_version", "case_id", "evaluation_split", "forecast_cutoff_at", "pre_source", "proposed_patch", "scenario", "operating_conditions", "history", "target", "shared_input"],
9
+ "properties": {
10
+ "schema_version": {"const": "1.0.0"},
11
+ "case_id": {"$ref": "#/$defs/id"},
12
+ "evaluation_split": {"enum": ["DEVELOPMENT_EXPOSED", "ACCEPTANCE_CANDIDATE"]},
13
+ "forecast_cutoff_at": {"$ref": "#/$defs/timestamp"},
14
+ "pre_source": {
15
+ "type": "object", "additionalProperties": false,
16
+ "required": ["repository_alias", "snapshot_sha256", "snapshot_at", "files"],
17
+ "properties": {
18
+ "repository_alias": {"$ref": "#/$defs/id"},
19
+ "snapshot_sha256": {"$ref": "#/$defs/sha256"},
20
+ "snapshot_at": {"$ref": "#/$defs/timestamp"},
21
+ "files": {"type": "array", "minItems": 1, "items": {"$ref": "#/$defs/sourceFile"}}
22
+ }
23
+ },
24
+ "proposed_patch": {
25
+ "type": "object", "additionalProperties": false,
26
+ "required": ["base_snapshot_sha256", "diff_sha256", "unified_diff", "proposed_at", "provenance"],
27
+ "properties": {
28
+ "base_snapshot_sha256": {"$ref": "#/$defs/sha256"},
29
+ "diff_sha256": {"$ref": "#/$defs/sha256"},
30
+ "unified_diff": {"type": "string", "minLength": 1},
31
+ "proposed_at": {"$ref": "#/$defs/timestamp"},
32
+ "provenance": {"const": "CANDIDATE_PATCH_AT_FORECAST"}
33
+ }
34
+ },
35
+ "scenario": {
36
+ "type": "object", "additionalProperties": false,
37
+ "required": ["origin", "available_at", "source_artifact_sha256", "selection_policy", "projection_policy", "program", "program_sha256", "setup_assignments", "invocations"],
38
+ "properties": {
39
+ "origin": {"enum": ["PRE_PATCH_WORKLOAD_TRACE", "PRE_PATCH_ISSUE_REPRO", "PRE_PATCH_TEST", "PROSPECTIVE_GENERATOR"]},
40
+ "available_at": {"$ref": "#/$defs/timestamp"},
41
+ "source_artifact_sha256": {"$ref": "#/$defs/sha256"},
42
+ "selection_policy": {"enum": ["FROZEN_BEFORE_OUTCOME", "RETROSPECTIVE_EXPOSED_DEVELOPMENT"]},
43
+ "projection_policy": {"const": "PRE_PATCH_INPUTS_WITHOUT_ASSERTIONS_OR_EXPECTED_OUTPUTS"},
44
+ "program": {"type": "string", "minLength": 1},
45
+ "program_sha256": {"$ref": "#/$defs/sha256"},
46
+ "setup_assignments": {"type": "array", "items": {"$ref": "#/$defs/assignment"}},
47
+ "invocations": {"type": "array", "minItems": 1, "items": {"$ref": "#/$defs/invocation"}}
48
+ }
49
+ },
50
+ "operating_conditions": {
51
+ "type": "object", "additionalProperties": false,
52
+ "required": ["python_version", "platform", "dependency_lock_sha256", "dependency_sources", "environment"],
53
+ "properties": {
54
+ "python_version": {"type": "string", "pattern": "^3\\.[0-9]+(?:\\.[0-9]+)?$"},
55
+ "platform": {"type": "string", "minLength": 1, "maxLength": 80},
56
+ "dependency_lock_sha256": {"$ref": "#/$defs/sha256"},
57
+ "dependency_sources": {"type": "array", "items": {"$ref": "#/$defs/sourceFile"}},
58
+ "environment": {"type": "object", "propertyNames": {"pattern": "^[A-Z][A-Z0-9_]*$"}, "additionalProperties": {"type": "string", "maxLength": 1024}}
59
+ }
60
+ },
61
+ "history": {"type": "array", "items": {"$ref": "#/$defs/historyEvent"}},
62
+ "target": {
63
+ "type": "object", "additionalProperties": false,
64
+ "required": ["kind", "branches", "horizon", "observation_classes", "observe"],
65
+ "properties": {
66
+ "kind": {"const": "PAIRED_INTERVENTION"},
67
+ "branches": {"const": ["NO_CHANGE", "PROPOSED_PATCH"]},
68
+ "horizon": {
69
+ "type": "object", "additionalProperties": false,
70
+ "required": ["kind", "max_seconds"],
71
+ "properties": {
72
+ "kind": {"const": "INVOCATION_COMPLETION"},
73
+ "max_seconds": {"type": "integer", "minimum": 1, "maximum": 3600}
74
+ }
75
+ },
76
+ "observation_classes": {"const": ["RETURN", "EXCEPTION", "TIMEOUT", "PROCESS_ERROR"]},
77
+ "observe": {"type": "array", "minItems": 1, "uniqueItems": true, "items": {"enum": ["observation_class", "return_value_sha256", "exception_type", "exit_code"]}, "contains": {"const": "observation_class"}}
78
+ }
79
+ },
80
+ "shared_input": {
81
+ "type": "object", "additionalProperties": false,
82
+ "required": ["input_sha256", "information_policy", "consumers"],
83
+ "properties": {
84
+ "input_sha256": {"$ref": "#/$defs/sha256"},
85
+ "information_policy": {"const": "IDENTICAL_REQUEST"},
86
+ "consumers": {"const": ["EXACT_EXECUTION", "STATIC_RULES", "RETRIEVAL", "LEARNED"]}
87
+ }
88
+ }
89
+ },
90
+ "$defs": {
91
+ "id": {"type": "string", "pattern": "^[a-z][a-z0-9._-]{2,127}$"},
92
+ "sha256": {"type": "string", "pattern": "^[0-9a-f]{64}$"},
93
+ "timestamp": {"type": "string", "format": "date-time"},
94
+ "path": {"type": "string", "pattern": "^(?!/)(?!.*(?:^|/)\\.\\.(?:/|$))[A-Za-z0-9._/-]+$"},
95
+ "sourceFile": {
96
+ "type": "object", "additionalProperties": false,
97
+ "required": ["path", "sha256", "content"],
98
+ "properties": {"path": {"$ref": "#/$defs/path"}, "sha256": {"$ref": "#/$defs/sha256"}, "content": {"type": "string"}}
99
+ },
100
+ "invocation": {
101
+ "type": "object", "additionalProperties": false,
102
+ "required": ["call_kind", "entrypoint", "args", "kwargs"],
103
+ "properties": {
104
+ "call_kind": {"enum": ["CONSTRUCTOR", "METHOD", "FUNCTION"]},
105
+ "entrypoint": {"type": "string", "pattern": "^[A-Za-z_][A-Za-z0-9_.:]*$"},
106
+ "receiver_ref": {"$ref": "#/$defs/id"},
107
+ "bind_result_as": {"$ref": "#/$defs/id"},
108
+ "args": {"type": "array", "items": {"$ref": "#/$defs/jsonValue"}},
109
+ "kwargs": {"type": "object", "propertyNames": {"pattern": "^[A-Za-z_][A-Za-z0-9_]*$"}, "additionalProperties": {"$ref": "#/$defs/jsonValue"}}
110
+ }
111
+ },
112
+ "assignment": {
113
+ "type": "object", "additionalProperties": false,
114
+ "required": ["target", "value"],
115
+ "properties": {
116
+ "target": {"type": "string", "pattern": "^[A-Za-z_][A-Za-z0-9_.]*$"},
117
+ "value": {"$ref": "#/$defs/jsonValue"}
118
+ }
119
+ },
120
+ "historyEvent": {
121
+ "type": "object", "additionalProperties": false,
122
+ "required": ["event_id", "observed_at", "source_snapshot_sha256", "origin", "invocation", "observation_class"],
123
+ "properties": {
124
+ "event_id": {"$ref": "#/$defs/id"},
125
+ "observed_at": {"$ref": "#/$defs/timestamp"},
126
+ "source_snapshot_sha256": {"$ref": "#/$defs/sha256"},
127
+ "origin": {"enum": ["PRE_PATCH_WORKLOAD_TRACE", "PRE_PATCH_BASE_EXECUTION"]},
128
+ "invocation": {"$ref": "#/$defs/invocation"},
129
+ "observation_class": {"enum": ["RETURN", "EXCEPTION", "TIMEOUT", "PROCESS_ERROR"]},
130
+ "return_value_sha256": {"$ref": "#/$defs/sha256"},
131
+ "exception_type": {"type": "string", "pattern": "^[A-Za-z_][A-Za-z0-9_.]*$"},
132
+ "exit_code": {"type": "integer", "minimum": 0, "maximum": 255}
133
+ }
134
+ },
135
+ "jsonValue": {"anyOf": [{"type": "null"}, {"type": "boolean"}, {"type": "number"}, {"type": "string"}, {"type": "array", "items": {"$ref": "#/$defs/jsonValue"}}, {"type": "object", "additionalProperties": {"$ref": "#/$defs/jsonValue"}}]}
136
+ }
137
+ }
evaluation/real-patch/source/code_review/contracts/real_patch_prediction_v1/validator.py ADDED
@@ -0,0 +1,179 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Fail-closed validator for the model-visible real-patch forecast request.
2
+
3
+ This module is deliberately loadable by path: the contract is not a runtime
4
+ package and callers need not install laxmi-review to validate an input packet.
5
+ """
6
+
7
+ from __future__ import annotations
8
+
9
+ import hashlib
10
+ import ast
11
+ import json
12
+ import re
13
+ from datetime import datetime, timezone
14
+ from pathlib import Path
15
+ from typing import Any
16
+
17
+ import jsonschema
18
+
19
+
20
+ SCHEMA_PATH = Path(__file__).with_name("forecast_request.schema.json")
21
+ _DIFF_HEADER = re.compile(r"^diff --git a/(\S+) b/(\S+)$", re.MULTILINE)
22
+ _TEST_PATH = re.compile(r"(?:^|/)(?:tests?|testing|fixtures?)(?:/|$)|(?:^|/)test_[^/]*\.py$|(?:^|/)[^/]*_test\.py$")
23
+
24
+
25
+ class ForecastContractError(ValueError):
26
+ """The request is not safe to expose to a forecast consumer."""
27
+
28
+
29
+ def _canonical_bytes(value: Any) -> bytes:
30
+ return json.dumps(
31
+ value, sort_keys=True, separators=(",", ":"), ensure_ascii=False, allow_nan=False
32
+ ).encode("utf-8")
33
+
34
+
35
+ def input_sha256(request: dict[str, Any]) -> str:
36
+ """Hash the complete consumer-visible request, excluding its own hash envelope."""
37
+ return hashlib.sha256(
38
+ _canonical_bytes({key: value for key, value in request.items() if key != "shared_input"})
39
+ ).hexdigest()
40
+
41
+
42
+ def source_snapshot_sha256(files: list[dict[str, Any]]) -> str:
43
+ """Content identity of the visible source slice, without public repository IDs."""
44
+ manifest = sorted(
45
+ ({"path": item["path"], "sha256": item["sha256"]} for item in files),
46
+ key=lambda item: item["path"],
47
+ )
48
+ return hashlib.sha256(_canonical_bytes(manifest)).hexdigest()
49
+
50
+
51
+ def _instant(raw: str) -> datetime:
52
+ try:
53
+ parsed = datetime.fromisoformat(raw.replace("Z", "+00:00"))
54
+ except ValueError as exc:
55
+ raise ForecastContractError(f"invalid timestamp: {raw!r}") from exc
56
+ if parsed.tzinfo is None:
57
+ raise ForecastContractError("timestamps must include a timezone")
58
+ return parsed.astimezone(timezone.utc)
59
+
60
+
61
+ def validate_forecast_request(request: dict[str, Any]) -> None:
62
+ """Validate shape, hashes, chronology, patch scope, and shared-input identity.
63
+
64
+ Source authenticity and semantic answer leakage still require custody checks:
65
+ a self-declared timestamp or hash cannot prove an external artifact existed.
66
+ """
67
+ schema = json.loads(SCHEMA_PATH.read_text(encoding="utf-8"))
68
+ try:
69
+ jsonschema.Draft202012Validator.check_schema(schema)
70
+ jsonschema.Draft202012Validator(
71
+ schema, format_checker=jsonschema.FormatChecker()
72
+ ).validate(request)
73
+ except (jsonschema.SchemaError, jsonschema.ValidationError) as exc:
74
+ raise ForecastContractError(f"request schema violation: {exc.message}") from exc
75
+
76
+ source = request["pre_source"]
77
+ patch = request["proposed_patch"]
78
+ scenario = request["scenario"]
79
+ cutoff = _instant(request["forecast_cutoff_at"])
80
+ proposed = _instant(patch["proposed_at"])
81
+ if source["snapshot_sha256"] != patch["base_snapshot_sha256"]:
82
+ raise ForecastContractError("patch base snapshot must match pre_source snapshot")
83
+ if _instant(source["snapshot_at"]) > proposed or proposed > cutoff:
84
+ raise ForecastContractError("source/proposal/forecast chronology is invalid")
85
+ if _instant(scenario["available_at"]) >= proposed:
86
+ raise ForecastContractError("scenario must be available before the proposed patch")
87
+ if scenario["origin"] == "PROSPECTIVE_GENERATOR" and scenario["selection_policy"] != "FROZEN_BEFORE_OUTCOME":
88
+ raise ForecastContractError("generator selection must be frozen")
89
+ if request["evaluation_split"] == "ACCEPTANCE_CANDIDATE" and scenario["selection_policy"] != "FROZEN_BEFORE_OUTCOME":
90
+ raise ForecastContractError("retrospective selection cannot be acceptance candidate")
91
+ program = scenario["program"]
92
+ if hashlib.sha256(program.encode("utf-8")).hexdigest() != scenario["program_sha256"]:
93
+ raise ForecastContractError("scenario program hash mismatch")
94
+ try:
95
+ syntax = ast.parse(program)
96
+ except SyntaxError as exc:
97
+ raise ForecastContractError("scenario program is not valid Python") from exc
98
+ for node in ast.walk(syntax):
99
+ if isinstance(node, ast.Assert):
100
+ raise ForecastContractError("scenario program contains an assertion")
101
+ if isinstance(node, ast.Name) and node.id.lower().startswith("expected"):
102
+ raise ForecastContractError("scenario program contains an expected-value variable")
103
+ if isinstance(node, ast.Raise) and isinstance(node.exc, ast.Call) and isinstance(node.exc.func, ast.Name) and node.exc.func.id == "AssertionError":
104
+ raise ForecastContractError("scenario program raises an assertion error")
105
+
106
+ paths: set[str] = set()
107
+ for item in source["files"]:
108
+ if item["path"] in paths:
109
+ raise ForecastContractError("duplicate source path")
110
+ paths.add(item["path"])
111
+ if hashlib.sha256(item["content"].encode("utf-8")).hexdigest() != item["sha256"]:
112
+ raise ForecastContractError(f"source hash mismatch: {item['path']}")
113
+ if source_snapshot_sha256(source["files"]) != source["snapshot_sha256"]:
114
+ raise ForecastContractError("source snapshot hash mismatch")
115
+ dependencies = request["operating_conditions"]["dependency_sources"]
116
+ dependency_paths: set[str] = set()
117
+ for item in dependencies:
118
+ if item["path"] in dependency_paths:
119
+ raise ForecastContractError("duplicate dependency path")
120
+ dependency_paths.add(item["path"])
121
+ if hashlib.sha256(item["content"].encode("utf-8")).hexdigest() != item["sha256"]:
122
+ raise ForecastContractError(f"dependency hash mismatch: {item['path']}")
123
+ if source_snapshot_sha256(dependencies) != request["operating_conditions"]["dependency_lock_sha256"]:
124
+ raise ForecastContractError("dependency lock hash mismatch")
125
+ diff = patch["unified_diff"]
126
+ if hashlib.sha256(diff.encode("utf-8")).hexdigest() != patch["diff_sha256"]:
127
+ raise ForecastContractError("proposed diff hash mismatch")
128
+ changed_paths = _DIFF_HEADER.findall(diff)
129
+ if not diff.startswith("diff --git ") or not changed_paths:
130
+ raise ForecastContractError("proposed diff needs git-style file headers")
131
+ if len(changed_paths) != sum(line.startswith("diff --git ") for line in diff.splitlines()):
132
+ raise ForecastContractError("proposed diff has an unrecognized file header")
133
+ for before, after in changed_paths:
134
+ if before != after:
135
+ raise ForecastContractError("renames need a separately reviewed contract")
136
+ if _TEST_PATH.search(before):
137
+ raise ForecastContractError("proposed diff contains a test or fixture path")
138
+ if before.startswith("/") or ".." in Path(before).parts:
139
+ raise ForecastContractError("proposed diff path escapes repository")
140
+ if before not in paths:
141
+ raise ForecastContractError("proposed diff file absent from visible pre source")
142
+
143
+ ids: set[str] = set()
144
+ for event in request["history"]:
145
+ if event["event_id"] in ids:
146
+ raise ForecastContractError("duplicate history event_id")
147
+ ids.add(event["event_id"])
148
+ if event["source_snapshot_sha256"] != source["snapshot_sha256"]:
149
+ raise ForecastContractError("history event is not from the pinned pre source")
150
+ if _instant(event["observed_at"]) >= proposed:
151
+ raise ForecastContractError("history event must predate patch proposal")
152
+
153
+ if request["shared_input"]["input_sha256"] != input_sha256(request):
154
+ raise ForecastContractError("shared input hash mismatch")
155
+
156
+
157
+ def load_forecast_request(path: Path) -> dict[str, Any]:
158
+ """Load strict JSON and validate before passing it to any consumer."""
159
+ def unique_pairs(pairs: list[tuple[str, Any]]) -> dict[str, Any]:
160
+ value: dict[str, Any] = {}
161
+ for key, item in pairs:
162
+ if key in value:
163
+ raise ForecastContractError(f"duplicate JSON key: {key}")
164
+ value[key] = item
165
+ return value
166
+
167
+ def reject_constant(value: str) -> None:
168
+ raise ForecastContractError(f"non-finite JSON number: {value}")
169
+
170
+ try:
171
+ request = json.loads(
172
+ path.read_text(encoding="utf-8"),
173
+ object_pairs_hook=unique_pairs,
174
+ parse_constant=reject_constant,
175
+ )
176
+ except json.JSONDecodeError as exc:
177
+ raise ForecastContractError(f"invalid JSON: {exc.msg}") from exc
178
+ validate_forecast_request(request)
179
+ return request
evaluation/real-patch/source/code_review/packaging/real_patch/api.py ADDED
@@ -0,0 +1,120 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Local, development-only real-patch validation and forecast interface.
2
+
3
+ The forecast path reads only the model-visible request. Oracle scoring is a
4
+ separate explicit operation, never part of forecast().
5
+ """
6
+
7
+ from __future__ import annotations
8
+
9
+ import argparse
10
+ import hashlib
11
+ import importlib.util
12
+ import json
13
+ from pathlib import Path
14
+
15
+
16
+ ROOT = Path(__file__).resolve().parent / "capsule/code_review"
17
+ CONTRACT = ROOT / "contracts/real_patch_prediction_v1/validator.py"
18
+ EXACT = ROOT / "scripts/run_real_patch_visible_exact_v1.py"
19
+ GATE = ROOT / "scripts/run_real_patch_baseline_gate_v2.py"
20
+
21
+
22
+ def _module(path: Path, name: str):
23
+ spec = importlib.util.spec_from_file_location(name, path)
24
+ if spec is None or spec.loader is None:
25
+ raise RuntimeError(f"packaged module cannot be loaded: {name}")
26
+ module = importlib.util.module_from_spec(spec)
27
+ spec.loader.exec_module(module)
28
+ return module
29
+
30
+
31
+ def _sha(path: Path) -> str:
32
+ return hashlib.sha256(path.read_bytes()).hexdigest()
33
+
34
+
35
+ def validate(request_path: Path) -> dict:
36
+ contract = _module(CONTRACT, "laxmi_real_patch_contract")
37
+ request = contract.load_forecast_request(request_path)
38
+ return {
39
+ "schema_version": "laxmi.real-patch-local-api.v1",
40
+ "status": "ADMITTED",
41
+ "case_id": request["case_id"],
42
+ "input_sha256": contract.input_sha256(request),
43
+ "request_file_sha256": _sha(request_path),
44
+ "evaluation_split": request["evaluation_split"],
45
+ "custody_verified": False,
46
+ "acceptance_eligible": False,
47
+ "executed": False,
48
+ }
49
+
50
+
51
+ def forecast(request_path: Path, output_dir: Path) -> dict:
52
+ admitted = validate(request_path)
53
+ exact = _module(EXACT, "laxmi_real_patch_visible_exact")
54
+ gate = _module(GATE, "laxmi_real_patch_gate")
55
+ output_dir.mkdir(mode=0o700, parents=True, exist_ok=False)
56
+ prediction, execution = exact.run(request_path, output_dir / "prediction")
57
+ sealed = gate.seal(request_path, output_dir / "prediction/predictions.json", output_dir / "seal")
58
+ if prediction["input_sha256"] != admitted["input_sha256"] or sealed["input_sha256"] != admitted["input_sha256"]:
59
+ raise ValueError("prediction changed the admitted input identity")
60
+ row = prediction["comparators"]["EXACT_VISIBLE_SOURCE"]
61
+ return {
62
+ "schema_version": "laxmi.real-patch-local-api.v1",
63
+ "status": "PREDICTED" if row["status"] == "predicted" else "UNSUPPORTED",
64
+ "engine_id": "EXACT_VISIBLE_SOURCE",
65
+ "information_tier": "SAME_INFORMATION",
66
+ "case_id": admitted["case_id"],
67
+ "input_sha256": admitted["input_sha256"],
68
+ "method_sha256": row.get("method_sha256"),
69
+ "reason": row.get("reason"),
70
+ "prediction_file_sha256": _sha(output_dir / "prediction/predictions.json"),
71
+ "seal_file_sha256": _sha(output_dir / "seal/seal.json"),
72
+ "oracle_read": execution["oracle_read"],
73
+ "custody_verified": False,
74
+ "acceptance_eligible": False,
75
+ "development_only": True,
76
+ }
77
+
78
+
79
+ def score(request_path: Path, oracle_path: Path, output_dir: Path) -> dict:
80
+ """Privileged local evaluation call; never invoked by forecast()."""
81
+ gate = _module(GATE, "laxmi_real_patch_gate")
82
+ receipt = gate.score(request_path, oracle_path, output_dir / "seal")
83
+ return {
84
+ "schema_version": "laxmi.real-patch-local-api.v1",
85
+ "status": "SCORED_DEVELOPMENT_ORACLE",
86
+ "case_id": receipt["case_id"],
87
+ "input_sha256": receipt["input_sha256"],
88
+ "receipt_file_sha256": _sha(output_dir / "seal/receipt.json"),
89
+ "same_information_scored_count": receipt["same_information_scored_count"],
90
+ "development_only": True,
91
+ }
92
+
93
+
94
+ def main() -> None:
95
+ parser = argparse.ArgumentParser(description=__doc__)
96
+ commands = parser.add_subparsers(dest="command", required=True)
97
+ validation = commands.add_parser("validate")
98
+ validation.add_argument("--request", type=Path, required=True)
99
+ prediction = commands.add_parser("forecast")
100
+ prediction.add_argument("--request", type=Path, required=True)
101
+ prediction.add_argument("--output-dir", type=Path, required=True)
102
+ scoring = commands.add_parser("score")
103
+ scoring.add_argument("--request", type=Path, required=True)
104
+ scoring.add_argument("--oracle", type=Path, required=True)
105
+ scoring.add_argument("--output-dir", type=Path, required=True)
106
+ args = parser.parse_args()
107
+ try:
108
+ if args.command == "validate":
109
+ result = validate(args.request)
110
+ elif args.command == "forecast":
111
+ result = forecast(args.request, args.output_dir)
112
+ else:
113
+ result = score(args.request, args.oracle, args.output_dir)
114
+ except (OSError, ValueError, RuntimeError) as exc:
115
+ parser.exit(2, f"Laxmi real-patch refused: {exc}\n")
116
+ print(json.dumps(result, sort_keys=True, allow_nan=False))
117
+
118
+
119
+ if __name__ == "__main__":
120
+ main()
evaluation/real-patch/source/code_review/scripts/build_real_patch_local_package_v1.py ADDED
@@ -0,0 +1,101 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Build a source-hash-preserving local wheel for real-patch development use."""
2
+
3
+ import argparse
4
+ import base64
5
+ import csv
6
+ import hashlib
7
+ import io
8
+ import json
9
+ from pathlib import Path
10
+ import zipfile
11
+
12
+
13
+ REPO = Path(__file__).resolve().parents[2]
14
+ VERSION = "0.1.0.dev1"
15
+ DIST = "laxmi_real_patch"
16
+ SOURCES = {
17
+ "api.py": REPO / "code_review/packaging/real_patch/api.py",
18
+ "LICENSE": REPO / "LICENSE",
19
+ "NOTICE": REPO / "NOTICE",
20
+ "LICENSE_SCOPE.md": REPO / "LICENSE_SCOPE.md",
21
+ "capsule/code_review/contracts/real_patch_prediction_v1/validator.py": REPO / "code_review/contracts/real_patch_prediction_v1/validator.py",
22
+ "capsule/code_review/contracts/real_patch_prediction_v1/forecast_request.schema.json": REPO / "code_review/contracts/real_patch_prediction_v1/forecast_request.schema.json",
23
+ "capsule/code_review/scripts/run_real_patch_visible_exact_v1.py": REPO / "code_review/scripts/run_real_patch_visible_exact_v1.py",
24
+ "capsule/code_review/scripts/run_real_patch_baseline_gate_v2.py": REPO / "code_review/scripts/run_real_patch_baseline_gate_v2.py",
25
+ }
26
+
27
+
28
+ def sha(raw: bytes) -> str:
29
+ return hashlib.sha256(raw).hexdigest()
30
+
31
+
32
+ def build(output: Path) -> dict:
33
+ if output.exists():
34
+ raise FileExistsError(output)
35
+ payload = {name: path.read_bytes() for name, path in SOURCES.items()}
36
+ if any(path.is_symlink() or not path.is_file() or len(payload[name]) > 1024 * 1024 for name, path in SOURCES.items()):
37
+ raise ValueError("package needs bounded regular source files")
38
+ payload["__init__.py"] = b"from .api import validate, forecast, score\n"
39
+ manifest = {
40
+ "schema_version": "laxmi.real-patch-local-package.v1",
41
+ "version": VERSION,
42
+ "development_only": True,
43
+ "learned_model_included": False,
44
+ "files": {name: sha(raw) for name, raw in sorted(payload.items())},
45
+ }
46
+ payload["manifest.json"] = (json.dumps(manifest, sort_keys=True, indent=2) + "\n").encode()
47
+ entries = {f"{DIST}/{name}": raw for name, raw in payload.items()}
48
+ info = f"{DIST}-{VERSION}.dist-info/"
49
+ entries[info + "METADATA"] = (
50
+ "Metadata-Version: 2.1\n"
51
+ "Name: laxmi-real-patch\n"
52
+ f"Version: {VERSION}\n"
53
+ "Summary: Development-only real-patch forecast and evaluation interface\n"
54
+ "Requires-Python: >=3.12,<3.13\n"
55
+ "Requires-Dist: jsonschema>=4.23,<5\n"
56
+ "License: Apache-2.0\n"
57
+ "Author: Navneet Prabhakar\n\n"
58
+ ).encode()
59
+ entries[info + "WHEEL"] = b"Wheel-Version: 1.0\nGenerator: laxmi-local-package\nRoot-Is-Purelib: true\nTag: py3-none-any\n"
60
+ entries[info + "entry_points.txt"] = b"[console_scripts]\nlaxmi-real-patch = laxmi_real_patch.api:main\n"
61
+ record = io.StringIO(newline="")
62
+ writer = csv.writer(record, lineterminator="\n")
63
+ for name, raw in sorted(entries.items()):
64
+ encoded = base64.urlsafe_b64encode(hashlib.sha256(raw).digest()).decode().rstrip("=")
65
+ writer.writerow([name, f"sha256={encoded}", len(raw)])
66
+ writer.writerow([info + "RECORD", "", ""])
67
+ entries[info + "RECORD"] = record.getvalue().encode()
68
+ output.mkdir(parents=True, exist_ok=False)
69
+ wheel = output / f"{DIST}-{VERSION}-py3-none-any.whl"
70
+ with zipfile.ZipFile(wheel, "x", compression=zipfile.ZIP_DEFLATED) as archive:
71
+ for name, raw in sorted(entries.items()):
72
+ member = zipfile.ZipInfo(name, (1980, 1, 1, 0, 0, 0))
73
+ member.compress_type = zipfile.ZIP_DEFLATED
74
+ member.external_attr = 0o100644 << 16
75
+ archive.writestr(member, raw)
76
+ receipt = {
77
+ "schema_version": "laxmi.real-patch-local-build.v1",
78
+ "version": VERSION,
79
+ "wheel": wheel.name,
80
+ "wheel_sha256": sha(wheel.read_bytes()),
81
+ "wheel_bytes": wheel.stat().st_size,
82
+ "payload_files": manifest["files"],
83
+ "builder_sha256": sha(Path(__file__).read_bytes()),
84
+ "linux_validation_pending": True,
85
+ "linux_exact_execution_pending": True,
86
+ "public_release": False,
87
+ }
88
+ (output / "build.json").write_text(json.dumps(receipt, sort_keys=True, indent=2) + "\n", encoding="utf-8")
89
+ return receipt
90
+
91
+
92
+ def main() -> None:
93
+ parser = argparse.ArgumentParser(description=__doc__)
94
+ parser.add_argument("--output-dir", type=Path, required=True)
95
+ args = parser.parse_args()
96
+ result = build(args.output_dir)
97
+ print(json.dumps({key: result[key] for key in ("wheel", "wheel_sha256", "wheel_bytes")}, sort_keys=True))
98
+
99
+
100
+ if __name__ == "__main__":
101
+ main()
evaluation/real-patch/source/code_review/scripts/run_real_patch_baseline_gate_v2.py ADDED
@@ -0,0 +1,264 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Seal same-request baseline forecasts before scoring a development oracle.
2
+
3
+ This is a protocol runner, not an implementation of any comparator. Unsupported
4
+ comparators stay in the denominator and no source or candidate code is executed.
5
+ """
6
+
7
+ import argparse
8
+ import hashlib
9
+ import importlib.util
10
+ import json
11
+ from pathlib import Path
12
+
13
+
14
+ SCHEMA_VERSION = "laxmi.real-patch-baseline-gate.v2"
15
+ PREDICTION_VERSION = "laxmi.real-patch-baseline-predictions.v2"
16
+ ORACLE_VERSION = "laxmi.real-patch-observation-oracle.v1"
17
+ COMPARATORS = ("EXACT_EXECUTION", "EXACT_VISIBLE_SOURCE", "STATIC_RULES", "RETRIEVAL")
18
+ INFORMATION_TIERS = ("SAME_INFORMATION", "PRIVILEGED_CEILING")
19
+ CONTRACT_PATH = (
20
+ Path(__file__).resolve().parents[1]
21
+ / "contracts/real_patch_prediction_v1/validator.py"
22
+ )
23
+
24
+
25
+ def canonical_bytes(value):
26
+ return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()
27
+
28
+
29
+ def sha256(data):
30
+ return hashlib.sha256(data).hexdigest()
31
+
32
+
33
+ def read_json(path):
34
+ def unique_pairs(pairs):
35
+ value = {}
36
+ for key, item in pairs:
37
+ if key in value:
38
+ raise ValueError(f"duplicate JSON key: {key}")
39
+ value[key] = item
40
+ return value
41
+
42
+ def reject_constant(value):
43
+ raise ValueError(f"non-finite JSON number: {value}")
44
+
45
+ with path.open("rb") as handle:
46
+ data = handle.read()
47
+ return json.loads(data, object_pairs_hook=unique_pairs, parse_constant=reject_constant), sha256(data)
48
+
49
+
50
+ def write_create_only(path, value):
51
+ encoded = json.dumps(value, sort_keys=True, indent=2, ensure_ascii=False).encode() + b"\n"
52
+ with path.open("xb") as handle:
53
+ handle.write(encoded)
54
+ return sha256(encoded)
55
+
56
+
57
+ def load_contract():
58
+ if not CONTRACT_PATH.is_file():
59
+ raise FileNotFoundError(f"forecast request validator missing: {CONTRACT_PATH}")
60
+ spec = importlib.util.spec_from_file_location("real_patch_forecast_validator", CONTRACT_PATH)
61
+ module = importlib.util.module_from_spec(spec)
62
+ spec.loader.exec_module(module)
63
+ return module
64
+
65
+
66
+ def validate_observation(observation, observe, classes):
67
+ if not isinstance(observation, dict) or set(observation) != set(observe):
68
+ raise ValueError("observation fields must exactly match request target.observe")
69
+ if observation["observation_class"] not in classes:
70
+ raise ValueError("observation_class outside frozen request vocabulary")
71
+ for field, value in observation.items():
72
+ if field == "return_value_sha256" and value is not None:
73
+ if not isinstance(value, str) or len(value) != 64 or any(
74
+ char not in "0123456789abcdef" for char in value
75
+ ):
76
+ raise ValueError("return_value_sha256 must be a lowercase SHA-256 or null")
77
+ elif field == "exception_type" and value is not None and not isinstance(value, str):
78
+ raise ValueError("exception_type must be a string or null")
79
+ elif field == "exit_code" and value is not None and (type(value) is not int):
80
+ raise ValueError("exit_code must be an integer or null")
81
+
82
+
83
+ def validate_observations(observations, target):
84
+ branches = target["branches"]
85
+ if not isinstance(observations, dict) or set(observations) != set(branches):
86
+ raise ValueError("observations must cover both frozen intervention branches")
87
+ for branch in branches:
88
+ validate_observation(
89
+ observations[branch], target["observe"], target["observation_classes"]
90
+ )
91
+
92
+
93
+ def validate_predictions(predictions, request, request_hash):
94
+ if predictions.get("schema_version") != PREDICTION_VERSION:
95
+ raise ValueError("wrong prediction schema_version")
96
+ if predictions.get("case_id") != request["case_id"]:
97
+ raise ValueError("prediction case_id differs from request")
98
+ if predictions.get("input_sha256") != request_hash:
99
+ raise ValueError("prediction input_sha256 differs from frozen shared input")
100
+ rows = predictions.get("comparators")
101
+ if not isinstance(rows, dict) or set(rows) != set(COMPARATORS):
102
+ raise ValueError("exactly four named comparator records are required")
103
+ for name, row in rows.items():
104
+ if not isinstance(row, dict) or row.get("status") not in {"predicted", "unsupported"}:
105
+ raise ValueError(f"{name} requires predicted or unsupported status")
106
+ if row.get("information_tier") not in INFORMATION_TIERS:
107
+ raise ValueError(f"{name} requires an explicit information_tier")
108
+ if name != "EXACT_EXECUTION" and row["information_tier"] != "SAME_INFORMATION":
109
+ raise ValueError(f"{name} must use same information")
110
+ if row["status"] == "unsupported":
111
+ if set(row) != {"status", "reason", "information_tier"} or not isinstance(row["reason"], str) or not row["reason"].strip():
112
+ raise ValueError(f"{name} unsupported requires only a nonempty reason")
113
+ else:
114
+ if set(row) != {"status", "observations", "method_sha256", "information_tier"}:
115
+ raise ValueError(f"{name} prediction requires paired observations and method SHA")
116
+ method = row["method_sha256"]
117
+ if not isinstance(method, str) or len(method) != 64 or any(
118
+ char not in "0123456789abcdef" for char in method
119
+ ):
120
+ raise ValueError(f"{name} method_sha256 must be a lowercase SHA-256")
121
+ validate_observations(row["observations"], request["target"])
122
+
123
+
124
+ def seal(envelope_path, predictions_path, output_dir):
125
+ contract = load_contract()
126
+ request, request_file_sha = read_json(envelope_path)
127
+ contract.validate_forecast_request(request)
128
+ request_hash = contract.input_sha256(request)
129
+ predictions, predictions_file_sha = read_json(predictions_path)
130
+ validate_predictions(predictions, request, request_hash)
131
+ output_dir.mkdir(mode=0o700, parents=True, exist_ok=False)
132
+ record = {
133
+ "schema_version": SCHEMA_VERSION,
134
+ "phase": "sealed_predictions",
135
+ "development_only": True,
136
+ "case_id": request["case_id"],
137
+ "input_sha256": request_hash,
138
+ "envelope_file_sha256": request_file_sha,
139
+ "prediction_file_sha256": predictions_file_sha,
140
+ "predictions": predictions,
141
+ "oracle_not_read_by_seal": True,
142
+ "baseline_methods_not_executed_by_runner": True,
143
+ }
144
+ seal_digest = write_create_only(output_dir / "seal.json", record)
145
+ write_create_only(output_dir / "seal.sha256.json", {"seal_sha256": seal_digest})
146
+ return record
147
+
148
+
149
+ def score(envelope_path, oracle_path, output_dir):
150
+ if (output_dir / "invalidated.json").exists():
151
+ raise ValueError("sealed prediction was invalidated; do not score")
152
+ seal_path = output_dir / "seal.json"
153
+ frozen, seal_file_sha = read_json(seal_path)
154
+ seal_digest_record, _ = read_json(output_dir / "seal.sha256.json")
155
+ if seal_digest_record != {"seal_sha256": seal_file_sha}:
156
+ raise ValueError("seal bytes differ from create-only digest record")
157
+ if frozen.get("schema_version") != SCHEMA_VERSION or frozen.get("phase") != "sealed_predictions":
158
+ raise ValueError("invalid seal")
159
+ if (output_dir / "receipt.json").exists():
160
+ raise FileExistsError(output_dir / "receipt.json")
161
+ contract = load_contract()
162
+ request, request_file_sha = read_json(envelope_path)
163
+ contract.validate_forecast_request(request)
164
+ request_hash = contract.input_sha256(request)
165
+ if (
166
+ frozen.get("case_id") != request["case_id"]
167
+ or frozen.get("input_sha256") != request_hash
168
+ or frozen.get("envelope_file_sha256") != request_file_sha
169
+ ):
170
+ raise ValueError("request differs from sealed request")
171
+ predictions = frozen["predictions"]
172
+ validate_predictions(predictions, request, request_hash)
173
+ # Oracle access begins only after the prediction seal has passed validation.
174
+ oracle, oracle_file_sha = read_json(oracle_path)
175
+ if oracle.get("schema_version") != ORACLE_VERSION or oracle.get("case_id") != request["case_id"]:
176
+ raise ValueError("oracle version or case_id differs from sealed request")
177
+ if set(oracle) != {"schema_version", "case_id", "observations"}:
178
+ raise ValueError("oracle contains unexpected fields")
179
+ target = request["target"]
180
+ observe = target["observe"]
181
+ branches = target["branches"]
182
+ validate_observations(oracle["observations"], target)
183
+ scores = {}
184
+ for name in COMPARATORS:
185
+ row = predictions["comparators"][name]
186
+ if row["status"] == "unsupported":
187
+ scores[name] = {
188
+ "status": "unsupported",
189
+ "reason": row["reason"],
190
+ "information_tier": row["information_tier"],
191
+ }
192
+ continue
193
+ branch_field_matches = {
194
+ branch: {
195
+ field: row["observations"][branch][field]
196
+ == oracle["observations"][branch][field]
197
+ for field in observe
198
+ }
199
+ for branch in branches
200
+ }
201
+ predicted_changed = (
202
+ row["observations"][branches[0]] != row["observations"][branches[1]]
203
+ )
204
+ observed_changed = (
205
+ oracle["observations"][branches[0]] != oracle["observations"][branches[1]]
206
+ )
207
+ scores[name] = {
208
+ "status": "scored",
209
+ "information_tier": row["information_tier"],
210
+ "method_sha256": row["method_sha256"],
211
+ "branch_field_matches": branch_field_matches,
212
+ "both_branches_match": all(
213
+ all(field_matches.values()) for field_matches in branch_field_matches.values()
214
+ ),
215
+ "effect_changed_match": predicted_changed == observed_changed,
216
+ }
217
+ receipt = {
218
+ "schema_version": SCHEMA_VERSION,
219
+ "phase": "scored_development_oracle",
220
+ "development_only": True,
221
+ "case_id": request["case_id"],
222
+ "input_sha256": request_hash,
223
+ "seal_file_sha256": seal_file_sha,
224
+ "oracle_file_sha256": oracle_file_sha,
225
+ "supported_comparator_count": sum(row["status"] == "scored" for row in scores.values()),
226
+ "unsupported_comparator_count": sum(row["status"] == "unsupported" for row in scores.values()),
227
+ "same_information_scored_count": sum(
228
+ row["status"] == "scored" and row["information_tier"] == "SAME_INFORMATION"
229
+ for row in scores.values()
230
+ ),
231
+ "privileged_ceiling_scored_count": sum(
232
+ row["status"] == "scored" and row["information_tier"] == "PRIVILEGED_CEILING"
233
+ for row in scores.values()
234
+ ),
235
+ "scores": scores,
236
+ "prediction_methods_self_declared_not_verified_by_runner": True,
237
+ "baseline_methods_not_executed_by_runner": True,
238
+ "no_learned_value_claim": True,
239
+ }
240
+ write_create_only(output_dir / "receipt.json", receipt)
241
+ return receipt
242
+
243
+
244
+ def main():
245
+ parser = argparse.ArgumentParser(description=__doc__)
246
+ subparsers = parser.add_subparsers(dest="phase", required=True)
247
+ seal_parser = subparsers.add_parser("seal")
248
+ seal_parser.add_argument("--envelope", type=Path, required=True)
249
+ seal_parser.add_argument("--predictions", type=Path, required=True)
250
+ seal_parser.add_argument("--output-dir", type=Path, required=True)
251
+ score_parser = subparsers.add_parser("score")
252
+ score_parser.add_argument("--envelope", type=Path, required=True)
253
+ score_parser.add_argument("--oracle", type=Path, required=True)
254
+ score_parser.add_argument("--output-dir", type=Path, required=True)
255
+ args = parser.parse_args()
256
+ if args.phase == "seal":
257
+ result = seal(args.envelope, args.predictions, args.output_dir)
258
+ else:
259
+ result = score(args.envelope, args.oracle, args.output_dir)
260
+ print(json.dumps({key: result[key] for key in ("case_id", "phase")}, sort_keys=True))
261
+
262
+
263
+ if __name__ == "__main__":
264
+ main()
evaluation/real-patch/source/code_review/scripts/run_real_patch_visible_exact_v1.py ADDED
@@ -0,0 +1,300 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Same-information exact execution from one validated visible forecast request.
2
+
3
+ No archive, Git repository, custody record, oracle, model, or network is read.
4
+ The request's source, dependency, diff, and scenario bytes are materialized in
5
+ a create-only workspace. Runtime code executes only in locked-down Docker.
6
+ """
7
+
8
+ import argparse
9
+ import hashlib
10
+ import importlib.util
11
+ import json
12
+ from pathlib import Path, PurePosixPath
13
+ import re
14
+ import subprocess
15
+
16
+
17
+ IMAGE = "python:3.12-slim"
18
+ IMAGE_ID = "sha256:78387bc3881b8273120a12ebe6c1ab22b018ccc2c9adf565ae1ac9b536e184ea"
19
+ PREDICTION_VERSION = "laxmi.real-patch-baseline-predictions.v2"
20
+ CONTRACT_PATH = Path(__file__).resolve().parents[1] / "contracts/real_patch_prediction_v1/validator.py"
21
+ HUNK = re.compile(r"^@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@")
22
+ HEADER = re.compile(r"^diff --git a/(\S+) b/(\S+)\n$")
23
+
24
+
25
+ class UnsupportedVisibleExecution(ValueError):
26
+ """The visible request cannot supply a valid runnable source slice."""
27
+
28
+
29
+ def digest(data):
30
+ return hashlib.sha256(data).hexdigest()
31
+
32
+
33
+ def canonical_bytes(value):
34
+ return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()
35
+
36
+
37
+ def write_create_only(path, data):
38
+ path.parent.mkdir(parents=True, exist_ok=True)
39
+ with path.open("xb") as handle:
40
+ handle.write(data)
41
+
42
+
43
+ def safe_path(root, relative):
44
+ path = PurePosixPath(relative)
45
+ if path.is_absolute() or ".." in path.parts or not path.parts:
46
+ raise UnsupportedVisibleExecution("patch path escapes visible workspace")
47
+ return root.joinpath(*path.parts)
48
+
49
+
50
+ def apply_file_hunks(source, hunk_lines):
51
+ original = source.splitlines(keepends=True)
52
+ output = []
53
+ cursor = 0
54
+ index = 0
55
+ while index < len(hunk_lines):
56
+ match = HUNK.match(hunk_lines[index])
57
+ if match is None:
58
+ raise UnsupportedVisibleExecution("patch has content outside a hunk")
59
+ old_start = int(match.group(1))
60
+ old_count = int(match.group(2) or "1")
61
+ new_start = int(match.group(3))
62
+ new_count = int(match.group(4) or "1")
63
+ start = old_start - 1 if old_count else old_start
64
+ if start < cursor or start > len(original):
65
+ raise UnsupportedVisibleExecution("overlapping or out-of-range patch hunk")
66
+ output.extend(original[cursor:start])
67
+ expected_new_start = len(output) + 1 if new_count else len(output)
68
+ if new_start != expected_new_start:
69
+ raise UnsupportedVisibleExecution("patch destination hunk position is inconsistent")
70
+ position = start
71
+ old_seen = 0
72
+ new_seen = 0
73
+ index += 1
74
+ while index < len(hunk_lines) and not hunk_lines[index].startswith("@@ "):
75
+ line = hunk_lines[index]
76
+ if not line or line[0] not in " +-" or not line.endswith("\n"):
77
+ raise UnsupportedVisibleExecution("unsupported patch line")
78
+ text = line[1:]
79
+ if line[0] in " -":
80
+ if position >= len(original) or original[position] != text:
81
+ raise UnsupportedVisibleExecution("patch context differs from visible source")
82
+ position += 1
83
+ old_seen += 1
84
+ if line[0] in " +":
85
+ output.append(text)
86
+ new_seen += 1
87
+ index += 1
88
+ if old_seen != old_count or new_seen != new_count:
89
+ raise UnsupportedVisibleExecution("patch hunk line counts are inconsistent")
90
+ cursor = position
91
+ output.extend(original[cursor:])
92
+ return "".join(output)
93
+
94
+
95
+ def apply_visible_diff(pre_root, post_root, diff, source_paths):
96
+ lines = diff.splitlines(keepends=True)
97
+ index = 0
98
+ changed = []
99
+ while index < len(lines):
100
+ match = HEADER.match(lines[index])
101
+ if match is None or match.group(1) != match.group(2):
102
+ raise UnsupportedVisibleExecution("only same-path git-style text diffs are supported")
103
+ path = match.group(1)
104
+ if path not in source_paths or path in changed:
105
+ raise UnsupportedVisibleExecution("patch file absent from visible source or repeated")
106
+ changed.append(path)
107
+ index += 1
108
+ if index + 1 >= len(lines) or lines[index] != f"--- a/{path}\n" or lines[index + 1] != f"+++ b/{path}\n":
109
+ raise UnsupportedVisibleExecution("patch file headers are inconsistent")
110
+ index += 2
111
+ hunks = []
112
+ while index < len(lines) and not lines[index].startswith("diff --git "):
113
+ hunks.append(lines[index])
114
+ index += 1
115
+ if not hunks:
116
+ raise UnsupportedVisibleExecution("patch file has no hunks")
117
+ before = safe_path(pre_root, path).read_text(encoding="utf-8")
118
+ after = apply_file_hunks(before, hunks)
119
+ safe_path(post_root, path).write_text(after, encoding="utf-8")
120
+ if not changed:
121
+ raise UnsupportedVisibleExecution("visible diff has no patch files")
122
+ return changed
123
+
124
+
125
+ def load_contract():
126
+ spec = importlib.util.spec_from_file_location("real_patch_forecast_validator", CONTRACT_PATH)
127
+ module = importlib.util.module_from_spec(spec)
128
+ spec.loader.exec_module(module)
129
+ return module
130
+
131
+
132
+ def check_image():
133
+ result = subprocess.run(
134
+ ["docker", "image", "inspect", IMAGE, "--format", "{{.Id}}"],
135
+ capture_output=True, text=True, check=True, timeout=20,
136
+ )
137
+ if result.stdout.strip() != IMAGE_ID:
138
+ raise ValueError("Docker image ID differs from pinned runtime")
139
+
140
+
141
+ def observation(completed, observe):
142
+ if completed.returncode == 0:
143
+ try:
144
+ payload = json.loads(completed.stdout)
145
+ except json.JSONDecodeError as exc:
146
+ raise UnsupportedVisibleExecution("visible program did not emit one JSON result") from exc
147
+ if not isinstance(payload, dict) or set(payload) != {"may", "actual"} or any(
148
+ type(value) is not bool for value in payload.values()
149
+ ):
150
+ raise UnsupportedVisibleExecution("visible program result must be raw may/actual booleans")
151
+ values = {
152
+ "observation_class": "RETURN",
153
+ "return_value_sha256": digest(canonical_bytes(payload)),
154
+ "exception_type": None,
155
+ "exit_code": 0,
156
+ }
157
+ elif "ModuleNotFoundError" in completed.stderr or "ImportError" in completed.stderr:
158
+ raise UnsupportedVisibleExecution("visible source/dependency slice cannot import")
159
+ else:
160
+ values = {
161
+ "observation_class": "PROCESS_ERROR",
162
+ "return_value_sha256": None,
163
+ "exception_type": None,
164
+ "exit_code": completed.returncode,
165
+ }
166
+ return {field: values[field] for field in observe}
167
+
168
+
169
+ def run_branch(program_path, source_root, deps_root, timeout_seconds, observe):
170
+ command = [
171
+ "docker", "run", "--rm", "--pull", "never", "--network", "none",
172
+ "--read-only", "--cap-drop", "ALL", "--security-opt", "no-new-privileges",
173
+ "--pids-limit", "64", "--memory", "256m", "--cpus", "1",
174
+ "--user", "65534:65534",
175
+ "--mount", f"type=bind,source={program_path.resolve()},target=/scenario.py,readonly",
176
+ "--mount", f"type=bind,source={source_root.resolve()},target=/src,readonly",
177
+ "--mount", f"type=bind,source={deps_root.resolve()},target=/deps,readonly",
178
+ "--env", "PYTHONPATH=/src:/deps", "--env", "PYTHONDONTWRITEBYTECODE=1",
179
+ IMAGE, "python", "/scenario.py",
180
+ ]
181
+ try:
182
+ result = subprocess.run(
183
+ command, capture_output=True, text=True, check=False, timeout=timeout_seconds,
184
+ )
185
+ except subprocess.TimeoutExpired:
186
+ values = {"observation_class": "TIMEOUT", "return_value_sha256": None,
187
+ "exception_type": None, "exit_code": None}
188
+ return {field: values[field] for field in observe}, {"command": command, "timed_out": True}
189
+ if result.returncode == 125:
190
+ raise RuntimeError(f"Docker setup failed: {result.stderr[-1000:]}")
191
+ return observation(result, observe), {
192
+ "command": command,
193
+ "exit_code": result.returncode,
194
+ "stdout": result.stdout,
195
+ "stderr": result.stderr,
196
+ "stdout_sha256": digest(result.stdout.encode()),
197
+ "stderr_sha256": digest(result.stderr.encode()),
198
+ }
199
+
200
+
201
+ def run(request_path, output_dir):
202
+ contract = load_contract()
203
+ request = contract.load_forecast_request(request_path)
204
+ if request["target"]["branches"] != ["NO_CHANGE", "PROPOSED_PATCH"]:
205
+ raise ValueError("paired branch order required")
206
+ observe = request["target"]["observe"]
207
+ if "observation_class" not in observe or "return_value_sha256" not in observe:
208
+ raise ValueError("raw paired result fields required")
209
+ output_dir.mkdir(mode=0o700, parents=True, exist_ok=False)
210
+ workspace = output_dir / "workspace"
211
+ pre_root = workspace / "no_change"
212
+ post_root = workspace / "proposed_patch"
213
+ deps_root = workspace / "deps"
214
+ source_paths = set()
215
+ for item in request["pre_source"]["files"]:
216
+ source_paths.add(item["path"])
217
+ for root in (pre_root, post_root):
218
+ write_create_only(safe_path(root, item["path"]), item["content"].encode())
219
+ for item in request["operating_conditions"]["dependency_sources"]:
220
+ relative = item["path"]
221
+ if not relative.startswith("deps/"):
222
+ raise ValueError("dependency path must be under deps/")
223
+ write_create_only(safe_path(workspace, relative), item["content"].encode())
224
+ program_path = workspace / "scenario.py"
225
+ write_create_only(program_path, request["scenario"]["program"].encode())
226
+ method = {
227
+ "script_sha256": digest(Path(__file__).read_bytes()),
228
+ "image_id": IMAGE_ID,
229
+ "input_sha256": contract.input_sha256(request),
230
+ "visible_source_snapshot_sha256": request["pre_source"]["snapshot_sha256"],
231
+ "visible_diff_sha256": request["proposed_patch"]["diff_sha256"],
232
+ "visible_dependency_lock_sha256": request["operating_conditions"]["dependency_lock_sha256"],
233
+ "visible_program_sha256": request["scenario"]["program_sha256"],
234
+ }
235
+ runs = {}
236
+ changed = []
237
+ try:
238
+ changed = apply_visible_diff(
239
+ pre_root, post_root, request["proposed_patch"]["unified_diff"], source_paths
240
+ )
241
+ check_image()
242
+ observations = {}
243
+ for branch, root in (("NO_CHANGE", pre_root), ("PROPOSED_PATCH", post_root)):
244
+ observations[branch], runs[branch] = run_branch(
245
+ program_path, root, deps_root,
246
+ request["target"]["horizon"]["max_seconds"], observe,
247
+ )
248
+ exact = {
249
+ "status": "predicted",
250
+ "information_tier": "SAME_INFORMATION",
251
+ "observations": observations,
252
+ "method_sha256": digest(canonical_bytes(method)),
253
+ }
254
+ except UnsupportedVisibleExecution as exc:
255
+ exact = {
256
+ "status": "unsupported",
257
+ "information_tier": "SAME_INFORMATION",
258
+ "reason": str(exc),
259
+ }
260
+ def unsupported(reason, tier):
261
+ return {"status": "unsupported", "information_tier": tier, "reason": reason}
262
+ prediction = {
263
+ "schema_version": PREDICTION_VERSION,
264
+ "case_id": request["case_id"],
265
+ "input_sha256": contract.input_sha256(request),
266
+ "comparators": {
267
+ "EXACT_EXECUTION": unsupported("Privileged full-source ceiling is separate", "PRIVILEGED_CEILING"),
268
+ "EXACT_VISIBLE_SOURCE": exact,
269
+ "STATIC_RULES": unsupported("No static comparator sealed for this case", "SAME_INFORMATION"),
270
+ "RETRIEVAL": unsupported("No pre-cutoff retrieval index sealed for this case", "SAME_INFORMATION"),
271
+ },
272
+ }
273
+ receipt = {
274
+ "schema_version": "laxmi.real-patch-visible-exact.v1",
275
+ "development_only": True,
276
+ "oracle_read": False,
277
+ "custody_or_post_archive_read": False,
278
+ "input_sha256": contract.input_sha256(request),
279
+ "method": method,
280
+ "changed_visible_paths": changed,
281
+ "runs": runs,
282
+ "status": exact["status"],
283
+ }
284
+ for name, value in (("predictions.json", prediction), ("execution.json", receipt)):
285
+ write_create_only(output_dir / name, json.dumps(value, sort_keys=True, indent=2).encode() + b"\n")
286
+ return prediction, receipt
287
+
288
+
289
+ def main():
290
+ parser = argparse.ArgumentParser(description=__doc__)
291
+ parser.add_argument("--request", type=Path, required=True)
292
+ parser.add_argument("--output-dir", type=Path, required=True)
293
+ args = parser.parse_args()
294
+ prediction, _ = run(args.request, args.output_dir)
295
+ print(json.dumps({"case_id": prediction["case_id"],
296
+ "status": prediction["comparators"]["EXACT_VISIBLE_SOURCE"]["status"]}, sort_keys=True))
297
+
298
+
299
+ if __name__ == "__main__":
300
+ main()
release.json CHANGED
@@ -2,33 +2,46 @@
2
  "author": "Navneet Prabhakar",
3
  "candidates": {
4
  "17-normalized": {
5
- "metadata_sha256": "225a71bb67d619e70f1aec0313d2c5791d0a94ac3942297ad12ba284269cddd8",
6
- "example_forecast_sha256": "e4c9661abcc03fe8aee0ae5bf57e1b12f9e8a56becc67e41d26ea106b0ae5267"
7
  },
8
  "17-original": {
9
- "metadata_sha256": "a958ceb64e802decf331d10e98ddfcdc042658834469a0cc4a131e44134de906",
10
- "example_forecast_sha256": "5cca70b946ccaac1a453f899fc54e088452728a24ae9e6ba7f5f2e8ee5b6e6e6"
11
  },
12
  "43-normalized": {
13
- "metadata_sha256": "8eb622127bb5a9d739967855b8d7d8b7290395d7ae3a83315325b1a20869146f",
14
- "example_forecast_sha256": "cbe28210ab8bce43ebe5223b38c62fdcd07cf796d26173651da03fd32c2135e8"
15
  },
16
  "43-original": {
17
- "metadata_sha256": "4e02bd6724288e074fb30f04164a83559eff5e370e94bc473ace3ac22bc7a742",
18
- "example_forecast_sha256": "c18c7cbb05b59973026e370a6ba8bfb2e35f1e9d772ff183a32dec94da6ccc93"
19
  }
20
  },
21
  "clean_install_verified": true,
22
  "dependency_hash_lock": false,
 
23
  "license": "Apache-2.0",
24
  "model_acceptance": false,
25
  "original_wheel_sha256": "e1bafeb8d72e231a3029cf11ffabb8e54aa28e1b953040e81e77cf1fa3ac270d",
26
  "package_sha256": "a74db8e3196267387e3a4cfb55b838d112a86c88adac031f6515b88230a29fcc",
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
  "runtime_sources": 133,
28
  "schema_version": "laxmi.experimental-public-package.v1",
 
29
  "supported_platform": "macOS ARM64, Python 3.12; other platforms unverified",
30
- "version": "0.1.0",
31
- "distribution_scope": "RELEASE_SCOPE.md; exact files enumerated by SHA256SUMS",
32
- "repository_checkpoint_commit": "4dae1a041be532d663f477effe9b654b428bc04e",
33
- "source_identity_basis": "original runtime wheel SHA256 and per-file checkpoint source locks; repository checkpoint is a recovery anchor, not a claim of identical historical source tree"
34
  }
 
2
  "author": "Navneet Prabhakar",
3
  "candidates": {
4
  "17-normalized": {
5
+ "example_forecast_sha256": "e4c9661abcc03fe8aee0ae5bf57e1b12f9e8a56becc67e41d26ea106b0ae5267",
6
+ "metadata_sha256": "225a71bb67d619e70f1aec0313d2c5791d0a94ac3942297ad12ba284269cddd8"
7
  },
8
  "17-original": {
9
+ "example_forecast_sha256": "5cca70b946ccaac1a453f899fc54e088452728a24ae9e6ba7f5f2e8ee5b6e6e6",
10
+ "metadata_sha256": "a958ceb64e802decf331d10e98ddfcdc042658834469a0cc4a131e44134de906"
11
  },
12
  "43-normalized": {
13
+ "example_forecast_sha256": "cbe28210ab8bce43ebe5223b38c62fdcd07cf796d26173651da03fd32c2135e8",
14
+ "metadata_sha256": "8eb622127bb5a9d739967855b8d7d8b7290395d7ae3a83315325b1a20869146f"
15
  },
16
  "43-original": {
17
+ "example_forecast_sha256": "c18c7cbb05b59973026e370a6ba8bfb2e35f1e9d772ff183a32dec94da6ccc93",
18
+ "metadata_sha256": "4e02bd6724288e074fb30f04164a83559eff5e370e94bc473ace3ac22bc7a742"
19
  }
20
  },
21
  "clean_install_verified": true,
22
  "dependency_hash_lock": false,
23
+ "distribution_scope": "RELEASE_SCOPE.md; exact files enumerated by SHA256SUMS",
24
  "license": "Apache-2.0",
25
  "model_acceptance": false,
26
  "original_wheel_sha256": "e1bafeb8d72e231a3029cf11ffabb8e54aa28e1b953040e81e77cf1fa3ac270d",
27
  "package_sha256": "a74db8e3196267387e3a4cfb55b838d112a86c88adac031f6515b88230a29fcc",
28
+ "real_patch_tool": {
29
+ "blind_acceptance_cases": 0,
30
+ "exposed_development_cases": 6,
31
+ "hosted_api": false,
32
+ "kind": "local_exact_execution_baseline",
33
+ "learned_real_patch_weights": false,
34
+ "linux_exact_execution_qualified": false,
35
+ "oracle_only_cases": 5,
36
+ "scored_forecast_cases": 1,
37
+ "source_commit": "63cbecbad1bd8832d9c4fae65e4cfad27c57fa76",
38
+ "wheel": "evaluation/real-patch/runtime/laxmi_real_patch-0.1.0.dev1-py3-none-any.whl",
39
+ "wheel_sha256": "a36e136b730b9b941af303ca23dcdaaff0268b3e3dfafab4122730ee45d8da0e"
40
+ },
41
+ "repository_checkpoint_commit": "4dae1a041be532d663f477effe9b654b428bc04e",
42
  "runtime_sources": 133,
43
  "schema_version": "laxmi.experimental-public-package.v1",
44
+ "source_identity_basis": "original runtime wheel SHA256 and per-file checkpoint source locks; repository checkpoint is a recovery anchor, not a claim of identical historical source tree",
45
  "supported_platform": "macOS ARM64, Python 3.12; other platforms unverified",
46
+ "version": "0.1.1"
 
 
 
47
  }