| --- |
| license: cc-by-4.0 |
| language: |
| - en |
| tags: |
| - chemistry |
| - drug-discovery |
| - structural-biology |
| - protein-ligand |
| - pose-prediction |
| - co-folding |
| - zika-virus |
| - openbind |
| pretty_name: OpenBind Blind Challenge 1 Method Report (KR LAB Bio) |
| --- |
| |
| # Zika NS2B-NS3 Protease Complexes from Same-Campaign Templates, Docking and Co-Folding, with Consensus-Based Pose Selection |
|
|
| **OpenBind Blind Challenge 1 - Method Report** |
| Structure prediction track · Leaderboard alias **KR LAB Bio** · KR LAB Bio Inc., Republic of Korea |
| Draft v1, 2026-11-06. Describes the submission currently on the leaderboard (internal label v18). |
| No proprietary data was used. No test-set structures or labels were used at any step. |
|
|
| --- |
|
|
| ## 1. Summary |
|
|
| We predicted all 400 blinded NS2B-NS3/ligand complexes with two independent pose generators |
| and one selection procedure. |
|
|
| - **Pipeline A, template transplantation plus docking.** Every public Zika NS2B-NS3 crystal |
| structure (294 PDB entries) was superposed onto the 29HN binding site. For each challenge |
| ligand, the maximum common substructure with the closest template ligands was placed on the |
| template's experimental coordinates, the rest of the molecule was built and relaxed, and the |
| template's own protein was used as the receptor. Exhaustive docking and gnina CNN rescoring |
| and refinement added candidates. A fixed rule selected one pose per ligand from a median of |
| 66 candidates, using spatial consensus between independent templates, recovery of the |
| template contact pattern and the gnina CNN rank. |
| - **Pipeline B, co-folding.** Protenix (`protenix_base_20250630_v1.0.0`, training cutoff |
| 2025-06-30, no fine-tuning) produced 10 samples per ligand (2 seeds × 5 diffusion samples). |
| The representative sample is the stereochemically correct sample around which most other |
| samples cluster. |
| - **Assembly.** The co-folding representative was used whenever it passed all 22 PoseBusters |
| checks (337 ligands). The remaining 63 ligands (39 with no stereochemically correct sample, |
| 24 that failed PoseBusters) used the Pipeline A selection. |
|
|
| Live leaderboard result (about 200 of the 400 compounds, collected 2026-11-06 11:34 UTC): |
|
|
| | Metric | Value | |
| |---|---| |
| | Success rate | **68.5%** (137 of 200) | |
| | BiSyRMSD < 1.5 Å | 137 | |
| | lDDT-PLI > 0.8 | 155 | |
| | PoseBusters pass | 200 / 200 | |
| | Mean BiSyRMSD | 1.70 Å | |
| | Mean lDDT-PLI | 0.837 | |
|
|
| Four findings account for most of the work: |
|
|
| 1. **The public record already contains this chemical series.** 369 of the 400 challenge |
| ligands share at least half of their heavy atoms with a ligand in a public structure |
| (median MCS coverage 0.833), and 391 of the 400 closest templates come from the same |
| crystal form (P4<sub>3</sub>2<sub>2</sub>) as the challenge. Template transplantation is |
| therefore a strong baseline on its own. |
| 2. **Within our own candidate set, selection plateaued.** Three successive changes to the |
| selection rule (sections 4 and 7) each improved a leave-one-compound-out backtest, yet the |
| live BiSyRMSD pass count stayed at 133 for all three submissions. |
| 3. **Co-folding broke that plateau without fine-tuning.** Replacing the selected pose by the |
| co-folding representative wherever it passed PoseBusters raised the live BiSyRMSD pass |
| count from 133 to 137 and the mean BiSyRMSD from 1.84 Å to 1.70 Å. On 24 structures |
| released after the model's training cutoff, the co-folding representative was within 1.5 Å |
| for 9 ligands against 5 for Pipeline A. |
| 4. **Backtest gains did not transfer to the live set.** A selection rule that won 11 of 12 |
| honest holdout splits by +5.2 percentage points moved the live score by +0.5 points. We |
| explain why in section 7, and we have stopped treating backtest gains alone as evidence of |
| improvement. |
|
|
| Negative results are listed in section 9: the levers we tested on the Pipeline A side, each |
| with the measurement that closed it. |
|
|
| --- |
|
|
| ## 2. Target, inputs and public data |
|
|
| - **Construct.** NS2B cofactor (chain A, 46 residues) and NS3 protease domain (chain B, |
| 168 residues) as given in `NS2B-NS3_protein_sequence.fasta`. The ligand is a single residue |
| `LIG`. Every submitted file has three chains or fewer. |
| - **Reference frame.** The apo structure 29HN. 8PN6, named in the dataset README, is the same |
| construct in the same space group. All coordinates in this work are expressed in the 29HN |
| frame. |
| - **Public structures.** An RCSB full-text search for Zika NS2B-NS3 returned 294 entries |
| (2026-10-23; the same query on 2026-10-28 returned no entries released after 2026-10-01). |
| From these we extracted **209 ligand instances from 176 entries (174 distinct chemical |
| components)**. Bonds and bond orders were taken from the PDB Chemical Component Dictionary |
| by atom name rather than inferred from coordinates, which recovered 43 ligands that |
| coordinate-based perception had broken. |
| - **Challenge ligands.** 400 SMILES, 9 to 36 heavy atoms, median 23; 77.2% have 20 or more |
| heavy atoms. |
|
|
| No proprietary or unpublished data was used. The challenge's own crystal structures were never |
| available to us. |
|
|
| --- |
|
|
| ## 3. Pipeline A: candidate generation |
|
|
| ### 3.1 Binding-site superposition |
|
|
| Each public structure was superposed onto 29HN using backbone atoms within 12 Å of the centre |
| of the catalytic triad (Kabsch). The evaluation superposes binding sites before computing |
| ligand RMSD, so a binding-site fit is the relevant one. When chain identification by triad |
| numbering failed, a residue-number offset was found by sequence alignment and the fit was |
| repeated. Median fit RMSD: 0.15 Å. |
|
|
| ### 3.2 Template matching and transplantation |
|
|
| 1. Morgan fingerprints shortlist templates; the MCS (rings match rings, complete rings only) |
| ranks them by the number of shared heavy atoms. Up to four templates per ligand are kept |
| (397 of 400 ligands have four). |
| 2. Because of molecular symmetry, an MCS admits several atom mappings. We enumerate them, |
| rigidly align each to the template, and keep the geometrically consistent ones. Using only |
| the first mapping returned by the matcher displaced poses by 4-5 Å; enumerating mappings |
| reduced the MCS RMSD to 0.17 Å. |
| 3. The common atoms are fixed at the template's experimental coordinates; the rest of the |
| ligand is built with ETKDGv3 and relaxed with MMFF94 in three stages (common atoms fixed, |
| then positionally restrained, then a looser restraint only if geometry is still broken). |
| 4. The template's own crystal structure, trimmed to the two protease chains nearest the active |
| site, is the receptor. Its side chains are already arranged around a closely related ligand. |
| 5. smina (Vinardo) local minimisation against that receptor removes clashes while keeping bond |
| lengths and angles as built. |
|
|
| ### 3.3 Docking and CNN rescoring |
|
|
| - smina (Vinardo), 22 Å box on the active site, exhaustiveness 16, 9 modes, against the |
| template receptors. Docking candidates matter: removing them lowered the backtest from |
| 58.1% to 48.0%, and to half in the lowest-similarity stratum. |
| - gnina 1.3.2: every candidate is rescored in place (`--score_only --cnn_scoring rescore`), |
| and the 12 best-ranked candidates per ligand are additionally refined with `--minimize` |
| (median movement 0.26 Å; all refined poses kept valid geometry). |
|
|
| ### 3.4 Validity gate |
|
|
| Every candidate considered for submission passes two checks in order: |
|
|
| - a **PDB round trip** identical to the path of the official validator. PDB files carry no |
| bond orders, so the reader perceives bonds from coordinates; a pose with a stretched ring can |
| gain a bond and fail validation even after passing PoseBusters; |
| - **PoseBusters 0.6.5**, the official `redock` configuration minus the RMSD checks |
| (22 checks), with the molecule built from the challenge SMILES in place of `mol_true`, so |
| that formula, bonds, double-bond stereochemistry and tetrahedral chirality are checked |
| against the input. |
|
|
| --- |
|
|
| ## 4. Pipeline A: pose selection |
|
|
| ### 4.1 The rule |
|
|
| Each candidate gets a score; lower is better. Candidates are walked in score order and the |
| first one that passes the validity gate is selected. |
|
|
| | Term | Weight | Meaning | |
| |---|---|---| |
| | spatial consensus | 1.0 | fraction of the ligand's other candidates within 1.2 Å RMSD | |
| | cross-template consensus | 2.0 | the same fraction counted only over candidates derived from *other* templates | |
| | contact recovery | 1.0 | fraction of the residues commonly contacted by the template ligands (median 14-15 residues) that this pose also contacts | |
| | contact Jaccard | 0.5 | Jaccard index between the pose's contact set and that reference set | |
| | gnina CNNscore rank | 0.3 | rank within the ligand's candidates | |
|
|
| Signs are set so that more consensus and more contact recovery lower the score. |
| Cross-template consensus carries the most weight because templates come from independent |
| crystal structures: agreement between them is evidence that does not come from the scoring |
| function. The CNN term uses the within-ligand rank, because CNNscore ranges differ between |
| ligands and absolute values would let a few ligands dominate the rule. |
|
|
| ### 4.2 Consensus nudge |
|
|
| After selection and validation, the selected pose alone is moved a quarter of the way toward |
| the atom-wise mean of the candidates within 3 Å, then relaxed with MMFF under a strong |
| positional restraint (force constant 200, tolerance 0.05 Å) and validated again; if it fails, |
| the unmoved pose is submitted. Median movement 0.14 Å, maximum 0.38 Å. In v18 this applies to |
| 60 of the 63 ligands that use Pipeline A. |
|
|
| The order matters. Our first version nudged the top-ranked candidates before selection, which |
| inflated the consensus terms (the median spatial consensus of selected poses rose from 0.36 to |
| 0.56) and reshuffled the selection. Selecting first and nudging only the selected pose keeps |
| the selection unchanged. |
|
|
| --- |
|
|
| ## 5. Pipeline B: co-folding |
|
|
| - **Model.** Protenix 2.0.0 with the `protenix_base_20250630_v1.0.0` checkpoint (training |
| cutoff 2025-06-30). Not fine-tuned. |
| - **Inputs.** The two protease chains and the ligand SMILES. A single MSA, computed once with |
| the Protenix MSA server, was reused for all inputs since the protein is identical. |
| - **Sampling.** Seeds 101 and 102, five diffusion samples each, ten samples per ligand |
| (five for five ligands where the second seed returned no output). About 13 seconds per input |
| and seed on one RTX 4090. |
| - **Conversion.** Each predicted complex is superposed onto the 29HN binding site after a |
| sequence-derived renumbering (NS3 +9, NS2B +43). Ligand atoms are mapped to the SMILES atom |
| order, and each sample is flagged for whether its stereochemistry matches the SMILES. The |
| co-folded protein, in the 29HN frame and numbering, is kept as that sample's receptor. |
| - **Representative sample.** Among samples with correct stereochemistry, the one with the |
| largest number of other samples within 2.0 Å symmetry-corrected RMSD. For the 337 ligands |
| where it was used, the median fraction of all ten samples within 2.0 Å of it was 1.0. |
| - **Gate.** The representative must pass the same validity gate (section 3.4). If it fails, its |
| geometry is corrected with a strongly restrained MMFF minimisation and retried; receptors are |
| tried in the order co-folded protein, Pipeline A receptor, receptor of the nearest |
| Pipeline A candidate, 29HN. |
|
|
| **On the training cutoff.** A 2025-06-30 cutoff covers 183 of the 209 public template |
| ligands. The challenge structures themselves are unpublished. This is the same situation as the |
| organisers' OpenBind-0 baseline, an OpenFold3 checkpoint with the same cutoff (64.5% on the live |
| leaderboard). We therefore measured co-folding against our own pipeline on structures released |
| after the cutoff (section 7.3). |
|
|
| --- |
|
|
| ## 6. The submitted assembly (v18) |
|
|
| | Source of submitted pose | Ligands | |
| |---|---| |
| | Co-folding representative | 337 | |
| | Pipeline A, no co-folding sample with correct stereochemistry | 39 | |
| | Pipeline A, co-folding representative failed PoseBusters | 24 | |
| | *of the 63 Pipeline A poses:* template transplant + smina | 39 | |
| | *of the 63:* docking | 22 | |
| | *of the 63:* template transplant, unminimised | 2 | |
|
|
| - PoseBusters 22 checks: 400 / 400. |
| - Official validator (`validation/structure_validation.py`, tutorial repository commit |
| `1b34ec2`): valid, 0 errors, 400 files. |
|
|
| --- |
|
|
| ## 7. Validation and what it could and could not tell us |
|
|
| ### 7.1 Backtest design |
|
|
| We had no access to challenge structures, so every design decision on the Pipeline A side was |
| measured on public complexes: |
|
|
| - **Leave-one-compound-out.** For each of 145 public complexes (one per chemical component, |
| best resolution), every structure containing that component was removed from the template |
| library before predicting its pose. 145 is the ceiling: of the 174 components, 28 have no |
| remaining template with enough MCS overlap to produce any pose. |
| - **Challenge-distribution weighting.** The backtest set is dominated by fragments (40% below |
| 15 heavy atoms against 14.8% in the challenge), and leave-one-out makes templates more |
| distant than in the challenge (median best Tanimoto 0.391 against 0.520). Unweighted, the |
| backtest underestimated the first live score by 25 points. We weight backtest compounds by |
| heavy-atom and similarity strata to match the challenge. |
| - **Honest holdout.** To estimate the optimism of choosing a rule on the same data used to |
| score it, we split the 145 compounds in half twelve times, re-selected every distance, |
| weight and criterion on one half, and scored on the other. |
|
|
| ### 7.2 The selection rule: backtest against live |
|
|
| | Submission | Selection | Backtest (145, weighted) | Holdout, 12 splits | Live success | Live BiSyRMSD pass | |
| |---|---|---|---|---|---| |
| | v7 | gnina CNN rank only | 43.6% | 43.6% | 66.0% | 133 | |
| | v11 | rule of section 4.1 | 51.7% | 48.9% (wins 11 of 12) | 66.5% | 133 | |
| | v16 | v11 with a feature-reading defect fixed, plus nudge | see note | - | 66.5% | 133 | |
| | **v18** | co-folding where valid, v16 elsewhere | not measurable | - | **68.5%** | **137** | |
|
|
| Note on v16: the submission code had not been reading the template identity of each pose, so |
| the cross-template term (weight 2.0) was zero in every submission from v10 to v15 while the |
| rule had been fitted with it. Fixing this moved the backtest from 51.6% to 54.0% under a |
| counting that holds similarity strata fixed per compound, and moved the live score by zero. |
|
|
| Three explanations are compatible with these numbers and we cannot separate them: |
|
|
| 1. **Resolution.** The live set is about 200 compounds; one compound is 0.5 points. |
| 2. **Leave-one-out is harder than the challenge.** With close templates available, as in the |
| challenge, the CNN rank alone already picks well (66.0% live), and consensus has less to |
| add. The advantage of consensus appeared mainly where templates were distant. |
| 3. **Weighting is incomplete.** Similarity-stratum weights were capped at 4.0. |
|
|
| After v11 we stopped treating a backtest gain alone as evidence of improvement, and we compare |
| methods by submitting whole versions and reading the change in the live score. We never changed |
| individual compounds to probe the live score, since the final ranking uses all 400. |
|
|
| ### 7.3 Co-folding against Pipeline A |
|
|
| | Evaluation set | Pipeline A | Co-folding representative | Either | Both | |
| |---|---|---|---|---| |
| | 24 structures released after 2025-06-30 (2025-cutoff model; 17 of 24 are fragments) | 5 | **9** | 10 | 4 | |
| | 145-compound backtest (2021-09-30-cutoff model `protenix_base_default_v1.0.0`, which saw 22 of the 209 templates) | **63** | 49 | 79 | 33 | |
|
|
| Counts are ligands within 1.5 Å. The two pipelines succeed on different compounds: together they |
| would reach 79 of 145. We found no signal that tells, for a given compound, which one is right |
| when they disagree. Areas under the ROC curve for that decision were 0.66 for co-folding |
| self-consistency, 0.60 for ligand pLDDT, 0.58 for ranking score and ipTM, and 0.48-0.54 for the |
| Pipeline A consensus and contact terms. A conditional switching rule (switch only when |
| self-consistency is at least 0.6 and cross-template consensus is below 0.15) went from 5 to 8 on |
| the 24-structure set and from 63 to 60 on the 145-compound backtest. |
|
|
| The 2025-cutoff model on the challenge is a different situation from either set, because it was |
| trained on earlier structures from the same series. Only the live leaderboard could measure it, |
| which is why v18 uses co-folding unconditionally wherever it is valid. |
|
|
| --- |
|
|
| ## 8. Leaderboard history |
|
|
| | Version | Submitted (UTC) | Change | Success | BiSyRMSD pass | lDDT-PLI pass | Mean BiSyRMSD | |
| |---|---|---|---|---|---|---| |
| | v7 | 2026-10-26 06:30 | Pipeline A, gnina CNN rank selection | 66.0% | 133 | 149 | 1.84 Å | |
| | v11 | 2026-10-26 18:32 | selection rule of section 4.1 | 66.5% | 133 | 150 | 1.83 Å | |
| | v16 | 2026-10-28 11:48 | feature-reading fix, consensus nudge | 66.5% | 133 | 152 | 1.84 Å | |
| | **v18** | 2026-10-28 23:51 | co-folding where valid | **68.5%** | **137** | **155** | **1.70 Å** | |
|
|
| PoseBusters was 200 / 200 for every version. |
|
|
| --- |
|
|
| ## 9. What did not work |
|
|
| All measured on the weighted 145-compound backtest unless stated. The reference is the rule of |
| section 4.1 at 58.1% on the 120-compound set used at the time, or the value given in the row. |
|
|
| **Candidate generation** |
|
|
| | Tried | Result | |
| |---|---| |
| | Eight templates per ligand instead of four | 65% to 55% on the subset regenerated for this test. The ceiling stayed at 94%; candidates rose from 54 to 96 per ligand and distant templates formed spurious consensus | |
| | One, two or three templates | 49.5-51.9% | |
| | Filtering templates by MCS coverage | 54.1-56.1% | |
| | Docking at exhaustiveness 64 and 20 modes | 65% to 58%; same ceiling, more candidates dilute the selection | |
| | Removing docking candidates | 58.1% to 48.0% | |
| | Boltz-2 co-folding without fine-tuning (fragment probe) | 10.7%, below Pipeline A | |
|
|
| **Selection features** |
|
|
| | Tried | Result | |
| |---|---| |
| | Rescoring on several receptors (mean, minimum, affinity, VS score) | 46.7-55.5% | |
| | Burial of ligand atoms | 46.0% | |
| | Distance to the template-ligand hotspot | 43.1% | |
| | Hydrogen bonds, salt bridges, hydrophobic contacts (geometric definitions) | 41.4-51.1% | |
| | Ensemble score spread, template resolution, transplant shift, MCS RMSD | all harmful | |
| | Learned models on 32 features (logistic regression, gradient boosting, random forest) | 45.7-53.1% | |
| | Pairwise ranking with learned linear weights | 47.2% (learned order matched our rule) | |
|
|
| The physical and chemical terms separate correct from incorrect poses at the pose level (for |
| example 4 against 3 hydrogen bonds, 73 against 50 hydrophobic contacts) but lower the success |
| rate when added to the rule. Favouring more contacts selects over-buried poses; we read this as |
| the balance already being encoded in the gnina CNN. |
|
|
| **Selection procedure** |
|
|
| | Tried | Result | |
| |---|---| |
| | Weighting consensus by CNN score or contact recovery | 56.1-57.0% (plain counts are better) | |
| | Two-step selection (site first, then representative) | 47.4-50.0% | |
| | Medoid of the top candidates | 50.9-56.8% | |
| | Cross-template consensus counted over template families | 53.1-54.8% | |
| | Docking poses first for fragments (fewer than 18 heavy atoms) | 54.0% to 51.8% after the v16 fix | |
| | Co-folding samples as additional consensus votes (weights 0.5-8, radii 1.5-3 Å; 2021-cutoff model) | 53-59 of 145 ligands against 61 | |
|
|
| **Fragments remain the hardest class.** In our early 120-compound backtest, compounds below |
| 18 heavy atoms succeeded at 9-29% against 50-71% above, although the best candidate is often within 0.3-0.8 Å. A |
| fragment has few contacts, and the same contacts are reproduced at several positions in the |
| pocket. This matches the live gap between lDDT-PLI passes (155) and BiSyRMSD passes (137). |
|
|
| --- |
|
|
| ## 10. Reproducibility and scope |
|
|
| | Component | Version | |
| |---|---| |
| | RDKit | 2026.09.1 | |
| | PoseBusters | 0.6.5 (same as the organisers' baseline reports) | |
| | smina | 2017.11.9 (Vinardo scoring) | |
| | gnina | 1.3.2 static release (`master:f23dd2b`) | |
| | Protenix | 2.0.0, checkpoint `protenix_base_20250630_v1.0.0` | |
| | Official validator and scoring code | `OpenADMET/OpenBind-Challenge-1-Tutorial` at commit `1b34ec2` | |
|
|
| **Compute.** Template transplantation, docking and selection ran on CPU nodes; gnina rescoring |
| and refinement on an 8 GB RTX 4060. Co-folding ran on one rented RTX 4090 for about eight hours, |
| at a cost of about USD 4. |
|
|
| **Code.** The code is not published at this time. This report is intended to describe every |
| step at a level that allows reimplementation with the public tools listed above. Parameters |
| not stated here were left at the tools' defaults. |
|
|
| **Data use.** Only public PDB entries, the challenge inputs and public model checkpoints were |
| used. No test-set structure, label or per-compound score entered any step. The live leaderboard |
| was used only to compare whole submissions. |
|
|
| --- |
|
|
| ## 11. Contact |
|
|
| KR LAB Bio Inc., Republic of Korea. Leaderboard alias: KR LAB Bio. |
|
|
