OpenBind Blind Challenge 1 method report (KR LAB Bio)

bb94c61 verified about 7 hours ago

21.1 kB

license: cc-by-4.0
language:
  - en
tags:
  - chemistry
  - drug-discovery
  - structural-biology
  - protein-ligand
  - pose-prediction
  - co-folding
  - zika-virus
  - openbind
pretty_name: OpenBind Blind Challenge 1 Method Report (KR LAB Bio)

Zika NS2B-NS3 Protease Complexes from Same-Campaign Templates, Docking and Co-Folding, with Consensus-Based Pose Selection

OpenBind Blind Challenge 1 - Method Report Structure prediction track · Leaderboard alias KR LAB Bio · KR LAB Bio Inc., Republic of Korea Draft v1, 2026-11-06. Describes the submission currently on the leaderboard (internal label v18). No proprietary data was used. No test-set structures or labels were used at any step.

1. Summary

We predicted all 400 blinded NS2B-NS3/ligand complexes with two independent pose generators and one selection procedure.

Pipeline A, template transplantation plus docking. Every public Zika NS2B-NS3 crystal structure (294 PDB entries) was superposed onto the 29HN binding site. For each challenge ligand, the maximum common substructure with the closest template ligands was placed on the template's experimental coordinates, the rest of the molecule was built and relaxed, and the template's own protein was used as the receptor. Exhaustive docking and gnina CNN rescoring and refinement added candidates. A fixed rule selected one pose per ligand from a median of 66 candidates, using spatial consensus between independent templates, recovery of the template contact pattern and the gnina CNN rank.
Pipeline B, co-folding. Protenix (protenix_base_20250630_v1.0.0, training cutoff 2025-06-30, no fine-tuning) produced 10 samples per ligand (2 seeds × 5 diffusion samples). The representative sample is the stereochemically correct sample around which most other samples cluster.
Assembly. The co-folding representative was used whenever it passed all 22 PoseBusters checks (337 ligands). The remaining 63 ligands (39 with no stereochemically correct sample, 24 that failed PoseBusters) used the Pipeline A selection.

Live leaderboard result (about 200 of the 400 compounds, collected 2026-11-06 11:34 UTC):

Metric	Value
Success rate	68.5% (137 of 200)
BiSyRMSD < 1.5 Å	137
lDDT-PLI > 0.8	155
PoseBusters pass	200 / 200
Mean BiSyRMSD	1.70 Å
Mean lDDT-PLI	0.837

Four findings account for most of the work:

The public record already contains this chemical series. 369 of the 400 challenge ligands share at least half of their heavy atoms with a ligand in a public structure (median MCS coverage 0.833), and 391 of the 400 closest templates come from the same crystal form (P4₃2₂) as the challenge. Template transplantation is therefore a strong baseline on its own.
Within our own candidate set, selection plateaued. Three successive changes to the selection rule (sections 4 and 7) each improved a leave-one-compound-out backtest, yet the live BiSyRMSD pass count stayed at 133 for all three submissions.
Co-folding broke that plateau without fine-tuning. Replacing the selected pose by the co-folding representative wherever it passed PoseBusters raised the live BiSyRMSD pass count from 133 to 137 and the mean BiSyRMSD from 1.84 Å to 1.70 Å. On 24 structures released after the model's training cutoff, the co-folding representative was within 1.5 Å for 9 ligands against 5 for Pipeline A.
Backtest gains did not transfer to the live set. A selection rule that won 11 of 12 honest holdout splits by +5.2 percentage points moved the live score by +0.5 points. We explain why in section 7, and we have stopped treating backtest gains alone as evidence of improvement.

Negative results are listed in section 9: the levers we tested on the Pipeline A side, each with the measurement that closed it.

2. Target, inputs and public data

Construct. NS2B cofactor (chain A, 46 residues) and NS3 protease domain (chain B, 168 residues) as given in NS2B-NS3_protein_sequence.fasta. The ligand is a single residue LIG. Every submitted file has three chains or fewer.
Reference frame. The apo structure 29HN. 8PN6, named in the dataset README, is the same construct in the same space group. All coordinates in this work are expressed in the 29HN frame.
Public structures. An RCSB full-text search for Zika NS2B-NS3 returned 294 entries (2026-10-23; the same query on 2026-10-28 returned no entries released after 2026-10-01). From these we extracted 209 ligand instances from 176 entries (174 distinct chemical components). Bonds and bond orders were taken from the PDB Chemical Component Dictionary by atom name rather than inferred from coordinates, which recovered 43 ligands that coordinate-based perception had broken.
Challenge ligands. 400 SMILES, 9 to 36 heavy atoms, median 23; 77.2% have 20 or more heavy atoms.

No proprietary or unpublished data was used. The challenge's own crystal structures were never available to us.

3. Pipeline A: candidate generation

3.1 Binding-site superposition

Each public structure was superposed onto 29HN using backbone atoms within 12 Å of the centre of the catalytic triad (Kabsch). The evaluation superposes binding sites before computing ligand RMSD, so a binding-site fit is the relevant one. When chain identification by triad numbering failed, a residue-number offset was found by sequence alignment and the fit was repeated. Median fit RMSD: 0.15 Å.

3.2 Template matching and transplantation

Morgan fingerprints shortlist templates; the MCS (rings match rings, complete rings only) ranks them by the number of shared heavy atoms. Up to four templates per ligand are kept (397 of 400 ligands have four).
Because of molecular symmetry, an MCS admits several atom mappings. We enumerate them, rigidly align each to the template, and keep the geometrically consistent ones. Using only the first mapping returned by the matcher displaced poses by 4-5 Å; enumerating mappings reduced the MCS RMSD to 0.17 Å.
The common atoms are fixed at the template's experimental coordinates; the rest of the ligand is built with ETKDGv3 and relaxed with MMFF94 in three stages (common atoms fixed, then positionally restrained, then a looser restraint only if geometry is still broken).
The template's own crystal structure, trimmed to the two protease chains nearest the active site, is the receptor. Its side chains are already arranged around a closely related ligand.
smina (Vinardo) local minimisation against that receptor removes clashes while keeping bond lengths and angles as built.

3.3 Docking and CNN rescoring

smina (Vinardo), 22 Å box on the active site, exhaustiveness 16, 9 modes, against the template receptors. Docking candidates matter: removing them lowered the backtest from 58.1% to 48.0%, and to half in the lowest-similarity stratum.
gnina 1.3.2: every candidate is rescored in place (--score_only --cnn_scoring rescore), and the 12 best-ranked candidates per ligand are additionally refined with --minimize (median movement 0.26 Å; all refined poses kept valid geometry).

3.4 Validity gate

Every candidate considered for submission passes two checks in order:

a PDB round trip identical to the path of the official validator. PDB files carry no bond orders, so the reader perceives bonds from coordinates; a pose with a stretched ring can gain a bond and fail validation even after passing PoseBusters;
PoseBusters 0.6.5, the official redock configuration minus the RMSD checks (22 checks), with the molecule built from the challenge SMILES in place of mol_true, so that formula, bonds, double-bond stereochemistry and tetrahedral chirality are checked against the input.

4. Pipeline A: pose selection

4.1 The rule

Each candidate gets a score; lower is better. Candidates are walked in score order and the first one that passes the validity gate is selected.

Term	Weight	Meaning
spatial consensus	1.0	fraction of the ligand's other candidates within 1.2 Å RMSD
cross-template consensus	2.0	the same fraction counted only over candidates derived from other templates
contact recovery	1.0	fraction of the residues commonly contacted by the template ligands (median 14-15 residues) that this pose also contacts
contact Jaccard	0.5	Jaccard index between the pose's contact set and that reference set
gnina CNNscore rank	0.3	rank within the ligand's candidates

Signs are set so that more consensus and more contact recovery lower the score. Cross-template consensus carries the most weight because templates come from independent crystal structures: agreement between them is evidence that does not come from the scoring function. The CNN term uses the within-ligand rank, because CNNscore ranges differ between ligands and absolute values would let a few ligands dominate the rule.

4.2 Consensus nudge

After selection and validation, the selected pose alone is moved a quarter of the way toward the atom-wise mean of the candidates within 3 Å, then relaxed with MMFF under a strong positional restraint (force constant 200, tolerance 0.05 Å) and validated again; if it fails, the unmoved pose is submitted. Median movement 0.14 Å, maximum 0.38 Å. In v18 this applies to 60 of the 63 ligands that use Pipeline A.

The order matters. Our first version nudged the top-ranked candidates before selection, which inflated the consensus terms (the median spatial consensus of selected poses rose from 0.36 to 0.56) and reshuffled the selection. Selecting first and nudging only the selected pose keeps the selection unchanged.

5. Pipeline B: co-folding

Model. Protenix 2.0.0 with the protenix_base_20250630_v1.0.0 checkpoint (training cutoff 2025-06-30). Not fine-tuned.
Inputs. The two protease chains and the ligand SMILES. A single MSA, computed once with the Protenix MSA server, was reused for all inputs since the protein is identical.
Sampling. Seeds 101 and 102, five diffusion samples each, ten samples per ligand (five for five ligands where the second seed returned no output). About 13 seconds per input and seed on one RTX 4090.
Conversion. Each predicted complex is superposed onto the 29HN binding site after a sequence-derived renumbering (NS3 +9, NS2B +43). Ligand atoms are mapped to the SMILES atom order, and each sample is flagged for whether its stereochemistry matches the SMILES. The co-folded protein, in the 29HN frame and numbering, is kept as that sample's receptor.
Representative sample. Among samples with correct stereochemistry, the one with the largest number of other samples within 2.0 Å symmetry-corrected RMSD. For the 337 ligands where it was used, the median fraction of all ten samples within 2.0 Å of it was 1.0.
Gate. The representative must pass the same validity gate (section 3.4). If it fails, its geometry is corrected with a strongly restrained MMFF minimisation and retried; receptors are tried in the order co-folded protein, Pipeline A receptor, receptor of the nearest Pipeline A candidate, 29HN.

On the training cutoff. A 2025-06-30 cutoff covers 183 of the 209 public template ligands. The challenge structures themselves are unpublished. This is the same situation as the organisers' OpenBind-0 baseline, an OpenFold3 checkpoint with the same cutoff (64.5% on the live leaderboard). We therefore measured co-folding against our own pipeline on structures released after the cutoff (section 7.3).

6. The submitted assembly (v18)

Source of submitted pose	Ligands
Co-folding representative	337
Pipeline A, no co-folding sample with correct stereochemistry	39
Pipeline A, co-folding representative failed PoseBusters	24
of the 63 Pipeline A poses: template transplant + smina	39
of the 63: docking	22
of the 63: template transplant, unminimised	2

PoseBusters 22 checks: 400 / 400.
Official validator (validation/structure_validation.py, tutorial repository commit 1b34ec2): valid, 0 errors, 400 files.

7. Validation and what it could and could not tell us

7.1 Backtest design

We had no access to challenge structures, so every design decision on the Pipeline A side was measured on public complexes:

Leave-one-compound-out. For each of 145 public complexes (one per chemical component, best resolution), every structure containing that component was removed from the template library before predicting its pose. 145 is the ceiling: of the 174 components, 28 have no remaining template with enough MCS overlap to produce any pose.
Challenge-distribution weighting. The backtest set is dominated by fragments (40% below 15 heavy atoms against 14.8% in the challenge), and leave-one-out makes templates more distant than in the challenge (median best Tanimoto 0.391 against 0.520). Unweighted, the backtest underestimated the first live score by 25 points. We weight backtest compounds by heavy-atom and similarity strata to match the challenge.
Honest holdout. To estimate the optimism of choosing a rule on the same data used to score it, we split the 145 compounds in half twelve times, re-selected every distance, weight and criterion on one half, and scored on the other.

7.2 The selection rule: backtest against live

Submission	Selection	Backtest (145, weighted)	Holdout, 12 splits	Live success	Live BiSyRMSD pass
v7	gnina CNN rank only	43.6%	43.6%	66.0%	133
v11	rule of section 4.1	51.7%	48.9% (wins 11 of 12)	66.5%	133
v16	v11 with a feature-reading defect fixed, plus nudge	see note	-	66.5%	133
v18	co-folding where valid, v16 elsewhere	not measurable	-	68.5%	137

Note on v16: the submission code had not been reading the template identity of each pose, so the cross-template term (weight 2.0) was zero in every submission from v10 to v15 while the rule had been fitted with it. Fixing this moved the backtest from 51.6% to 54.0% under a counting that holds similarity strata fixed per compound, and moved the live score by zero.

Three explanations are compatible with these numbers and we cannot separate them:

Resolution. The live set is about 200 compounds; one compound is 0.5 points.
Leave-one-out is harder than the challenge. With close templates available, as in the challenge, the CNN rank alone already picks well (66.0% live), and consensus has less to add. The advantage of consensus appeared mainly where templates were distant.
Weighting is incomplete. Similarity-stratum weights were capped at 4.0.

After v11 we stopped treating a backtest gain alone as evidence of improvement, and we compare methods by submitting whole versions and reading the change in the live score. We never changed individual compounds to probe the live score, since the final ranking uses all 400.

7.3 Co-folding against Pipeline A

Evaluation set	Pipeline A	Co-folding representative	Either	Both
24 structures released after 2025-06-30 (2025-cutoff model; 17 of 24 are fragments)	5	9	10	4
145-compound backtest (2021-09-30-cutoff model `protenix_base_default_v1.0.0`, which saw 22 of the 209 templates)	63	49	79	33

Counts are ligands within 1.5 Å. The two pipelines succeed on different compounds: together they would reach 79 of 145. We found no signal that tells, for a given compound, which one is right when they disagree. Areas under the ROC curve for that decision were 0.66 for co-folding self-consistency, 0.60 for ligand pLDDT, 0.58 for ranking score and ipTM, and 0.48-0.54 for the Pipeline A consensus and contact terms. A conditional switching rule (switch only when self-consistency is at least 0.6 and cross-template consensus is below 0.15) went from 5 to 8 on the 24-structure set and from 63 to 60 on the 145-compound backtest.

The 2025-cutoff model on the challenge is a different situation from either set, because it was trained on earlier structures from the same series. Only the live leaderboard could measure it, which is why v18 uses co-folding unconditionally wherever it is valid.

8. Leaderboard history

Version	Submitted (UTC)	Change	Success	BiSyRMSD pass	lDDT-PLI pass	Mean BiSyRMSD
v7	2026-10-26 06:30	Pipeline A, gnina CNN rank selection	66.0%	133	149	1.84 Å
v11	2026-10-26 18:32	selection rule of section 4.1	66.5%	133	150	1.83 Å
v16	2026-10-28 11:48	feature-reading fix, consensus nudge	66.5%	133	152	1.84 Å
v18	2026-10-28 23:51	co-folding where valid	68.5%	137	155	1.70 Å

PoseBusters was 200 / 200 for every version.

9. What did not work

All measured on the weighted 145-compound backtest unless stated. The reference is the rule of section 4.1 at 58.1% on the 120-compound set used at the time, or the value given in the row.

Candidate generation

Tried	Result
Eight templates per ligand instead of four	65% to 55% on the subset regenerated for this test. The ceiling stayed at 94%; candidates rose from 54 to 96 per ligand and distant templates formed spurious consensus
One, two or three templates	49.5-51.9%
Filtering templates by MCS coverage	54.1-56.1%
Docking at exhaustiveness 64 and 20 modes	65% to 58%; same ceiling, more candidates dilute the selection
Removing docking candidates	58.1% to 48.0%
Boltz-2 co-folding without fine-tuning (fragment probe)	10.7%, below Pipeline A

Selection features

Tried	Result
Rescoring on several receptors (mean, minimum, affinity, VS score)	46.7-55.5%
Burial of ligand atoms	46.0%
Distance to the template-ligand hotspot	43.1%
Hydrogen bonds, salt bridges, hydrophobic contacts (geometric definitions)	41.4-51.1%
Ensemble score spread, template resolution, transplant shift, MCS RMSD	all harmful
Learned models on 32 features (logistic regression, gradient boosting, random forest)	45.7-53.1%
Pairwise ranking with learned linear weights	47.2% (learned order matched our rule)

The physical and chemical terms separate correct from incorrect poses at the pose level (for example 4 against 3 hydrogen bonds, 73 against 50 hydrophobic contacts) but lower the success rate when added to the rule. Favouring more contacts selects over-buried poses; we read this as the balance already being encoded in the gnina CNN.

Selection procedure

Tried	Result
Weighting consensus by CNN score or contact recovery	56.1-57.0% (plain counts are better)
Two-step selection (site first, then representative)	47.4-50.0%
Medoid of the top candidates	50.9-56.8%
Cross-template consensus counted over template families	53.1-54.8%
Docking poses first for fragments (fewer than 18 heavy atoms)	54.0% to 51.8% after the v16 fix
Co-folding samples as additional consensus votes (weights 0.5-8, radii 1.5-3 Å; 2021-cutoff model)	53-59 of 145 ligands against 61

Fragments remain the hardest class. In our early 120-compound backtest, compounds below 18 heavy atoms succeeded at 9-29% against 50-71% above, although the best candidate is often within 0.3-0.8 Å. A fragment has few contacts, and the same contacts are reproduced at several positions in the pocket. This matches the live gap between lDDT-PLI passes (155) and BiSyRMSD passes (137).

10. Reproducibility and scope

Component	Version
RDKit	2026.09.1
PoseBusters	0.6.5 (same as the organisers' baseline reports)
smina	2017.11.9 (Vinardo scoring)
gnina	1.3.2 static release (`master:f23dd2b`)
Protenix	2.0.0, checkpoint `protenix_base_20250630_v1.0.0`
Official validator and scoring code	`OpenADMET/OpenBind-Challenge-1-Tutorial` at commit `1b34ec2`

Compute. Template transplantation, docking and selection ran on CPU nodes; gnina rescoring and refinement on an 8 GB RTX 4060. Co-folding ran on one rented RTX 4090 for about eight hours, at a cost of about USD 4.

Code. The code is not published at this time. This report is intended to describe every step at a level that allows reimplementation with the public tools listed above. Parameters not stated here were left at the tools' defaults.

Data use. Only public PDB entries, the challenge inputs and public model checkpoints were used. No test-set structure, label or per-compound score entered any step. The live leaderboard was used only to compare whole submissions.

11. Contact

KR LAB Bio Inc., Republic of Korea. Leaderboard alias: KR LAB Bio.