PXR structure prediction: when induced-fit is everything

I recently scored 4th out of 100 participants in the blind OpenADMET PXR structure prediction challenge. My submission landed within the tier-1 band (the top 10 entries, statistically indistinguishable from each other).

This was a blind competition: all participants were given the compound SMILES and the amino acid sequence, and from that had to create and submit 3D structures. These were then scored centrally against the ground truth, without participants ever having access to the test data. This is as close to prospective validation as you can get, making it a great way to fairly benchmark different approaches.

In this blogpost I will break down my three key takeaways, and finish by assessing how well we, as a community, actually performed on PXR.

Takeaway 1: Co-folding beats docking on PXR

PXR is a notoriously difficult target to model. It is an incredibly flexible target: many side chains move, and even the protein backbone often shifts to accommodate the ligand. The crystal structures released when the competition closed illustrate this well. We see in the figure how several side-chains change orientation, and in some places even the backbone shifts (the Cα of S208 shifted 2.9 Å, for example!). The protein’s purpose explains this flexibility, it has evolved to detect foreign molecules and upregulate the production of proteins that metabolise them.

PXR crystal structures for compounds x03015-1 (blue) and x03094-1 (amber) showing the flexible nature of this target.

The participants could choose which method they used for structure prediction. Both co-folding methods and a range of classic docking methods were tested by different teams. This makes it a great test bed: as each team tries to optimise their method to win, we get a fair assessment of what’s possible with different methods on this target. What stands out from the leaderboard is how much better co-folding methods performed on this target. My entry, using co-folding, achieved a LDDT-PLI of 0.518 whereas the best docking method reached 0.390, showing a statistically significant difference (p-value <0.001).

I have seen a similar trend for other flexible targets. In rigid pockets with only minor movements of some side chains, docking performs well. However, in induced-fit situations where side-chains and maybe even the backbone need to move to accommodate the ligand, co-folding methods find better structures than docking.

I believe this is explained by the fact that co-folding simultaneously assigns positions for atoms in the protein and in the ligand. Therefore, it is naturally set up to handle this flexibility. The figure below shows an example of this. There are various workarounds when using docking. For example, several submissions tried docking into multiple conformations. However, this sequential approach is inherently limited: if the space of possible induced-fit conformations is larger than the number you can feasibly dock against, you're likely to miss the right one. More steps mean more places for the pipeline to break, making the overall system more brittle.

Crystal structures for compound x03094-1 and PXR (amber) and co-folded structure with OpenFold3 (purple), LDDT-PLI = 0.77.

Takeaway 2: Combining co-folding with physics may be the future

My entry (position 4) and several of the other best entries (positions 1, 2, and 10) combined co-folding for structure generation and physics for pose scoring. My solution was to let ESMFold2, OpenFold3, and Boltz-2 generate a set of poses each. Then I used an internal physics-based tool I built that identifies likely binders by analysing the co-folded structures. If this tool scored any pose over 75% likelihood of being a binder, I chose the highest-scored pose. Otherwise I fell back to the ESMFold2 predicted poses.

The philosophy is that machine learning is good at suggesting 3D poses. The co-folding models can draw on a large set of hazily memorised binding motifs to suggest different possible conformations. However, when determining whether those poses actually translate into binding events, we need the rigor of physics. Instead of determining binding based on “do I recognise this?”, we want to determine it based on “is the physics there?”. I interpret my high leaderboard placement as good validation that my system works as intended. Seeing other top-scoring participants converge on a similar approach is a promising sign for this direction more generally.

Takeaway 3: Different situations require different co-folding models

The participants also tried several different co-folding methods. I personally tested ESMFold2, OpenFold3, Boltz-2, Chai-1, and Protenix-v2. My final submission consisted of poses from ESMFold2, OpenFold3, and Boltz-2. Once the test set structures were released after the competition, I ran an additional analysis on how the different co-folding models performed on their own. The figure below shows the result (bootstrapped 95% CI). This shows that ESMFold2 and OpenFold3 are slightly better on this target than the others. ESMFold2 is especially ahead on the fragment subset (MW 120-260), while on the larger compounds (MW 240-480), OpenFold3 and ESMFold2 tied.

Given that the exact training data and model architectures differ across these tools, it's expected that different models excel at different tasks. The results presented here should not be used to conclude that one model is always better than another. Instead, I believe it's worth trying a few different co-folding algorithms to find the one that works best for your specific target. I’m grateful to the groups who open source these, giving the community a wide range of choice.

Performance of different co-folding methods (bootstrapped 95% CI).

There’s still a long way to go until we’ve mastered PXR structure prediction

Reflecting on where we ended up, it’s important to be clear-eyed: PXR is not a solved target for structure prediction. Even with 100 participants across the competition, the best entry only achieved an average LDDT-PLI of 0.530. This is a low value. Even the best participants get a large fraction of poses wrong. However, let’s remember that PXR is an unusually difficult protein due to its biological purpose. Most targets are much more constrained in what they can interact with, and hence easier to model.

In parallel with the structure competition, OpenADMET also ran an activity competition. I participated in this competition as well, finishing 14th out of 350 (also landing in tier-1). While some of the top entries, mine included, used co-folding as part of their submissions, most did not. Instead, they all leaned heavily on ligand-only ML methods. This likely reflects the low LDDT-PLI discussed above. If structure prediction isn’t reliable enough, and we have similar compounds to train models on, ligand-only ML methods will dominate; the value provided by co-folding simply won’t be there.

To improve performance on PXR structure prediction, the next step for the field is to fine-tune co-folding models on the freshly published PXR structures. This should have a large impact, and thanks to OpenADMET, the data to make it happen now exists.

Summary

• Co-folding methods beat docking-based methods on PXR. I argue this is explained by PXR’s highly flexible nature, and that the same holds for other flexible targets generally.

• Co-folding combined with physics-based pose scoring is starting to gain momentum as a direction.

• Not all co-folding methods worked equally well. Choosing the right one for the target at hand is important.

• We still have a long way to go to solve PXR structure prediction; the new data released by OpenADMET will certainly help.

Thank you to OpenADMET for organising such an engaging competition, and to all the participants. And thanks for reading!

---

I offer consulting services to help biotechs and startups leverage ML and physics-based computational chemistry methods. If you are interested, please reach out!