Here’s a true story from a number of years ago. A postdoc in the group comes to me in frustration. He has built a cool “semi-atomistic” coarse-grained protein model that has generated disappointing results. An alpha helix that’s clearly resolved in the X-ray structure of his protein completely unravels. Disappointment. But playing the optimistic supervisor, I ask, “Are we sure you’re wrong? Could that helix be marginally stable?” Further digging revealed an isoform of the protein where the helix in question was not resolvable via X-ray. Relief! I was pretty pleased with myself, I must say.
But now I’m disappointed that I was pleased.
The computational molecular biophysics community has well understood its fundamental limitations (sampling, mainly, but force field too) yet relishes those chances to declare victory when we can make a “story” that appears to shed light on an experimental result. To put the most negative gloss on this, we’re happy when we’re “right” for whatever the reason and when we’re “wrong” we may not even attempt to publish the results. And, no, I can’t claim to be completely innocent of these tendencies.
Given the severe limitations on sampling that remain with us even now, and which were ridiculous in years past, how many of our “right” results really were right for the right reasons? Count me as a skeptic. Whenever you start getting too optimistic about sampling, remember the Shaw group’s msec BPTI simulation, when new behaviors emerged only after hundreds of microseconds – in a small protein not thought to behave dynamically!
Let’s pursue the issue of chasing biological findings a bit more. There are a couple of issues here. The low-hanging fruit is that experiments can be wrongly performed and raw data can be misinterpreted. It’s difficult to map experimental finding unambiguously to computable observables. The reproducibility crisis is well-documented.
But let’s go a step further and ask about possible preconceptions motivating the design and interpretation of experiments. Ask yourself how much biologists know and don’t know about cellular behavior. The list of til-recently-misunderstood elements of cell biology could fill an entire essay: proteins with multiple functions depending on context, RNA molecules playing roles entirely different from mRNA or tRNA, the critical role of non-coding DNA regions (formerly ‘junk DNA’), the role of disorder in structural biology, the roles of phase-separated (non-membrane-bound) cellular regions, and so much more. In biology, anything that can happen, and which aids evolutionary fitness, will happen.
So to review, we gauge the value of our poorly sampled and somewhat inaccurate simulations by comparing to the results of potentially flawed experiments whose design and interpretation may rely on fundamentally mistaken assumptions. That’s cynical, I know, but it’s not crazy.
Where do we want to be as a field? I think we want to be predicters, discoverers, and understanders. If we chase the results of others and publish once we have a “story” then where are we?
Ironically, although in the past, our experimental colleagues were almost always ready to point out the limitations of atomistic simulations, now the culture has changed. MD simulation is often seen as essential to round out a high-profile experimental structural biology paper or well-designed grant proposal. I have mixed feelings about this. MD can certainly provide insights about small-scale dynamics, but I believe sampling limitations predominate, especially for the larger systems of interest today. I try to explain these limitations to potential collaborators, but of course we all have a vested interest in not quite believing that which is not good for our careers.
Is there good news? What’s the path forward? What can we learn from the past?
Historically, in my view, our field fell in love with atomistic detail and the accompanying beautiful pictures, and lost sight of the bigger picture in terms of biophysics and statistical mechanics. Don’t forget that atomistic models are parameterized for equilibrium properties, not dynamics, and anyway we don’t know the true predictions until we can sample well … which we can’t!
One trend that I think has been good for the field over the last ~20 years is the growing use of coarse-grained models. With a CG model, the chemical inaccuracy should be obvious, but there is hope for sampling. I like our field better when we make statements like, “The well-sampled predictions of this approximate model are X, Y, and Z, with uncertainties dX, dY, and dZ, but of course the model limitations imply additional systematic uncertainty that could be as large as DX, DY, and DZ.” All of our models (force fields) are approximate, but let’s sample well what we publish.
A step beyond the CG realm are discrete-state kinetic models – those working-cycle cartoons we see in so many papers and textbooks. Here I see limitations again but also promise. I do have trouble believing that correct cycles can be derived from structural biology alone – which sometimes amounts to drawing transition arrows among solved structures. Those cartoons tend to build in numerous assumptions, stated and unstated. How certain is it that every important state’s structure can be solved? When drawing arrows to connect states, how certain is it that the full function of the system is known? How likely is it that a stochastic system allows only one pathway?
Discrete-state models (a set of states and rate constants) do offer very concrete predictions and so in principle can be reconciled with the right experiments. With this in mind, and with no need for conformational sampling, one can think instead about the space of models. That is, what are the possible models, and which best describes observed behavior? This question should be one that biophysicists can answer.
To sum up, I am advocating publishing studies in a more sampling-forward way. We must continue to be extremely wary of our ability to sample complex atomistic biomolecules and even coarse-grained complex systems. We should not let ourselves be seduced by “stories” with a high-impact ring to them when our sampling is highly uncertain. I also think we should publish negative results so the community can learn from them. We should continue to pursue coarse and discrete-state models, to push the limits of their power and of course characterize their limitations.
Further Reading
Shirts MR, Pitera JW, Swope WC, Pande VS. Extremely precise free energy calculations of amino acid side chain analogs: Comparison of common molecular mechanics force fields for proteins. The Journal of chemical physics. 2003 Sep 15;119(11):5740-61.
Pan AC, Xu H, Palpant T, Shaw DE. Quantitative characterization of the binding and unbinding of millimolar drug fragments with molecular dynamics simulations. Journal of chemical theory and computation. 2017 Jun 21;13(7):3372-7.
Grossfield A, Patrone PN, Roe DR, Schultz AJ, Siderius DW, Zuckerman DM. Best Practices for Quantification of Uncertainty and Sampling Quality in Molecular Simulations [Article v1. 0]. Living journal of computational molecular science. 2018;1(1):5067.
Anandakrishnan R, Zhang Z, Donovan-Maiye R, Zuckerman DM. Biophysical comparison of ATP synthesis mechanisms shows a kinetic advantage for the rotary process. Proceedings of the National Academy of Sciences. 2016 Oct 4;113(40):11220-5.
Tobi D, Elber R. Distance‐dependent, pair potential for protein folding: Results from linear optimization. Proteins: Structure, Function, and Bioinformatics. 2000 Oct 1;41(1):40-6.
Zuckerman, DM, The “Lessons Learned” category: When negative results are useful, LiveCoMS blog post, 2018.
Shaw DE, Maragakis P, Lindorff-Larsen K, Piana S, Dror RO, Eastwood MP, Bank JA, Jumper JM, Salmon JK, Shan Y, Wriggers W. Atomic-level characterization of the structural dynamics of proteins. Science. 2010 Oct 15;330(6002):341-6.
Henderson RK, Fendler K, Poolman B. Coupling efficiency of secondary active transporters. Current opinion in biotechnology. 2019 Aug 1;58:62-71.