It was at breakfast during a recent conference that Prof. X leaned toward me and quietly said, “We tried using weighted ensemble but it didn’t work.” I got the sense he was trying not to broadcast this to other conference attendees, as a courtesy.
Needless to say, I pressed him on what he meant. After all, the weighted ensemble (WE) method has been shown to work for quite a number of challenging systems – including ligand binding/unbinding and protein folding. On the other hand, in my group, we’ve seen WE fall short in notable instances: protein folding from certain initial conformations and for both unbinding and conformational change in the estrogen receptor’s ligand binding domain (LBD). I’ve been interested in the estrogen receptor for a number of years, but it’s a tricky system.
By coincidence, it turned out Prof. X had attempted to study the famous conformational change in the estrogen receptor LBD, when one of its helices undergoes a huge conformational swing. But no luck.
Are such failures something to whisper about, only? Absolutely not.
Every method has its limitations, and these fall into two categories. There are general limitations common to a broad class of approaches. For PMF calculations, for instance, there is the generic issue of sampling orthogonal coordinates, as discussed previously. For path sampling approaches like WE, there is the intrinsic cost of the path ensemble itself, which includes the path length and the number of instances obtained, as was also noted early on in this blog. Then there are limitations specific to a particular method – e.g., based on the binning scheme in WE. We have tried to be open about the challenges of using WE, for instance in a review article and in our online WESTPA documentation.
And yet, generic or abstract warnings may not be enough. They’re not so scary when you’re feeling ambitious and have a large compute allocation in your hands.
I think we need concrete examples to learn from, and that is the motivation for the “Lessons Learned” category of the new LiveCoMS journal, which I helped Michael Shirts and David Mobley in founding. Let’s face it: our community would greatly benefit from airing out the numerous shortcomings of prior efforts – most of which, I believe, are buried in didntwork/ directories. We are doomed to burn ever more computing time attempting impossible tasks. Ten times more computing won’t help if pertinent sampling times have been under-estimated by multiple orders of magnitude.
I strongly encourage you to consider the value of your failed efforts in educating the community. You might even get a highly cited paper out of it! Help us learn our lessons!