How many times has our community solved the sampling problem? I think it\u2019s a fair question. You know I\u2019m talking about claims rather than actual solutions. And many if not most of those claims are made in the abstracts of papers, even when the data paints a more limited story. I think our abstracts are the problem.<\/p>\n
<\/p>\n
Few people read papers in detail and we often make judgements largely based on the abstract and conclusions supplied by authors. When I read the abstracts of papers from our community, I wonder if we don\u2019t exaggerate too often.<\/p>\n
I have become that annoying referee that bullies authors into toning down their abstracts to match their actual results. (Yes, that was me.) I want to encourage you to develop that same annoying tendency when you referee papers.<\/p>\n
We all will benefit from more accurate reporting. Our colleagues won\u2019t have unreasonable expectations of us, and we won\u2019t have to read every paper with a big claim. What a delight it would be to read, \u201cAlthough our method worked well by design in our system of interest, it\u2019s unlikely to work well in your system.\u201d<\/p>\n
Now it hit me, and I\u2019m sure a few of you had the same thought, that I may have solved the sampling problem myself more than once! I went back to some abstracts from papers of mine that I genuinely thought were major advances at the time. What had I said? It was a relief that most were pretty honest and precise. But a few had statements that may not have been exaggerations but perhaps were ill-considered in retrospect. OK, they were exaggerations.<\/p>\n
One abstract<\/a> said the following: \u201cThe absolute free energy\u2014or partition function, equivalently\u2014of a molecule can be estimated computationally using a suitable reference system. Here, we demonstrate a practical method for staging such calculations by growing a molecule based on a series of fragments.\u201d Hmm, a practical method \u2026 for any size system? Not really. Should have qualified that.<\/p>\n Another<\/a> said, \u201cFor a two-dimensional \u2018\u2018toy\u2019\u2019 test system, the new minimally optimized method performs roughly one hundred times faster than either optimized \u2018\u2018traditional\u2019\u2019 Jarzynski calculations or conventional thermodynamic integration. The simplicity of the underlying formalism suggests the approach will find broad applicability in molecular systems.\u201d I really thought so. But it wasn\u2019t quite true<\/a>.<\/p>\n A third<\/a> said, \u201cthe [weighted ensemble] WE method is readily applied to more chemically accurate models, and by studying a series of lower temperatures, our results suggest that the WE method can increase efficiency by orders of magnitude in more challenging systems.\u201d This sort-of proved to be true, but it\u2019s been a hard road and I\u2019ve already confessed that WE cannot solve every problem<\/a>.<\/p>\n Abstracts are a key tool in our business, so be honest. Your later self will thank you. And be a tough referee. Their later selves will thank you.<\/p>\n Postscript.<\/em> While we\u2019re on the subject of abstracts, I have another suggestion. Try writing your abstract before doing your research. (Of course, check it after you\u2019re done to be sure it\u2019s accurate!) The idea is to establish the big picture of your project, so you can see where you\u2019re going. You want to avoid unimportant rabbit holes during long months of research – and make sure you accomplish something important and new. I often encourage students and postdocs to draft an abstract at the start of a project. Try it!<\/p>\n UPDATE [Feb. 13, 2020]: Ten days after I made this post (Dec 7, 2019), the New York Times published a related piece<\/a> suggesting that one gender was more likely to up-sell their work with self-congratulatory description. (Guess which one?) This made me think more about the gender issue in our own field. Have something to say on this issue? Write me: zuckermd@ohsu.edu<\/a>. I hope to do a post on this in the future, ideally relating the experiences of several individuals, anonymously if they wish.<\/p>\n","protected":false},"excerpt":{"rendered":" How many times has our community solved the sampling problem? I think it\u2019s a fair question. You know I\u2019m talking about claims rather than actual solutions. And many if not most of those claims are made in the abstracts of papers, even when the data paints a more limited story. I think our abstracts are […]<\/p>\n","protected":false},"author":6,"featured_media":304,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[21,23],"tags":[],"class_list":["post-302","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-science-culture-integrity","category-writing"],"_links":{"self":[{"href":"https:\/\/statisticalbiophysicsblog.org\/index.php?rest_route=\/wp\/v2\/posts\/302","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/statisticalbiophysicsblog.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/statisticalbiophysicsblog.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/statisticalbiophysicsblog.org\/index.php?rest_route=\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/statisticalbiophysicsblog.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=302"}],"version-history":[{"count":6,"href":"https:\/\/statisticalbiophysicsblog.org\/index.php?rest_route=\/wp\/v2\/posts\/302\/revisions"}],"predecessor-version":[{"id":315,"href":"https:\/\/statisticalbiophysicsblog.org\/index.php?rest_route=\/wp\/v2\/posts\/302\/revisions\/315"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/statisticalbiophysicsblog.org\/index.php?rest_route=\/wp\/v2\/media\/304"}],"wp:attachment":[{"href":"https:\/\/statisticalbiophysicsblog.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=302"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/statisticalbiophysicsblog.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=302"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/statisticalbiophysicsblog.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=302"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}