Initial reactions to the UMU & IMI

October 29, 2015 • PD Schloss • 7 min read •

I must admit that I always feel a certain euphoria when I find people talking about microbiome research in the popular press or policy pieces. I’m a microbiome homer. So you can imagine my feelings yesterday when Science published an article by US scientists calling for a “interdisciplinary Unified Microbiome Initiative (UMI) to discover and advance tools to understand and harness the capabilities of Earth’s microbial ecosystems.” Not to be outdone, Nature published a piece indicating that each country would have its own UMI and these would be assembled under an International Microbiome Initiative (IMI).

If you’re an outsider you’d think there might be universal agreement among microbiologists that this is great for the home team. There has already been some predictable “microbiome hype” critique and the FacePalm reaction that many people (even at universities with big journal budgets) can’t freely access the articles. But is there a substantive reason to not do cartwheels?

Let’s focus on the strength of the idea. I strongly agree that microbiology is too fractioned. Look at my own university. My Department of Microbiology & Immunology in the School of Medicine does relatively little with the relatively large number of non-medical microbiologists across campus. It was striking last week that when we received an ASM Milestone in Microbiology site there was little to no participation from outside our department. At Michigan we’re going through a process to unify the biological sciences and we’ll see how it goes, but we’ve been through this before. Whatever, there are too many silos and most of it is imposed at the level of individual PIs. I make the decision to keep working on my pet project instead of walking across campus to a seminar.

For funding, there are exceptions to the silo model. The NSF’s Microbial Observatories program partnered some with the USDA for a few projects, the NSF has partnered with NIH to study the ecology of infectious diseases, and there are likely others. In the Science article, It would have been interesting to hear a discussion of how those pilot programs for this larger effort worked out. Were the projects more than the sum of their parts? If there was a way to cut across agencies to fund specific, well-defined problems that would be awesome. For example, there has been a lot of interest in studying the human microbiome and in studying the built environment. Where would you go to get money to study the interaction? There are clear problems where we need a cross-agency approach. Alas, the cynic in me questions what mechanisms will be put in place to make sure such a venture doesn’t become ensilaged too and wonder whether these types of projects were the impetus behind the NSF in the first place.

Despite my homerism, I admit that my initial reaction to the announcement of the UMI and IMI was to be dismissive and probably not well captured in 140 characters. Let me unpackage two comments…

we have a global effort. it's called the scientific method.

— Pat Schloss (@PatSchloss) October 28, 2015

Part of my business is developing tools that others use to analyze microbial ecology data. I think we’re pretty good at it the best at it. I have competed against and benefited from the contributions of people from around the world to produce the most cited tool for analyzing 16S rRNA gene sequences. I don’t need a new international effort to disseminate my ideas further. Similarly, we’ve developed the best method for generating 16S rRNA gene sequences from microbial communities. For all of this work we benchmark our methods relative to objective standards and other methods. In spite of our success, we are still routinely ignored by many (go check out the mothur forum for tales of woe), including many of the co-authors on the Science article. More than anything, people need to learn to do a literature search and use the scientific method.

"reorient the field from correlative studies to hypothesis-driven approaches" - feel like I'm doing that already. maybe i'm doing it wrong.

— Pat Schloss (@PatSchloss) October 28, 2015

It’s really hard not to go ad hominem here. This perpetual refrain really pisses me off because it points out the perceived splinter in the eye of others without noticing the beam in their own. Meanwhile myself and others plod along doing our best to do hypothesis-driven research using microbial communities that vary widely across biological replicates. The Earth Microbiome Project, International Census of Marine Life, and much of the NIH Human Microbiome Project and the European Commission’s MetaHit Project all lacked a specific question. So microbial ecology gets thrown under the bus because these expensive projects were poorly designed and didn’t have a falsifiable hypothesis? Please.

I was asked what I would have said differently. There are always funding issues. But I think most of our issues are sociological and the UMI/IMI announcements side-step that. Here are a few examples of what I’d do differently and where I’m happy to lead by example…

• A return to boutique science. Having participated in several large initiatives, I’m ready for more Small Science where I can set up SOPs for my questions and I can assemble and join teams where my colleagues and I share the same scientific values. Some day I’d love to see an analysis of whether the large ventures (HMP, EMP, etc) have gotten us that funding individual investigators wouldn’t have gotten us.

• We need microbiome-specific review panels. I’ve been on the fence about this and can see arguments on both sides. I’m consistently the only microbiome person on a study section. As hard as I fight it, my biases come through and I can single-handedly kill a proposal. That is too much power for me and it really freaks me out. Someone else has the same power when my proposal goes in for review. There has to be a better way.

• We need better peer review. If peer reviewers miss flaws, then we need man/woman-up and call people on it using tools for post-publication review. If someone says something outlandish in a seminar or to a reporter, we need to publicly call them on it - not wait until we’re back in the lab gossiping with our colleagues. I’m as guilty as everyone else on this and know I need to do better. In this spirit, why didn’t the PNAS editors tell Craig Venter that this does not belong in PNAS?

• We need negative results. The microbiome does not control everything and knowing the structure of the microbiome is not critical to every problem. In the race to publish, data are sliced and diced to find a positive result. We need to develop a culture that values negative results because it helps define what is actually important. When everything is special….

• A moratorium on generating new data. OK, this is a bit of a joke. But we need greater transparency in our data and methods and we to use other people’s data more. There are so many microbiome studies out there and meta-analyses are the exception rather than the rule. We need to flip this. Did you just do an obesity study? Great. Now validate your results using other people’s data. We can’t do this if we don’t have access to those data or if a previous author’s methods description says “We used mothur/QIIME to analyze our 16S rRNA gene sequence data”.

I think these would be a good start.