Around 6 in 10 people use the Internet to search for health information, and LLM platforms receive a growing amount of consumers’ attention online. LLM tools can answer users’ questions by quickly assessing a large volume of information and provide responses tailored to the question – but do their responses meet the standard for evidence-based advice? To explore that question, an expert panel convened by the National Academy of Medicine (NAM) is informing Google’s research into how a Gemini-based model can provide evidence-based answers to common questions about one of the most hotly debated subjects in health – nutrition.  

Project Design

Nutrition information is popular and plentiful online, but around half of what is shared may be inaccurate, according to a systematic review of information samples using a quality criteria checklist from the Academy of Nutrition and Dietetics. For this reason, nutrition makes a great test case for an LLM’s capacity to cut through the noise. To that end, a panel of nutrition experts convened by the NAM will evaluate a set of human-generated and LLM-generated answers to 100 frequently asked nutrition questions.  

In phase 1 of Google’s work on the model, Google collaborated directly with the Food Is Medicine Institute at Tufts University to curate a dataset of nutrition queries and answers. In phase 2, Google asked the NAM to convene a panel to complete an independent, third-party evaluation. After incorporating input from the NAM panel, Google will use the panel’s feedback and an updated query-and-answer dataset to refine its LLM research. 

In evaluating the nutrition answer set, the NAM panel has two aims. The first is to improve the quality of nutrition information generated by Gemini-based models. The second is, more broadly, to understand and evaluate the capacity of a sample LLM to generate high-quality health information in nuanced fields of science by comparing AI-generated responses to those of human experts through a blinded process. The NAM will publish insights in a peer-reviewed publication to share details and learnings from the process and provide guidance that will inform the development of future tools and processes. 

Answer Evaluation

To evaluate the quality of answers provided by the Gemini LLM, the panel will consider four dimensions: 

• Accuracy (defined as alignment with the best available evidence; consistency with medical consensus; and broad representation across the population)
• Safety (defined as the likelihood and potential severity of harm that could result from the content of a response)
• Completeness (defined as how thorough a response is, especially concerning clinical implications of missing or incorrect content)
• Overall quality (defined as a holistic judgment to integrate all dimensions into an assessment of the response’s overall reliability and potential impact)     

To ensure the objectivity of the panel’s review, the panelists will be blinded to whether answers are generated solely by the LLM tool or using information from human experts. Each question–answer pair will be reviewed by at least two panelists, and panelists will be rating answers independently. 

NAM Paper

Following the completion of the answer set evaluation process, the NAM panel will publish a paper through NAM Perspectives that a) describes its approach to evaluating the sample nutrition answer set; b) outlines the multifaceted nature of evaluating AI-generated content; c) presents key challenges, opportunities, and lessons learned; and d) proposes considerations for applying similar processes across other rapidly evolving areas of science and technology. 

Expert Panel

Members of the expert panel serve without compensation and were selected following a call for volunteers.  

• Sue Curry (Chair), Distinguished Professor, College of Public Health, University of Iowa
• Cheryl Anderson, Professor and Dean of the Herbert Wertheim School of Public Health and Human Longevity Science, University of California, San Diego   
• Wayne Campbell, Professor, Department of Nutrition Science, Purdue University
• Teresa Fung, Professor, and Ruby Winslow Linn Endowed Professor, Nutrition C0-Chair, and Director of Didactic Program in Dietetics, Simmons University 
• Hannah Holscher, Professor of Nutrition, College of Agricultural, Consumer and Environmental Sciences, University of Illinois Urbana-Champaign   
• Bradley Johnston, Associate Professor, Department of Nutrition, and Director of EvidenceBasedNutrition.org, Texas A&M University 
• Linda Snetselaar, Professor and Chair, Preventive Nutrition Education, and Director, Nutrition Center, University of Iowa   
• Qi Sun, Associate Professor in the Departments of Nutrition and Epidemiology, Harvard T.H. Chan School of Public Health 
• Taylor Wallace, CEO, Think Healthy Group, LLC; and Adjunct Associate Professor, Gerald J. and Dorothy R. Friedman School of Nutrition Science and Policy, Tufts University

Sponsorship and Sponsor Involvement

The NAM received $140,000 from Google to support NAM staff facilitation of the expert panel and production of a publication between May 2025 and May 2026. Members of the expert panel conduct their evaluations of the answer set independently to ensure objectivity. Google staff participate in working meetings with panel members to review and clarify evaluation outputs.  

Conflict-of-Interest Disclosures

The following participants disclosed no relevant financial relationships: Cheryl Anderson, Theresa Fung, Bradley Johnston, Linda Snetselaar, Qi Sun, and Taylor Wallace.  

The following participants disclosed relevant financial relationships: Wayne Cambell (Mushroom Council, National Cattlemen’s Beef Association, Foundation of Meat and Poultry Research and Education, and the U.S. Whey Protein Research Consortium) and Hannah Holscher (Almond Board of California, General Mills, Hass Avocado Board). 

NAM Staff

• Aisha Salman, Senior Program Officer 
• Emma Freiling, Research Associate