Researchers tested four popular AI chatbots—ChatGPT, DeepSeek, Gemini, and AI Clinical Nutritionist—to see how well they answer questions about nutrition and disease. They asked each chatbot 276 multiple-choice questions about topics like diabetes, heart disease, and anemia. The chatbots got about 72% of the answers right on average, which is decent but not perfect. Some topics, like diabetes, were answered correctly almost every time, while others, like anemia, were much harder. The study shows that AI chatbots could be helpful tools for learning about nutrition, but they still need improvement and shouldn’t replace talking to a real dietitian or doctor.

The Quick Take

  • What they studied: How accurate are AI chatbots when answering questions about nutrition and diseases? Can you trust them to give you correct information?
  • Who participated: This wasn’t a study with people. Instead, researchers tested four different AI chatbots by asking them 276 nutrition questions covering 16 different diseases and health conditions.
  • Key finding: AI chatbots answered about 72% of nutrition questions correctly. DeepSeek did slightly better at 73%, while Gemini scored lowest at 69%. When asked the same questions twice, the chatbots gave consistent answers, which is a good sign.
  • What it means for you: AI chatbots might help you learn basic nutrition information, but they’re not perfect. They work better for some topics (like diabetes) than others (like anemia). You should always check important health information with a real doctor or registered dietitian before making changes to your diet.

The Research Details

Researchers created a test with 276 multiple-choice questions about nutrition and diseases. These questions came from a professional nutrition textbook and covered 16 different health conditions, including diabetes, heart disease, and digestive problems. They tested four different AI chatbots by asking each one the same questions twice—once in the first round and again later—to see if the chatbots would give the same answers both times.

The researchers measured two things: accuracy (did the chatbot pick the right answer?) and reliability (did the chatbot give the same answer when asked the same question again?). They used special statistical methods to measure reliability, looking at whether the chatbots were consistent in their responses.

The questions covered different areas of nutrition knowledge: what causes diseases, how diseases work in the body, medical treatments, and nutritional treatments. This gave a complete picture of how well the chatbots understood clinical nutrition.

This approach is important because AI chatbots are becoming more popular for health information, but we don’t know if they’re reliable. By testing them with real nutrition questions, researchers can see which chatbots are trustworthy and which topics they struggle with. This helps people understand when it’s safe to use AI for health information and when they need to talk to a real expert.

The study used a large number of questions (276) from a respected professional textbook, which makes the test thorough and realistic. The researchers tested consistency by asking the same questions twice, and the chatbots were very consistent (showing high reliability scores). However, this study only tested basic versions of the chatbots, so newer versions might perform differently. The study didn’t include real patients or doctors, just the chatbots answering questions.

What the Results Show

On average, the four AI chatbots answered about 72% of nutrition questions correctly. This is like getting a C+ or B- grade on a test—decent, but not excellent. DeepSeek performed best with 73% accuracy, followed by ChatGPT at 73%, AI Clinical Nutritionist at 73%, and Gemini at 70%. When the researchers asked the same questions a second time, the accuracy rates were similar, showing the chatbots were consistent.

When looking at specific diseases, the chatbots performed very differently. They answered questions about diabetes and upper digestive system diseases almost perfectly (100% correct). However, they struggled badly with anemia questions, getting only 36% correct. This big difference shows that AI chatbots are much better at some topics than others.

When looking at different types of questions, the chatbots did worst on medical treatment questions (64% correct) and better on nutritional treatment questions (67% correct). This suggests the chatbots have more reliable information about nutrition than about medical treatments.

The study found that all four chatbots were very consistent in their answers. When asked the same question twice, they gave the same answer about 77% of the time or better. This consistency is important because it means the chatbots aren’t just guessing randomly—they’re following some kind of logic, even if that logic sometimes leads to wrong answers. The researchers also noticed that the different chatbots performed similarly to each other, suggesting they might have similar strengths and weaknesses.

This is one of the first studies to test how well AI chatbots handle clinical nutrition questions specifically. Previous research has tested AI chatbots on medical questions in general, and they usually score around 50-80% accuracy. This study’s finding of 72% accuracy for nutrition questions fits within that range, suggesting AI chatbots are moderately reliable for health information but not perfect.

This study only tested basic versions of the chatbots, so newer or upgraded versions might perform better. The questions all came from one textbook, so the results might be different with questions from other sources. The study didn’t test whether the chatbots could explain their answers well or handle follow-up questions. Also, the study didn’t include real patients or real-world situations where nutrition advice matters most. Finally, AI chatbots are constantly being updated, so these results might change over time.

The Bottom Line

AI chatbots can be a helpful starting point for learning about nutrition and diseases, but they should not be your only source of information. Use them to get basic facts, but always verify important health information with a doctor, registered dietitian, or other qualified healthcare professional. Be especially careful with topics where the chatbots performed poorly, like anemia or medical treatments. (Confidence: Moderate—based on 72% accuracy rate)

Students learning about nutrition might find AI chatbots helpful for studying. People curious about how their diet relates to health conditions could use them for general information. However, people with serious health conditions, people taking medications, or anyone making major diet changes should talk to a real healthcare provider instead of relying on AI. Pregnant women, children, and people with multiple health problems should especially avoid using AI chatbots as their main source of nutrition advice.

If you use AI chatbots to learn about nutrition, you might understand basic concepts within a few conversations. However, don’t expect them to replace professional nutrition counseling, which typically takes weeks or months to show real health benefits. Any major diet changes based on AI information should be discussed with a healthcare provider first.

Want to Apply This Research?

  • Track which nutrition topics you ask AI chatbots about and note when you verify the answers with a healthcare provider. Create a simple log: ‘Question asked,’ ‘AI answer,’ ‘Professional verification,’ and ‘Was it accurate?’ This helps you learn which topics the AI handles well.
  • Use the app to set reminders to fact-check AI nutrition advice with a registered dietitian or doctor before making diet changes. For example, if an AI chatbot suggests a specific diet for a health condition, schedule a quick call with your doctor to confirm it’s right for you.
  • Over time, track your confidence in AI chatbot answers by rating them 1-5 stars after you verify them with professionals. This personal tracking helps you learn which chatbots and topics you can trust more. Share this data with your healthcare provider to help them understand what health information you’re using.

This research tests AI chatbots’ ability to answer nutrition questions, but it does not mean AI chatbots can replace professional medical advice. Always consult with a licensed healthcare provider, registered dietitian, or doctor before making any changes to your diet or nutrition plan, especially if you have a health condition, take medications, are pregnant, or are caring for children. AI chatbots can provide general information but may contain errors. Never use AI chatbots as your only source of medical or nutrition advice. If you have questions about your specific health situation, please speak with a qualified healthcare professional.