Can you trust AI with health questions?

Asking a chatbot about a rash, a pill or a blood test is now ordinary. A KFF poll of 1,343 US adults, run in February and March 2026, found that 32% had used an AI chatbot for health information in the past year, and 16% had asked one about mental health. OpenAI says more than 230 million people a week ask ChatGPT health and wellness questions.

So the useful question is not whether to use AI for health, but for what. The short version: it is a decent explainer and a poor decision-maker.

What are chatbots actually good at?

Chatbots are strong at language, and a lot of medicine reaches you as confusing language. Good uses include:

• Explaining terms. What "benign", "HbA1c" or "mild cardiomegaly" means, in plain words.
• Decoding a letter or leaflet. Turning a hospital letter or a medicine's patient leaflet into a short summary you can follow.
• Preparing for an appointment. Listing the questions to ask, or helping you write down when a symptom started and what makes it worse, so a ten-minute visit goes further.
• Understanding a diagnosis you already have. Background on a condition your doctor has named, so you can follow the conversation about treatment.

In each case a professional stays in charge of the decision. The chatbot helps you understand and ask better questions.

Where do they go wrong?

The big problem is that a chatbot sounds equally sure whether it is right or wrong. It works by predicting likely words, not by checking facts, and sometimes it produces fluent answers that are simply invented. This is called a hallucination, and our explainer on why AI makes things up covers how it happens.

In health, that shows up in a few familiar ways:

• Confident wrong answers. ECRI, a US patient-safety organisation, put misuse of AI chatbots at the top of its list of health technology hazards for 2026. Fierce Healthcare reports that in ECRI's tests, chatbots suggested wrong diagnoses, recommended unnecessary tests and even invented body parts. ECRI says chatbots are "programmed to sound confident and to always provide an answer."
• Missing context. In January 2026 a Guardian investigation found Google's AI Overviews giving "normal" ranges for liver blood tests without saying that normal values vary with age, sex and ethnicity, Android Authority reports. Google removed the summaries for the flagged searches, but a slightly reworded search could still produce one.
• Telling you what you want to hear. A study in npj Digital Medicine, described by Mass General Brigham, asked five chatbot models to write a note telling people to take acetaminophen instead of Tylenol because of "new side effects". The two are the same drug. The OpenAI models went along with the request every time. Researchers call this people-pleasing "sycophancy".
• Non-local or outdated guidance. Medicine names, approved doses, screening ages and how you reach a doctor differ between countries. A chatbot may answer with US brand names or old guidelines unless you tell it where you live, and even then it may not know the latest local rules.

What does the evidence say about real people using them?

Chatbots score well on medical exam questions. Real conversations are harder, and the best evidence so far comes from a randomised trial led by the University of Oxford and published in Nature Medicine in February 2026.

Nearly 1,300 people in the UK were each given ten medical scenarios and asked what the likely condition was and what to do next, from staying home to going to A&E. Some used GPT-4o, Llama 3 or Command R+; a control group used whatever they normally would, such as a search engine. According to the study summary on Scimex, the chatbots on their own identified the right conditions in 94.9% of cases. But people using those same chatbots identified a relevant condition in less than 34.5% of cases and chose the right course of action in less than 44.2%. That was no better than the control group.

The Oxford Biomedical Research Centre explains why: people often didn't know what details to give, the answers mixed good and poor advice, and small changes in wording produced very different replies. Dr Rebecca Payne, a GP on the team, put it bluntly: "AI just isn't ready to take on the role of the physician."

What about the official health features from AI companies?

The big companies have noticed how many people ask health questions, and some now offer dedicated tools.

OpenAI's ChatGPT Health, announced in January 2026, is a separate space inside ChatGPT that can draw on connected medical records and wellness apps to help you understand test results or prepare for appointments. OpenAI says health conversations are not used to train its foundation models, are kept apart from your other chats, and get extra encryption. It also says Health "is not intended for diagnosis or treatment." Medical record connections are US-only, and at launch the feature was not offered in the European Economic Area, Switzerland or the UK.

Anthropic launched similar connectors for its Claude chatbot the same month. Fortune reports that US subscribers on paid plans can link their medical records, and that Anthropic says their health data "is never used for model training."

These promises are about privacy and training. Neither company claims its product can replace a doctor, and the Oxford findings about how people and chatbots talk past each other apply here too.

What happens to my health data when I paste it in?

In EU law, health information is a special category. The European Commission lists "health-related data" among the sensitive personal data that the General Data Protection Regulation (GDPR) protects most strictly. That gives you rights over it, such as asking a company for a copy or for deletion. It does not stop a chatbot company storing what you choose to type in.

In the US, the health privacy law HIPAA covers doctors, hospitals, insurers and the companies working for them. Once your information goes to an app that is none of these, the US health department says it is "no longer subject to the protections of the HIPAA Rules." A general chatbot you signed up for yourself usually falls outside HIPAA, so its own privacy policy is what protects you.

People share a lot. KFF found that 41% of those who used AI for health had uploaded personal medical information, such as test results or doctors' notes, and 77% of all adults were worried about the privacy of medical information shared with AI tools.

What applies to any chat applies with extra force here: your words are stored, may be read by reviewers, and on many personal accounts may be used as training data unless you switch it off. Our guide to your data and AI chatbots shows where the off switches are. Before pasting a lab report, delete your name, date of birth, patient number and address. The chatbot needs the values, not your identity.

Is it safe to talk to a chatbot about mental health or a crisis?

Many people do. KFF found 58% of those who asked AI about mental health did not follow up with a professional afterwards.

The companies say they have built safeguards. In October 2025 OpenAI said that about 0.15% of ChatGPT's weekly users have conversations with explicit signs of suicidal planning or intent. It said it worked with more than 170 mental health experts and that ChatGPT now points people to crisis hotlines more often, moves sensitive conversations to safer models, and suggests breaks during long sessions. OpenAI reported 65% fewer unwanted responses in conversations about self-harm and suicide, while saying its estimates may change as its methods improve.

Those safeguards came after serious failures. In August 2025, the BBC reported that the parents of Adam Raine, a 16-year-old from California who took his own life, were suing OpenAI. The lawsuit alleges that ChatGPT discussed suicide methods with him and "recognised a medical emergency but continued to engage anyway." OpenAI said "there have been moments where our systems did not behave as intended in sensitive situations," according to the BBC report. The claims have not been decided by a court.

A chatbot can be a place to put feelings into words at 3 a.m. It is not a therapist and cannot send help. If you or someone near you is in danger, call 112 or your local emergency number, or a crisis line in your country.

How do I use AI safely with a health question?

A short routine that keeps the chatbot in its useful lane:

1. Decide first if it is urgent. Chest pain, difficulty breathing, signs of a stroke, a serious injury or thoughts of harming yourself mean calling emergency services, not typing.
2. Ask it to explain, not to decide. "What does this term mean?" and "What should I ask my doctor about this?" are safer than "What do I have?"
3. Give context, not identity. Say your age range, your country and the medicines you take, but leave out names and ID numbers.
4. Ask for the reasoning and the source. Then check the key point on an official site such as your national health service or the medicine's leaflet.
5. Ask the same question another way. If the answer changes a lot, treat both versions with suspicion.
6. Never change or stop a medicine on a chatbot's word. Ask your doctor or pharmacist.
7. Take your notes to a professional. If something worries you, or symptoms persist or get worse, book an appointment. The chatbot's best job is getting you there better prepared.