← All research storiesHealth
A hidden tweak inside an AI chatbot quietly shifted its advice about treating depression
A researcher showed that secretly nudging an AI model’s inner workings changed how much it talked about medication versus self-care, with no visible sign for the user.
What the scientists did
More and more people ask AI chatbots health questions. Usually we think a chatbot’s answers come from its training and the instructions it is given. But there is another, hidden way to change an AI’s behavior while it is answering, called activation steering. It nudges the model’s internal signals in a chosen direction without changing the model itself or its instructions, so a user would have no way to notice.
Dr. Roy Perlis tested this on an open-weights AI model (DeepSeek version 4 Flash) set up as a mental health information assistant. He built one “steering direction” by comparing how the model responded to 16 pairs of prompts, one asking for medication to be presented first and the other asking for self-care such as diet, exercise, meditation or supplements without medical involvement. He then wrote 12 made-up scenarios of people asking about depression treatment (4 wanting medication, 4 wanting to avoid it, 4 with no preference) and collected 372 answers at 31 different steering strengths. Another AI model scored each answer, and a human expert re-scored a sample to check it.
Words to know
Large language model (LLM)
An AI trained on huge amounts of text that can write answers in everyday language.
- The technology behind chatbots like ChatGPT, Claude or DeepSeek
Activation steering
Secretly pushing an AI’s internal number patterns in one direction while it runs, so its answers lean a certain way.
- Making answers favor one treatment over another
- Making answers sound more positive or negative
Open-weights model
An AI model whose internal settings are published so anyone can download, run and modify it.
- DeepSeek models
- Meta’s Llama models
What they found
Steering clearly shifted the advice. On a scale where positive numbers mean more emphasis on medication, the average balance was 1.25 at the strongest push toward medication, 0.92 with no steering and 0.17 at the strongest push toward self-care. Counts of medication words and self-care words in the answers moved the same way.
The effect depended on the user. It was strongest when the person stated no preference, smaller when they asked for medication, and not detectable when they said they wanted to avoid medication. A recommendation to see a clinician appeared in 322 of 372 answers (87%), and steering did not change that.
Why it matters to you
When you ask a chatbot about your health, its answer can be shaped by settings you can’t see, and this study suggests the push matters most when you haven’t made up your mind. Treat chatbot health answers as a starting point, compare sources, and notice what options are left out. This story is not medical advice: if you or someone you know is dealing with depression, talk to a doctor, school counselor or another trusted adult.
Where this could lead
Independent AI audits
The author calls for outside testing and benchmarks for AI tools that give health information, to catch hidden bias.
1–3 yearsDisclosure rules
Regulators or companies could require AI health tools to reveal when and how their behavior has been adjusted.
3–10 yearsThese are possibilities the research points toward, not promises. Most early findings take years of testing before they reach everyday life, and some never do.
Why scientists care
Most worries about AI bias focus on what a model learns during training. This study shows behavior can also be changed quickly at the moment it runs, without touching the published model. That makes it a new kind of risk for any AI used in medicine, and a reason scientists want ways to check what AI systems are really doing inside.
Keep in mind
This was a proof-of-concept with one model, one steering direction and made-up scenarios, with no real patients. Whether large commercial chatbots can be manipulated the same way is unknown. Answers were scored mainly by another AI, though a human expert’s scores agreed fairly well.
How far along is the science?
- Peer-reviewedReviewed by independent experts for a journal before publication. Peer review is a quality check, not a grade of evidence strength: confidence still depends on study design, limitations, replication and the wider evidence.
- PreprintA manuscript shared publicly, often before or alongside journal review. Check its separate journal and review status (each story says what we know). Results may change.
- Conference talkPresented to other scientists at a meeting. Usually short and early; a full paper may come later.
- Agency reportPublished by a science agency such as NASA, NOAA or NIH. Reviewed internally, but not by an outside journal.
- Press releaseAn announcement from a university or company. Useful, but always check the research it describes.
How soon could it reach you?
- Could reach you soonCould show up in products, services or advice soon. Our rough editorial guess is about 1 to 3 years, not a forecast from the research.
- A few years outPromising, but needs more testing or engineering first. Our rough editorial guess is several years, with a lot of uncertainty.
- Long-term scienceBasic research that builds knowledge. Real-world uses, if any, are far off.
- Small stepA small update to earlier work. Useful to researchers, but not a change you would notice yet.
Every story links to its original source: the paper itself, or the conference program when no paper is out yet. Explanations are written with AI, usually from the paper’s abstract (each story says what it was written from), then checked against the source. We only use journals, recognized preprint servers, science agencies, scientific conferences and established science news outlets.