Health, money and legal questions: green lights and red lines
What an assistant can safely do with a health, money or legal question, where you stop, who to ask instead, and how to ask for a person to review a decision software may have made about you.
- What this is
- Which health, money and legal tasks are safe to give an AI chat assistant, which need checking, and which to leave to a professional.
- Why it matters
- You arrive better prepared for appointments and advisers, without acting on a confident answer that is wrong or leaves something out.
- The short version
- Use an assistant to prepare for a professional: order your papers, explain terms already used and draft your questions.
- Refused a loan, claim, job or benefit? Ask in writing whether AI was used, what data about you it used, and how a person can review it.
- Remove names and numbers before you paste, check every figure and deadline against the originals, and never decide, dose, file, sign or invest on its word.
On this page
Short answer
Use an AI assistant (a chat app such as ChatGPT) to get ready for a professional, not to replace one. Green light: put your papers in order, explain terms a professional or a letter has already used, and draft the questions you will ask. Amber: compare options and costs, then check every figure and deadline in the original documents. Red line: never decide, dose, file, sign or invest on the assistant's word. Take names and numbers out before you paste anything.
Who this is for. Anyone who has typed a symptom, a bill or a letter from a court or a landlord into a chat box. No setup needed.
The model often knows more than people get out of it
In a 2026 randomised trial, the AI models (the systems behind chatbots) tested on their own named the relevant conditions 95% of the time. When 1,298 real people used the same models, they found the right condition in under 35% of cases and chose the right next step in under 45%, no better than people without AI (Bean et al., Nature Medicine, randomised trial). The trial measured the gap, not every cause of it.
The step people skip is the person. In a 2026 US survey, most people who asked a chatbot about mental health (58%) did not follow up with a professional, and neither did 42% of those who asked about physical health (KFF, survey).
Money answers fail more quietly. In an industry test of 18 chatbots on 121 personal-finance questions, answers were wrong 57% of the time on average (Saturn, vendor industry test). So the risk is a confident answer that is wrong or leaves something out. Many people would not catch it: in a vendor survey of 1,000 US chatbot users, 57% said they would act on a chatbot's money advice without checking it; 23% had received wrong information and 5% found out only after acting (PensionBee, vendor survey).
Legal answers carry the most risk for people without a lawyer. In 2026, about six in ten reported US instances of AI-fabricated material in court filings involved people representing themselves (MSBA, count of reported instances to July 2026). If you represent yourself, nobody checks your filing before the court reads it.
Green lights and red lines
The table is our suggestion; no study tests it. Amber means you may do it, then check every line against the original document or an official source.
| Health | Money | Legal | |
|---|---|---|---|
| Green | Put symptoms, dates and medicines in order before an appointment. Explain a term your clinician or a letter used. Draft the questions to ask. | Sort a bank export (a downloaded list of transactions) into categories. Explain a term in a statement or contract. List questions for your bank or an adviser. | Summarise a letter in plain words, with the original beside you. Build a dated timeline of what happened. List questions for a lawyer or the court help desk. |
| Amber | Ask what a test or a leaflet usually means, then ask your clinician what it means for you. | Compare two offers, plans or loans side by side, then check each rate, fee and deadline in the provider's own documents. | Ask which official form or court page might apply, then open that page yourself and follow it, not the summary. |
| Red | Diagnose, start, stop or change a dose, or decide whether worrying symptoms need care. | Invest, move money, take on debt or file a tax return on its word. | File, sign or send anything it drafted without a qualified check. Cite a case or law you have not opened in an official source. |
One amber case is easy to miss: a salary or price you plan to ask for (see Prepare for a negotiation with AI: a pay rise, a job offer or a price for why). Treat any figure it gives you as a starting point and check it against published pay data for your role.
Try the table on nine requests (illustrative). Pick a light for each before you see the table's answer.
Prepare, then ask a person
The assistant helps you get ready for the appointment, the adviser or the court office, and the person makes the call.
Example (invented). Dana's insurer has refused a claim for a scan, and the letter gives 30 days to appeal. She removes her name, policy number and address, pastes the letter, and asks the assistant to explain the terms, list the dates and draft questions. It points out that the letter names a deadline and a form. She opens both on the insurer's own site, then calls her doctor's office and the insurer with five questions written down. She does not let it draft the appeal on its own, because that is a red-line step.
If AI may have decided about you
A refusal for a loan, an insurance claim, a job, a flat, a school place or a benefit may have been assessed by software. You can ask the company three things, anywhere: whether an automated or AI system was used, what information about you it used, and how a person can review the decision. Send this by email or through the company's form. The questions are our suggestion: a law may give you more, and a company may not be required to answer.
I received a decision on <DECISION_DATE> about <APPLICATION>. Please tell me:
1) whether an automated or AI system was used in this decision,
2) what information about me it used,
3) how I can have the decision reviewed by a person.
Replace the following:
<DECISION_DATE>: the date you received the decision, such as2026-10-12.<APPLICATION>: what it was about, with any reference number, such asmy application for the analyst role, reference 4471(illustrative).
Save the decision, the notice, your request, the dates and any reply in one folder. A dated copy of what you sent helps if you later go to a regulator or a lawyer.
Some places now require a notice or a way to ask. New York City's hiring-AI audit law (Local Law 144) has applied since July 2023, and law-firm summaries say Illinois has required notice since 1 January 2026 when employers use AI in decisions such as hiring, promotion or discharge (NYC DCWP, official; WLF, law-firm summary). In Colorado, the official fiscal note for SB 26-189 as introduced describes notice when automated tools materially influence consequential decisions and information within 30 days after an adverse outcome; a law-firm summary says it was signed on 14 May 2026 and takes effect on 1 January 2027, which we could not confirm on an official page on 2026-10-11 (Colorado fiscal note, official, introduced bill; Holland & Knight, law-firm summary). These rules change often, so check the official page or your regulator on the day you rely on it.
What to take out before you paste
Replace names, dates of birth, addresses, and account, policy, case and ID numbers with placeholders such as [NAME] and [POLICY]. The assistant does not need them to explain a term or draft a question. What not to paste has the full rule.
Do not assume a health app protects what you type the way your doctor's office must. In the US, HIPAA generally applies to a health app only when it handles health data on behalf of a provider, health plan or their contractor; an app you choose and use yourself is usually not covered (HHS, 2016, regulator guidance). Check what your assistant keeps and trains on with the Privacy settings pass.
Use it
Copy this into any assistant (our suggestion).
I have an appointment with a <DOCTOR / ADVISER / LAWYER / COURT HELP DESK> about <TOPIC>.
Do not diagnose me, tell me what to decide, or give me a figure to act on.
Using only the notes below:
1. Put the facts in date order and say what looks missing.
2. Explain in plain words each term I have marked [?].
3. List any deadline, amount or form named in my notes, quoted exactly.
4. Write the five questions I should ask, most important first.
5. List what I should bring.
Notes (names and numbers removed):
<PASTE NOTES>
Before you rely on anything it says, run Check before you trust. Check point 3 against the original letter or statement, word for word.
Stop and contact a person now
These are our suggestions, not a complete list.
- Severe or sudden symptoms. Call your local emergency number. Do not ask a chatbot first.
- Thoughts of harming yourself, or someone else in danger. Use the Get help now line at the top of this page.
- A court, tax or eviction deadline within days. Call the court office, the tax authority or a legal aid service today.
- A call or message asking you to move money now. Hang up and call back on a number you know: AI voice scams.
- The assistant tells you that you do not need a professional, or you notice you are using it to put one off.
Where to go instead
- Health: your doctor or nurse; a pharmacist for questions about medicines; your country's official health service for non-urgent advice; your local emergency number for anything urgent.
- Money: your bank or provider for your own account; a free government or non-profit money guidance service; a regulated or certified adviser before an investment decision; the tax authority's own pages for tax.
- Legal: the court's own self-help pages or help desk; legal aid or a law clinic; a lawyer, found through your bar association or law society referral service.
Limits
Sources
- Bean et al. Nature Medicine, 2026. Randomised trial with 1,298 people. Link
- KFF. Poll on AI chatbots and health information, 2026. Survey of US adults. Link
- Saturn. "Artificial Authority". Industry test by a vendor. Link
- PensionBee. AI and your money report, 2026. Vendor survey. Link
- MSBA. Count of US court decisions flagging AI-fabricated material, 2026. Field study. Link
- HHS. Health app developer scenarios, 2016. Regulator guidance. Link
- Colorado. Fiscal note and bill page for SB 26-189, 2026, with law-firm summaries from McDermott, DLA Piper and Holland & Knight. Official (introduced bill) and law-firm summaries. Link · Link · Link · Link · Link
- US hiring-AI rules: New York City DCWP page and New York State Comptroller audit, 2025 (official), and law-firm summaries for Illinois and Connecticut. Link · Link · Link · Link
Changes
- v1.010 Oct 2026first version
- Sources
- 8
- Updated
- 10 Oct 2026
- Last checked
- 11 Oct 2026