HIPAA Obligations When Your Practice Uses AI Tools

The one-line summary

If an AI tool receives, stores, transmits, or processes PHI on behalf of a covered entity, the vendor of that tool is a business associate under HIPAA and requires a signed Business Associate Agreement (BAA) before it can be used. Nothing about the fact that the tool is "just" summarizing, "just" transcribing, or "just" drafting changes this. If PHI passes through it, the vendor is a business associate.

The corollary matters more: if you're using an AI tool with PHI and there is no BAA, that is a HIPAA violation. It doesn't matter how careful you are with the outputs. The violation happens at the moment PHI leaves your compliance perimeter.

Where consumer AI tools quietly fail

The most common failure mode we see is not malicious. It's a busy provider or staff member pasting a chart note into a general-purpose chat assistant to get a plain-language summary for a patient, or to draft a letter, or to translate a discharge instruction. The output is useful. Nobody at the practice is trying to leak data. But the tool's terms of service, in most consumer configurations, either explicitly deny use with PHI, do not offer a BAA, or reserve the right to train on submitted content.

Enterprise-grade offerings from the major model vendors do offer BAAs — but typically only under specific enterprise plans, with specific configuration requirements, and only for specific endpoints. The default account your staff created with a personal email is not covered. The distinction between "the vendor offers a BAA" and "we have a signed BAA covering this exact account and this exact API" is where practices get caught.

The three obligations most practices miss

1. Update the Security Risk Assessment

The HIPAA Security Rule requires an accurate and thorough risk analysis. Introducing a new tool that processes PHI is a material change. If your current SRA doesn't mention the AI tool by name, doesn't describe the data flow, and doesn't assess the threats it introduces, your SRA is now out of date. In an OCR investigation, an SRA that predates your AI adoption is treated as a compliance gap.

The fix is not complicated — an addendum to the current SRA describing the new tool, its BAA status, the data types it processes, the residual risk, and the safeguards in place — but it has to actually exist in writing.

2. Update the Notice of Privacy Practices (sometimes)

If the AI tool changes how PHI is used or disclosed in a way patients would reasonably want to know about, the Notice of Privacy Practices may need updating. Automated transcription of clinical visits, AI-generated communications sent to patients, and any use of PHI for model training (which should not be happening) are the categories where this most commonly comes up. When in doubt, ask a healthcare attorney — not the AI vendor's sales team.

3. Log what the AI actually did

The Security Rule requires audit controls. If your AI tool touches PHI and produces no log of what it read, what it wrote, and when — you have a technical safeguard gap. This is one of the areas where custom-built automation has a genuine advantage over off-the-shelf tools: you can log every action against every record and produce that log on demand. Off-the-shelf tools rarely give you that surface, and when they do it's usually in a format designed for their SRE team, not your compliance officer.

Practical guardrails

A short list of what a compliant AI-in-practice posture actually looks like:

• A signed BAA for every tool that receives, stores, transmits, or processes PHI. No exceptions for "we only use it occasionally."
• Named accounts and named endpoints — the BAA covers a specific product configuration, not the vendor as a corporate entity in general. Confirm which SKUs and API endpoints are inside the BAA.
• A current Security Risk Assessment that names each AI tool, describes the data flow, and scores the residual risk.
• Least-privilege access — the AI service account sees the minimum data required, not the entire chart, and not the entire patient roster.
• No training on your data — an explicit contractual prohibition on the vendor using your PHI to train or improve their models. This is usually a checkbox in the BAA addendum; make sure it's checked.
• Audit logs retained for the period your policies require (typically six years), covering every AI action on PHI.
• A staff policy that lists which AI tools are approved for which use cases, and forbids the rest. Include the consumer tools employees are most likely to reach for by name — that's the paste-into-ChatGPT problem, and it needs a named prohibition to be enforceable.

The economics are misunderstood

The cost of compliant AI tooling is real but not the number people fear. A BAA-covered enterprise API tier costs modestly more per token than the consumer version. A properly scoped custom automation adds engineering cost up front but eliminates most of the recurring risk. Compared to the cost of a breach notification, a corrective action plan, or a resolution agreement with OCR — where six-figure settlements are routine and seven-figure settlements are no longer unusual — the compliance overhead is inexpensive.

The economic mistake practices make is treating the consumer version of a tool as "free" because there is no invoice. The correct accounting includes the expected value of the compliance risk it introduces. Once that math is on the whiteboard, the decision usually gets made quickly.

Where to start

If your practice has adopted any AI tooling in the last two years — scribing, drafting, patient communication, document classification, chatbots on the website, anything — and you cannot immediately produce the BAA covering each tool and the SRA update reflecting its adoption, that is where to start. Inventory the tools. Chase down the BAAs. Update the SRA. In that order.

If you don't have a current SRA at all, that comes first. Everything else is downstream of it.