According to Fortune, Utah has launched the first U.S. pilot program where an AI system can prescribe medication to patients without any human doctor’s oversight. The program, which began last month, is a year-long trial run in partnership with health-tech startup Doctronic, founded in 2023 by Matt Pavelle and Dr. Adam Oskowitz. The AI is authorized to verify patient histories, conduct clinical interviews, and send approved prescription renewals directly to pharmacies, with the goal of easing burdens on doctors and preventing medication lapses. The state’s commerce department, led by Executive Director Margaret Busse, has waived certain rules for this “regulatory mitigation” test. Doctronic claims its system matches human clinician decisions 99.2% of the time and is limited to 190 common medications, excluding pain management and ADHD drugs. The co-founders hope to scale to tens of thousands of Utah patients this year and eventually handle new prescriptions, like antibiotics, if proven safe.
The Safety Dance and the 99% Problem
Okay, so they have safeguards. The first 250 renewals per drug category get a human review, and then 10% are randomly audited. They’ve got malpractice insurance for the AI. It’s all very measured. But here’s the thing: medicine isn’t software QA. A 0.8% error rate sounds fantastic in tech. In medicine, at scale, that’s a lot of real people getting the wrong drug or dose. John Whyte from the AMA nailed it—accuracy claims don’t replace judgment. An AI can check boxes based on history, but can it sense the hesitation in a patient’s voice when they answer a question? Can it pick up on a subtle symptom a patient didn’t think was relevant? That’s the “context” everyone keeps talking about, and it’s everything.
The Real Driver Isn’t AI, It’s Cost
Let’s be blunt. This pilot isn’t happening because the tech is suddenly perfect. It’s happening because, as Margaret Busse said, “we have a crisis.” Healthcare costs are out of control, and there’s a desperate shortage of clinicians, especially in rural areas. The promise is super efficiency—a prescription in under 30 minutes! That’s a powerful incentive for a strained system. But it sets a terrifying precedent. Once you accept that “low-risk” renewals don’t need a doctor, the definition of “low-risk” has a funny way of expanding. Doctronic is already talking about new antibiotic scripts next. It’s a classic slippery slope, driven by economic pressure, not necessarily by what’s clinically sound.
A Regulatory Waiver Is Not a Solution
This is the part that really gets me. The entire pilot exists because Utah’s Commerce Department issued a waiver. They’re bypassing the existing rules to “test” the tech. But that highlights the core issue: our regulations are completely unequipped for this. Medical regulation is a state-by-state patchwork that moves at a glacial pace. Tech moves at light speed. So now we have a situation where a single state can effectively green-light autonomous clinical AI for a year, based on a partnership with a startup that’s barely a year old itself. That’s not a robust regulatory framework; it’s a Hail Mary pass. What happens after the year? If it’s “deemed safe,” does the waiver become permanent? Other states will feel pressure to follow, and the race to the bottom on oversight begins.
The Future Is Here, and It’s Messy
Look, I’m not a Luddite. AI should be a powerful tool in healthcare. Automating admin, flagging potential interactions, summarizing records—that’s all good. But there’s a bright, red line at autonomous clinical decision-making without a human in the loop. This pilot blurs that line into oblivion. The physicians are right: the tool should support the doctor, not replace them. Utah’s experiment feels less like a careful step forward and more like jumping off a cliff because the elevator is too slow and expensive. We’ll be watching closely, but I think the medical community’s skepticism is more than justified. When the goal is to “drive down costs,” patient safety can’t just be another metric to optimize.
