Senators Weigh Whether Health Care AI Needs a Leash

The Senate Finance Committee contemplated the future yesterday: artificial intelligence and its potential applications to health care.

And it turns out the future looks an awful lot like the past and present: Democrats want regulations. And the industry wants money.

“There are a lot of reasons to be optimistic,” Finance Committee Chair Ron Wyden (D-Ore.) said. But, he warned, algorithms need regulation and accountability to make sure they aren’t producing bad or biased recommendations that deny care or lead to inappropriate care.

He expressed outrage at the results of 2019 research led by Ziad Obermeyer, a University of California at Berkeley associate professor, who found that one commercial algorithm recommended less health care for Black patients based on historical cost data.

“How does such a flawed system make its way into general use?” Wyden said. “Nobody’s watching. No guardrails. No guardrails to protect the patients from flawed algorithms and AI systems.” 

It’s unclear whether this algorithm is still being marketed, Obermeyer testified later.

The hearing marked Congress’s latest attempt to wrap its head around the newest AI systems, which can mimic some forms of human reasoning to make predictions and calculations, or generate text and images that look deceptively human-created.

Wyden touted his “Algorithmic Accountability Act,” a bill intended to force companies to assess their own products and require the Federal Trade Commission to collect and report data on AI systems. But Republicans indicated that they don’t want to move quickly on the emerging technology. 

AI is already prevalent in health care; doctors use the systems to distill patient visits into clinical notes and to point out potentially cancerous lesions or polyps, for example. Accordingly, there’s big Washington muscle behind the algorithms: CNBC counts a 185 percent surge — from 158 to over 450 — in the number of organizations lobbying on AI regulations.

Industry leaders — and the committee’s Republicans — didn’t explicitly refute the need for regulations at Thursday’s hearing. But their vision for it was more constrained and, in one instance, raised the question: Can tech watchmen watch the tech watchmen?

Take Sen. Bill Cassidy (R-La.), who mused at the end of the hearing that AI models could wind up rating AI models. Asking for responses from witnesses, he reminded them to “please be tight with your answers” — and cut them off soon after most told him that humans needed to evaluate the models.

“Practically, that seems like that’s going to be incredibly cumbersome,” he informed University of Chicago Provost Katherine Baicker. “That seems impossible,” he told Mark Sendak, co-leader of Duke’s Health AI Partnership, about the researcher’s proposals on the need for local human oversight of algorithms.

The committee’s senior Republican, Sen. Mike Crapo (Idaho), acknowledged in a statement the importance of “transparency” in AI systems. But he also decried the idea of quickly legislating on AI through “one-size-fits-all, overly rigid, and unduly bureaucratic laws.”

While proposals to regulate AI were contentious, one idea drew support from Republicans, the panel’s witnesses and even some Democrats: the need for industry to be paid for its innovations.

“As game-changing AI-enabled devices and other technologies emerge, Medicare coverage and payment policies must keep pace,” Crapo said in his statement.

Sen. Bob Menendez (D-N.J.) noted that according to recent research, Medicare reimburses “fewer than 20” AI services, before asking whether more payments would be useful. Other senators also indicated support for more cash to the industry.

Peter Shen, head of digital and automation at Siemens Healthineers, proposed that tech manufacturers submit cost data to Medicare and receive five years of payments. Otherwise, the government’s “inconsistent, unpredictable” approach would stifle innovation, he argued.

Sendak said health providers lack infrastructure and training to use AI, noting the billions of dollars the government spent to encourage them to adopt electronic health records. “We need similarly bold action now,” he said in written testimony.

This article is not available for syndication due to republishing restrictions. If you have questions about the availability of this or other content for republication, please contact NewsWeb@kff.org.