Health systems still struggle to scale AI beyond successful pilots, Healthcare IT News finds

The “successful pilot, failed rollout” pattern is one of the most consistently documented problems in health IT — it was the central critique of EHR implementations in the 2010s, and it is recurring with AI a decade later. Healthcare IT News’s August analysis identifies the same root causes: governance structures that weren’t built for ongoing model management, integration patterns that worked in one EHR environment but broke in another, training programs that weren’t sustained past go-live.

The AI-specific wrinkle is that pilots tend to involve motivated clinical champions and carefully selected patient populations, while enterprise rollout exposes the AI to the full clinical diversity of the health system. A sepsis prediction model that performed well in the academic medical center’s ICU may fire with very different characteristics in the community hospital’s step-down unit, where the patient population, documentation practices, and alert-response workflows are all different.

The governance dimension is the most fixable but least invested. Most health systems still don’t have a standing AI governance committee with the authority and expertise to set standards, review model performance, and retire underperforming AI. The ones that do — and Healthcare IT News profiles several in the analysis — are the ones making credible progress on enterprise scale.

The missing piece in most roadmaps is what could be called “AI operations” — the set of workflows, staffing, and tooling needed to manage AI as an ongoing operational function rather than a series of project implementations. This is distinct from IT operations (keeping systems running) and clinical informatics (optimizing clinical workflows). It includes model performance monitoring, bias surveillance, incident investigation, and lifecycle management. The organizations that build this capacity will be the ones that scale successfully.