Hello, healthcare innovators—lately I’ve been exploring Vertex AI, Google’s new offering in agent-based generative AI. In the spirit of open innovation in healthcare, I'm sharing what I've learned below:

What agent-based generative AI (or agentic AI) is
How Vertex AI works in healthcare systems
4 potential uses to improve patient care – and key challenges
My recommendations for when and how healthcare leaders should adopt Vertex AI

What is agentic AI?

To tackle complex healthcare problems, individual AI agents are brought together to collaborate on patient diagnosis and treatment in a multi-agent AI system, or agentic AI. Think of agentic AI like a team of specialists in a hospital, each bringing their own expertise to the table. The radiologist interprets medical images, the pathologist analyzes tissue samples, and the oncologist devises a treatment plan based on combined findings.

In agentic AI, each AI agent is specially trained for a specific task. They share findings and assess variables, working together seamlessly and tirelessly toward common healthcare goals – from informing initial consultations to optimizing patient scheduling. Sounds like a healthcare support dream team, right? Possibly – but the concept is still in its early days, with Google's recent release of Vertex AI and Search for Healthcare.

How does Vertex AI work?

Google's advanced agentic AI offerings are still in preview mode, but developers are already using them to build assistive AI technologies for clinicians and health workers. Vertex AI Search runs medically-tuned searches on a wide spectrum of data, from clinical notes to FHIR (fast healthcare interoperability resources) data. To surface relevant insights from this sea of data for more informed care, Vertex AI is tightly integrated with MedLM (Google's medically-tuned language models), the Healthcare Data Engine (HDE), and Cloud Healthcare FHIR APIs.

This enhanced toolkit promises to unlock deeper, more contextually relevant insights from structured and unstructured patient records. But as I discovered through testing, it performs some healthcare functions better than others.

What's the potential of Vertex AI to impact care?

To assess Vertex AI's capabilities for advancements in healthcare, I tested Vertex AI Agent Builder with Google's Search for Healthcare component. Here are the four most promising use cases I've found.

1. Task specialization and efficiency

Healthcare can get hectic with fragmented, inefficient workflows. Agentic AI systems developed on Vertex AI promise to streamline processes through focused specialization and enhanced search capabilities. For example, I can see potential for AI agents tightly integrated with Vertex AI Search for Healthcare features to improve patient scheduling in three particular areas:

Querying FHIR data. When a patient requests an appointment, an AI agent could use Vertex AI Search's medically-tuned search on FHIR data to quickly retrieve the patient's medical history. It could summarize past procedures and lab results, check for allergies, surface medication interactions, and identify other potential conflicts with the appointment being scheduled.
Search across clinical notes. AI scheduling agents could also examine unstructured, in-depth notes using Vertex AI Search's capabilities. Depending on the complexity of the patient's needs and related history, AI could assist in suggesting appropriate appointment durations and specialist consultations.
Search in conjunction with MedLM. Integrating with MedLM would allow AI scheduling agents to scan the patient's records, surface relevant guidelines, and find the latest diagnostic and treatment recommendations for patient needs.

2. Patient triage

Vertex AI Search has potential to improve initial patient interactions with search-driven triage. Currently, many AI triage systems support busy care teams by matching keywords to a decision tree. Vertex AI Search could enhance this process in three ways:

Understanding medical nuance. When a patient reports "shortness of breath," Vertex AI Search can quickly scan records not only for that phrase, but also for contextually relevant diagnostic terms like asthma, cardiac events, or COPD (chronic obstructive pulmonary disease). Search results might also uncover useful insights in notes where the patient described this symptom in different terms, like wheezing or gasping.
Risk factor identification. An AI agent could use Vertex AI Search on structured FHIR data to find relevant lab results. It could also highlight possible contraindications, such as medications that may exacerbate the issue or demographic factors (like age or smoking) that increase risk levels.
Suggesting next steps. During triage, teams need to assess risks quickly and take decisive action. Beyond assigning a case a generic risk level of high or low, the AI agent could take search results into account to identify specific risks. It could also suggest an appropriate course of urgent action, such as "immediate care needed," "consult with primary care physician within 24 hours," or "monitor symptoms and report any worsening."

3. Collaborative decision-making capabilities

Agent-based AI could help care teams provide informed, immediate answers to patient questions about their existing health records and medical information. This promotes informed, collaborative decision-making on a range of patient care concerns, from medication coverage to post-procedure care instructions. Here are three ways that might work in practice:

FHIR data check. Doctors hear one question regularly: "Does my insurance cover this medication?" To provide an immediate answer, an AI agent could query their insurance details and medication formulary via the Healthcare FHIR API. The agent could deliver a "yes/no" answer, as well as any copay information. If the medication isn't covered, Vertex AI Search could scan clinical notes and relevant guidelines for therapeutically equivalent alternatives—potentially even suggesting ones known to be on the patient's formulary. Vertex AI Search recognizes related medical terminology both within the question and the patient's record, opening up the search for viable, affordable options.
Locate relevant discharge notes. Following a surgical procedure, patients often ask, "What should I expect after surgery?" Using FHIR APIs, the AI agentic can retrieve the patient's most recent discharge instructions.
Tailored guidance. Vertex AI Search would then scan relevant clinical guidelines or post-operative care protocols to provide more comprehensive information. The AI agent combines the findings into an easy-to-understand response, addressing the patient's specific surgery and potential recovery timeline.

4. Regulatory compliance and data quality

To uphold standards of patient care and privacy, patient data must be reliable and secure. Vertex AI Search for Healthcare could improve compliance and data management in three specific ways:

Enhanced compliance checks. AI agents could leverage Vertex AI Search for Healthcare to analyze medical records and transcribed notes for potential HIPAA violations, such as unexpected mentions of PHI in easily accessible fields.
Flagging inconsistencies. Using medically-tuned search, AI agents could flag potential discrepancies within records—including conflicts between structured data (such as listed diagnoses) and unstructured data (like physicians' notes). This helps keep healthcare data accurate and up-to-date.
Verifying factuality. The factuality feature of Vertex AI Search for Healthcare could provide healthcare providers with a critical cross-check for documentation. This feature promises to check, source and cite related data points, increasing confidence in documentation. An AI agent might also use the factuality feature to recommend appropriate clinical intake templates for specific use cases, or suggest standardized terminologies for consistency in documentation.

Key challenges of agentic AI in healthcare

While testing gives me reason for cautious optimism about Vertex AI's potential in the four healthcare instances outlined above, key challenges remain to its broader adoption. For healthcare leaders, the following concerns aren't technicalities, but top of mind before deploying agentic AI in sensitive scenarios.

Patient safety. Even with Vertex AI Search's medically-tuned search and factuality features, errors or biases within AI agents could lead to adverse outcomes. Rigorous human oversight and testing in simulated environments are crucial safeguards.
Data security. Privacy is paramount in healthcare. Integrating AI agents requires meticulous security protocols to prevent breaches and maintain patient trust.
Explainability. The "black box" nature of some AI models can be an obstacle to effective healthcare. Explaining why an AI agent suggested a specific course of action is essential for both trust and legal accountability.
Unintended consequences. Even when focused on administrative or support tasks, AI integration can have ripple effects. Healthcare leaders must consider potential disruptions to established workflows, unexpected liabilities, and the possibility of deskilling if staff become overly reliant on AI outputs.

Watch this space: 3 recommendations for agentic AI

Through testing, I've uncovered four areas of significant promise for agentic AI in healthcare—but also four persistent challenges. Agentic AI systems hold potential to transform healthcare workflows, and Vertex AI's medically tuned search and factuality capabilities could make it a useful ally for care teams handling urgent, complex care challenges. But healthcare leaders must also factor established standards for safety, privacy, and patient care into their adoption plans.

I see a way forward to equip providers with agentic AI tools for informed, efficient care—while protecting vulnerable patients against system error and breaches of standards. An informed approach to agentic AI in healthcare starts with a gradual, rigorously monitored rollout, including three key adoption strategies:

Aim to augment human analysis. Vertex AI Search for Healthcare could provide valuable, timely insights with its ability to scan vast datasets and cite credible sources. This can help healthcare professionals make informed care plans that work for patients, building trust over time with improved outcomes and explainability.
Build in cross-disciplinary collaboration. Successful deployment hinges on broad, deeply specialized expertise. To guide adoption from planning stages through rollout, team leadership should include medical technologists, healthcare practitioners, and legal specialists.
Evaluate in strict staging environments. Before affecting live systems, pressure-test any agentic AI on narrow, specific use cases in sandboxed testing environments. Promising though it seems, Vertex AI Search's integration with FHIR data and clinical notes should undergo rigorous evaluation to ensure search results are both accurate and clinically relevant.

That said, these are my findings to date – always happy to discuss and compare notes.