EN

EN

AI Governance in healthcare: the questions every supervisory board should ask

A hospital introduces an AI tool to reduce clinical documentation. A department pilots an algorithm to prioritise referrals. An employee uses a public chatbot to summarise a document containing patient information.


Each activity may begin as a local attempt to improve work. Together, they raise a board-level question: does the organisation understand where AI is being used, what value it creates, and who is accountable when something goes wrong?


For healthcare supervisory boards, AI oversight spans four connected areas: strategic opportunity, accountability, legal exposure, and stakeholder trust. Across all four, value must include patient safety, access to care, workforce capacity, and financial sustainability.


Effective oversight begins by connecting AI to the organisation’s mission.

1. Assess opportunity through the lens of better care


Boards should challenge management on the opportunities AI presents as seriously as they challenge its risks. Whether an application supports documentation, scheduling, or clinical decisions, it needs a clear account of the problem being solved.


A compelling demonstration is only the starting point. What improvement does management expect, how will it be measured, and when should an initiative expand or stop? For documentation tools, time savings must account for reviewing and correcting output. Does their use lead to more patient contact, less overtime, or greater capacity while maintaining quality?


Which two or three healthcare priorities could AI materially improve, and how will we know that it has?


This keeps investment focused on outcomes. Inaction also has consequences: failing to assess useful technology may mean missing opportunities to improve care and relieve pressure on staff.

2. Make accountability explicit


For each significant AI application, management should identify who owns the outcome, who approves deployment, who monitors performance, and who can suspend its use. For applications that influence care, clinicians and clinical leaders must be meaningfully involved.


“A human remains in the loop” is insufficient. That person needs the time, competence, information, and authority to challenge the output. Otherwise, errors may go unnoticed because people place too much confidence in AI.


This becomes especially important when AI can update records, send communications, or initiate workflows. Permission to suggest an action and permission to execute it require separate decisions.


What decisions and actions have we delegated to AI, within what limits, and who can intervene?

3. Treat compliance as a foundation


AI assessment should reflect the intended use and the consequences of failure. A tool drafting meeting agendas requires different scrutiny from a system influencing clinical triage.


Legal and regulatory review belongs within this process, but compliance alone does not establish suitability for the organisation’s patients, staff, and workflows. Has performance been evaluated in the intended setting? Could particular patient groups be disadvantaged? What happens if the tool fails or produces unreliable results? How are changes assessed after deployment?


The board does not need to conduct technical evaluations itself. It needs assurance that appropriate evaluations happen and that unresolved concerns reach people authorised to act.

4. Protect trust through everyday practice


An AI policy matters only if staff understand it and can follow it. Management therefore needs visibility across enterprise initiatives, departmental experiments, and individual use, covering both clinical and administrative work.


This includes AI embedded in existing software and public chatbots used because approved alternatives are unavailable or impractical. Clear boundaries must be supported by accessible tools, understandable guidance, training, and routes for reporting mistakes and concerns.


Do staff know how to use AI responsibly, and have we made that possible in their daily work?


Patients also deserve attention: how will they be informed when AI materially affects their care, and how will their concerns inform decisions?

Keep suppliers in view


AI enters organisations through new purchases and software updates. Management should understand what information suppliers can access and use, how system changes are communicated, and what happens if a service fails. For critical applications, supplier dependence and the feasibility of switching deserve attention. Outsourcing does not remove the organisation’s responsibility to decide whether and how AI should be used.

Give the board information it can act on


The supervisory board’s role remains to oversee strategy, challenge management, and assess whether governance is adequate. It should exercise informed scrutiny while leaving implementation with management.


That requires enough AI literacy to distinguish a promising demonstration from evidence of reliable performance. Periodic learning sessions can help, particularly when they use examples from the organisation’s own operations.


Reporting should then connect opportunity, performance, and risk. A useful board report would show the most consequential applications, the outcomes achieved, significant incidents or concerns, and decisions requiring board attention. It should include clear triggers for escalation between scheduled meetings.


For the next board meeting, five questions provide a practical starting point:

  1. Where is AI already being used, including informal use and supplier features?

  2. Which priorities should it advance, and what evidence will demonstrate value?

  3. Who owns the outcomes, risks, and authority to stop each significant application?

  4. How are patient safety, privacy, equity, and staff concerns assessed and monitored?

  5. What developments require immediate escalation to the board?


Healthcare boards do not need certainty about every future development in AI. They need a governance approach that makes consequential uses visible, assigns responsibility, and responds when evidence changes.


That is how oversight can support innovation while remaining anchored in the purpose of healthcare: better care, delivered safely and responsibly.

See it yourself

Discover what Delphyr can do for your organisation, and how to make AI work safely in daily care.

Discover what Delphyr can do for your organisation, and how to make AI work safely in daily care.

Delphyr

Helping healthcare professionals reclaim their time.

Contacts

Delphyr B.V.

IJsbaanpad 2

1076 CV Amsterdam

Netherlands

Follow us

2026 Delphyr. All rights reserved.

Delphyr

Helping healthcare professionals reclaim their time.

Contacts

Delphyr B.V.

IJsbaanpad 2

1076 CV Amsterdam

Netherlands

Follow us

2026 Delphyr. All rights reserved.

Delphyr

Helping healthcare professionals reclaim their time.

Contacts

Delphyr B.V.

IJsbaanpad 2

1076 CV Amsterdam

Netherlands

Follow us

2026 Delphyr. All rights reserved.