Ethics of AI in Healthcare: The Questions Asked First

Almost everything written about the ethics of AI in healthcare is a list of principles, and the striking thing about those lists is how completely they agree. That is not a scandal. It is a field that finished its definitional argument some time ago and has been restating the conclusion ever since. What has changed, and what almost no page on this subject has caught up with, is that since 2 August 2026 several of those principles have stopped being propositions and become questions that a specific person with authority will ask you, in a specific order, with a documented answer expected.

This is the article about the questions.

The principle lists agree with each other, which is the problem

The World Health Organization published six principles in 2021 after eighteen months of expert deliberation: protecting human autonomy, promoting well-being and safety and the public interest, ensuring transparency and explainability, fostering responsibility and accountability, ensuring inclusiveness and equity, and promoting AI that is responsive and sustainable.

The FUTURE-AI consortium, 117 interdisciplinary experts from 50 countries working over two years, published six of its own: fairness, universality, traceability, usability, robustness and explainability, operationalised as 30 best practices covering design, development, validation, regulation, deployment and monitoring.

And a scoping review of 227 peer-reviewed papers from 33 countries found that every patient-related ethical issue in the healthcare AI literature mapped onto the four principles of biomedical ethics that predate AI entirely: beneficence, non-maleficence, respect for autonomy and justice. Its recommendation is that these four should serve as the foundation for AI governance and policy in healthcare, which is a fair recommendation and also a concession that they are not yet the operative test for anything.

A systematic review published in February 2026 screened 243 publications down to 22 and drew the most useful conclusion available on this shelf. It found a strategic imbalance, in which transparency and bias mitigation are heavily emphasised while continuous monitoring and stakeholder involvement receive less attention, and named the consequence: an over-reliance on technical solutions at the expense of governance processes.

That is the gap. A settled vocabulary is not a decision procedure. Below is what the procedure now looks like.

Question one: is this system high risk, and who made that call?

The EU AI Act applies from 2 August 2026, with staggered dates under Article 113, and systems that are themselves products or safety components of products covered by the harmonisation legislation in Annex I Section A apply from 2 August 2027. For most clinical software, the classification is not an ethics committee judgement at all. It follows from what the device already is.

The Commission's own worked example is a surgical robot with an AI navigation module. The robot is a medical device requiring third-party conformity assessment under Regulation (EU) 2017/745, so the navigation module is a high-risk AI system, and compliance is assessed inside the conformity assessment already being carried out, producing a single declaration of conformity. Nothing about that turns on how ethically sensitive anyone judges the tool to be.

The practical consequence for a buyer is that the first question is answerable from the paperwork, not from a workshop. Whoever tells you the classification should be able to name the legislation it comes from.

Question two: are you the provider or the deployer, and do you know?

This is the one in-house teams get wrong, and it is worth reading twice.

Obligations under the Act attach not only to placing a system on the market but to putting it into service. A provider that develops an AI system and uses it exclusively for its own purposes within the Union has put that system into service, and is subject to the obligations of a provider even though it never supplied the system to anyone. The same legal person is then both provider and deployer, and carries both sets of obligations.

Read that against how hospitals actually build. A trust that fine-tunes a model on its own record data and runs it in its own wards has not avoided the vendor obligations by not being a vendor. It has acquired them. This is where the ethics literature and what AI regulatory compliance involves in practice stop being separate subjects.

Question three: can a named person override it, and do they know they can?

Article 14 of the consolidated text requires that high-risk systems be designed so they can be effectively overseen by natural persons during the period in which they are in use, with measures commensurate with the risk, level of autonomy and context of use, built into the system by the provider or implemented by the deployer, or both.

The substance is in paragraph 4, which reads like a checklist because it is one. The person assigned oversight must be enabled to understand the system's capacities and limitations well enough to detect anomalies and unexpected performance; to remain aware of the tendency to over-rely on the output, which the statute names as automation bias; to correctly interpret the output given the interpretation tools available; to decide in any particular situation not to use the system or to disregard, override or reverse its output; and to intervene or interrupt through a stop button or similar procedure that brings the system to a halt in a safe state.

Automation bias is a finding from human factors research. It is now a design requirement in European law. That transition, from something true about people to something a provider must build for, is the clearest single illustration of what has happened to this field.

Note also what the question is not. It is not whether a human is nominally in the loop. It is whether a named person has the training, the interface and the authority to say no, and whether anyone has checked that they know it.

Question four: what happens the next time the model changes?

WHO's responsiveness principle asks designers, developers and users to continuously and transparently assess AI applications during actual use, to determine whether the system responds adequately to communicated and legitimate expectations. As a principle, that is unobjectionable and unenforceable.

The implementation guidance converts it into a documentation test for systems that keep learning. A change avoids triggering a fresh conformity assessment only if it was pre-determined before the initial assessment, described in the technical documentation together with the technical solutions ensuring continuous compliance, and stays inside the envelope so described. And the guidance is blunt about what does not count: it is not sufficient for a provider to state in general terms that the system will continue to learn. The documentation should describe the nature and bounds of permitted change, the data learning may take place on, the performance and fairness metrics to be monitored, the thresholds beyond which the system will be halted or rolled back, and the human oversight measures applying to the learning process itself.

Ask for that document. If it does not exist, the honest reading is that nobody has decided in advance what would count as the model going wrong. Budget for the monitoring accordingly, because it is one of the costs a correctly working system still imposes rather than a one-time expense.

Question five: what was the patient actually told?

Ambient clinical documentation is among the most prominent emerging uses of AI in healthcare, adopted by health systems at scale, and it is the sharpest test of the autonomy principle because it captures the consultation itself.

These systems passively record and transcribe conversations between provider and patient, which means they capture not only medical information but personal details, family dynamics and intimate health concerns, and research on current consent and disclosure practices suggests patients may not receive adequate information before ambient AI is used, including details about data storage or speech analysis. Without that, consent is not meaningfully informed. The same recordings generate large volumes of data that may be valuable for research, quality improvement or commercial purposes, so the consent question extends past the appointment into secondary use.

The question to ask, then, is not whether consent was obtained. It is what the patient was told, when, and by whom, and whether the disclosure covered where the recording goes afterwards. Who ends up carrying the consequences of getting that wrong is set out in the risks of AI in healthcare, sorted by who carries them.

Question six: whose values are in the objective function?

This is the question no conformity assessment asks, and the one most likely to determine whether a deployment was a good idea.

Every ranking system needs a metric, and choosing the metric is a value judgement performed by engineers. The example that made the point concrete is IBM's Watson for Oncology, which ranked treatments according to improvements in length rather than quality of life, and did not encourage doctors and patients to recognise treatment decision making as value-laden at all. The authors call this machine paternalism, and it generalises to any system that prioritises one recommendation over another, because prioritising requires a metric and every metric excludes something a patient might have valued more.

The same analysis notes that where sensitivity and specificity thresholds are set involves complex value judgements that warrant public ethical deliberation, and gives the DermAssist case, where it has been claimed that high false-positive rates were accepted in dark-skinned patients, where data was scarce, to protect against missing a melanoma. Both directions of that trade cause harm. Someone chose which harm to accept.

There is a corollary here that cuts against a common procurement reflex. Explainability is not the answer to this question. What matters most is justification, not explanation: explanation says how the system reached its output, justification says whether the act was right, and it is grounded in values the patient endorses plus adequate scientific validity. A perfectly interpretable model optimised for the wrong endpoint remains a model optimised for the wrong endpoint, and it will now explain that to you fluently.

The useful version of this question at a procurement meeting is short. What is the system maximising, who chose that, and what did choosing it rule out?

The question the compliance frame does not ask

The six questions above test whether a system is safe for the people it is used on. None of them tests who never gets used on.

Justice and equitable care sit alongside the procedural concerns in every serious survey of this field. The PLOS Digital Health analysis of clinical integration places justice and fairness, transparency, consent and confidentiality, accountability, and patient-centered and equitable care as five concerns of equal standing, and observes that most regulatory frameworks are post-hoc, discouraging rigorous ethical review at the early stages where it would change a design. No gate in the conformity procedure asks whether a tool was validated on the population it will meet, or whether the populations left out of the training data are the same ones left out of the market.

Which is why the answer to it cannot be a document. It has to be a room. A review of the ethical and legal issues in healthcare AI proposes exactly that: an interdisciplinary committee including ethicists, AI engineers and developers, healthcare professionals, patients and health organisation administrators, plus a lawyer, assessing a technology against predefined criteria before it can be used, with the assessment covering individual and collective benefits and risks rather than scientific validity alone.

That is the composition problem the EX Future Summit is built around. The AI Ethics track runs alongside Health Tech and Government among the nine program tracks, because the answers to questions one through six live in different professions and none of those professions can write the assessment alone. The summit runs 18 to 20 November 2026 as a single continuous thirty-hour broadcast between Las Palmas and Bali, twelve hours apart, and online attendance is free for verified researchers, students and the EX community. The structural version of this argument is AI governance in healthcare; the practice built on top of it is responsible AI in healthcare.

FAQ

What are the four ethical principles of AI in healthcare?

Beneficence, non-maleficence, respect for autonomy and justice. These are the four principles of biomedical ethics, and they were not written for AI. A scoping review of 227 peer-reviewed papers from 33 countries found that every patient-related ethical issue raised in the healthcare AI literature mapped onto at least one of them, which is a strong argument for using the framework clinicians already know rather than adopting a new one for every technology.

Is AI in healthcare regulated by the EU AI Act?

Yes, and for most clinical tools by way of already being a medical device. The Act applies from 2 August 2026 under staggered dates, and systems that are themselves products or safety components of products under the Annex I Section A harmonisation legislation, which includes the Medical Device Regulation, apply from 2 August 2027 and are assessed inside the existing conformity assessment procedure. Providers and deployers of high-risk systems intended for use by public authorities have until 2 August 2030 to comply.

Who is legally responsible when medical AI causes harm?

Start by establishing whether you are the provider, the deployer or both, since a hospital that builds a tool for its own use is both. On the ethical question, responsibility tracks the degree of control an agent has over the outcome and the foreseeability of the consequences, so the buck generally stops with the treating clinician who has taken on responsibility for the patient, unless there was a fault in the technology that could not have been foreseen or discovered by that clinician.

Does explainable AI solve the ethics problem in healthcare?

No, and treating it as the answer is a common and expensive mistake. Explanation and justification are different things: explanation describes how a system reached its output, while justification concerns whether the act was right, and rests on values the patient endorses together with adequate scientific validity. Explainability also trades against privacy, since the data sharing needed for auditing and improvement increases the risk of re-identification.

What ethical review should a hospital run before deploying an AI tool?

An interdisciplinary committee assessing the technology against predefined criteria before use, covering ethicists, AI engineers and developers, healthcare professionals, patients and administrators, with a lawyer included to handle the legal questions and track how the law changes. The assessment should weigh individual and collective benefits and risks, not scientific validity alone. The larger design flaw in current practice is that continuous monitoring and stakeholder involvement receive far less attention than transparency and bias mitigation, so a one-time sign-off is the failure mode to design against.