Runtime Safety Mechanisms for Clinical Large Language Model Agents: A Review and Empirical Comparison

Language models (LLMs) that reason and act through tools are close to clinical deployment, where an irreversible action of prescribing an inappropriate drug cannot be undone by reasoning. This paper explores if any of the existing mechanisms can mitigate this risk. We review four categories of defense in the model, context, orchestration, and action planes of the agent stack, grouping them against five categories of known failures. The review discovers that the defense is abundant in the model and context planes and entirely absent in the action plane that is the only place where any irreversible harm happens, as well as no existing solution generates a deterministic decision log necessary for clinical audit. Finally, we compare the reviewed methods against the data from 1,846 agent instances programatically evaluated for six different base models. Deterministic filtering of tool calls resulted in 67.1% reduction of unsafe clinical actions and elimination of prompt injection attacks for all models tested, while context plane intervention was effective in reverse direction.