Gregory Axton

Case study 02 · Interaction Design / AI / Research

Making AI decisions more understandable.

Designing and evaluating an explainable-AI interface for ambulance IT systems, with a focus on how EMS clinicians could understand, question and use dynamic risk predictions during high-stakes decision making.

Project
Explainable AI for prehospital care
Role
UX / Interaction Design Research
Context
Chalmers master’s thesis with David Wallsten + published paper
Location / Year
Gothenburg, Sweden · 2023
An ambulance IT tablet showing a trauma patient's vital signs, with an AI decision support panel giving a 78% moderate-to-high risk prediction, the key factors behind it and a missing variable.
Refined high-fidelity prototype: the AI decision support panel alongside the patient overview. Patient data is fictional.

01Context

High-stakes decisions leave little room for opaque technology.

Ambulance clinicians work under time pressure, cognitive load and rapidly changing conditions. The project explored how AI-driven risk prediction could fit into existing prehospital IT systems without removing the clinician’s ability to understand and judge the recommendation.

Problem / 01

AI may improve triage, but trust matters.

Explainability was treated as part of the interaction design problem: clinicians still make the final decision and need to understand what the system is basing a prediction on.

Design question / 02

How can dynamic AI decision support fit the ambulance workflow?

The design needed to work with existing systems, stay accessible during the workflow and communicate risk prediction without increasing unnecessary cognitive load.

02Research & Fieldwork

Understand the work before designing the intelligence.

The study combined literature research, ethnography, expert interviews and analysis of existing ambulance IT systems. Fieldwork included ambulance environments in two Swedish regions using different triage solutions.

  1. Discover / 01

    Literature review

    XAI, EMS, triage, healthcare AI and UX.

  2. Discover / 02

    Ethnography

    Observation of ambulance work and clinical routines.

  3. Discover / 03

    Expert interviews

    Five domain experts across clinical, research and industry contexts.

  4. Define / 04

    Workflow synthesis

    Journey maps, personas, task analysis and requirements.

A very wide Miro board showing the whole project: the literature map, fieldwork photos, sticky-note clusters, personas, flow diagrams, wireframes, prototype screens and test photos.
Process overviewThe whole project on one Miro board, from literature and fieldwork through synthesis, concepts and wireframes to prototyping and testing.
Six fieldwork photos: two ambulance crew members by their vehicle, the ambulance cab with navigation screens, hands typing on a rugged keyboard, a paramedic at a laptop in the station, a phone showing a dispatch map, and a researcher talking with a paramedic in the garage.
Contextual inquiryFieldwork at ambulance stations in Region Halland and Västra Götaland: the crews, in-cab screens, input devices and the tools they rely on.

Field notes

  • Some medics prefer a keyboard rather than typing on the screen.
  • The journal is filled in later when a case isn’t urgent.
  • The in-cab interface could be improved.
  • Maybe more AI in the questions they work through.
A literature map titled Future IT System for Improved Prehospital Care, branching into Cognition, Care, AI and UX, with groups for regulations, EMS, AI clinical decision support, EMS AI decision support and EMS decision support without AI.
Literature mapSources grouped under cognition, care, AI and UX, with clinical decision support studies, EMS research and Swedish regulations.
Four persona cards: Alfonso, an inexperienced paramedic; Maria, an EMS nurse trainee; David, an EMS nurse; and John, a senior EMS paramedic, each with motivations, obstacles, questions and experience ratings.
PersonasFour personas from a newly started paramedic to a senior EMS paramedic, each with their own questions about trusting AI.

03Workflow

Design around the clinician’s sequence of decisions.

Research mapped the prehospital workflow from dispatch through assessment, triage, treatment or transport, and later documentation. The AI concept was designed to remain available across that flow rather than forcing clinicians into a separate tool.

EMS workflow

  1. 01Receive mission
  2. 02Assess patient
  3. 03Collect parameters
  4. 04Triage + decide
  5. 05Treat / transport / record

AI overlay available throughout — not a separate tool

A hierarchical task analysis tree starting from triage patient and transport to hospital, broken into seven steps and detailed sub-steps for measuring vital parameters.
Task analysisHierarchical task analysis of triaging a patient and transporting them to hospital, down to how each vital sign is measured.
Two flow charts: an incoming 112 call routed through SOS Alarm and the Emergency Medical Dispatch Center to an ambulance, and the EMS nurse’s work from arriving at the patient to filling in the digital journal.
Flow chartsFrom a 112 call to an assigned ambulance, and the EMS nurse’s work from arrival to the digital journal.

04Requirements

Translate research into constraints the interface had to respect.

  1. 01Integrate with current and future ambulance IT systems rather than requiring an entirely separate platform.
  2. 02Fit the complex, time-sensitive EMS workflow and remain accessible during use.
  3. 03Increase trust and understanding by showing why an AI prediction was made.
  4. 04Reduce unnecessary interaction and cognitive load in stressful environments.
  5. 05Remain adaptable beyond trauma to other potential AI-supported clinical scenarios.

05Prototyping

Move from rough interaction ideas to a testable system.

Prototyping progressed from sketching and tangible concepts through digital wireframes and a high-fidelity Figma prototype. Expert feedback was used throughout before usability testing with clinicians.

A board of greyscale tablet wireframes showing patient input screens, a body map for trauma, vital-parameter tables and several dark AI-prediction overlay variants.
WireframesGreyscale wireframes of the tablet flow, from situation and warning symptoms to vital parameters, the trauma body map and early versions of the AI prediction overlay.
A Figma prototype map with blue high-fidelity tablet screens connected by many interaction links, including body-map screens and AI overlay screens.
Interactive prototypeThe high-fidelity Figma prototype, wired so clinicians could move through the full EMS workflow during testing.

06Final XAI Concept

Explain the prediction without taking control away from the clinician.

The tested explainable-AI overlay on an ambulance tablet: a yellow risk prediction with confidence 3 of 5, ranked predictors for and against a serious condition, and the most important missing variables.
Tested prototypeThe XAI overlay used in usability testing: triage colour, confidence on a five-step scale, predictors for and against a serious condition, and the most important missing variables. Figure 1 in the published paper.

An overlay that stays available throughout the workflow.

The final concept used an interactive overlay rather than a separate AI page. It could be opened from a notification, the top bar or a swipe gesture, while the underlying clinical workflow remained in place.

  • Risk prediction + confidence + adaptable guideline
  • Ranked predictors for and against a serious condition
  • Important missing variables that could improve confidence
  • Quick access without forcing clinicians through another navigation path
Five tablet wireframes connected by arrows: input data, an AI alert notification, the AI overlay, a confirmation popup for adding a missing predictor, and the overlay again with updated results.
Overlay interactionHow the overlay fits the workflow: input data, AI alert, overlay, quick entry of a missing variable, and an updated prediction. Figure 2 in the published paper.

07Usability Testing

Test the interaction with the people responsible for the decision.

The final prototype was evaluated with seven EMS clinicians, all ambulance nurses, using think-aloud testing and follow-up interviews. Participants worked through two patient scenarios and used the AI-supported interface as part of the triage process.

Two Figma prototype flows labelled Age 29, Traffic and Age 80, fall, each a chain of connected tablet screens.
Test scenariosTwo trauma cases based on real incidents: a 29-year-old in a traffic accident, a case that is often over-triaged, and an 80-year-old after a fall on a hard floor, which is often under-triaged. Each ran as its own interactive flow.

  1. Finding / 01

    Explainability supported trust.

    All EMS clinicians agreed that the design features explaining why a prediction was made were essential for trust.

  2. Finding / 02

    Predictors acted as reminders.

    Clinicians valued seeing important predictors and missing information because it could prompt them to reconsider what they had assessed.

  3. Finding / 03

    Disagreement could trigger reflection.

    When the AI prediction differed from a clinician’s assessment, participants described thinking again about whether something had been missed.

08Refined Prototype

Extend the concept across the whole ambulance workflow.

A refined prototype takes the idea beyond a single overlay. It covers the full case, from finding the patient to handover. AI decision support becomes a slide-in panel next to the patient record.

  1. Step / 01

    Assess

    Patient search, overview, vital parameters and structured clinical assessment.

  2. Step / 02

    Analyse

    AI risk prediction with key factors, missing variables and a what-if view to see how changes in vital signs shift the risk.

  3. Step / 03

    Support

    Interventions, timeline, auto-generated documentation, hospital pre-notification, transport and send-off.

Overview of twelve tablet screens for an ambulance IT system: patient search, overview, vital parameters, clinical assessment, AI decision support panel, interventions, timeline, what-if scenario, documentation, communication, transport and summary.
Refined prototypeTwelve screens across assess, analyse and support. The AI panel is labelled as a demo, and patient data is fictional.

09Publication

From thesis project to published research.

The work was developed as a 2023 master’s thesis at Chalmers University of Technology, co-authored with David Wallsten, and later published (Wallstén et al.) in Artificial Intelligence, Social Computing and Wearable Technologies.

Design for Integrating Explainable AI for Dynamic Risk Prediction in Prehospital IT Systems

AHFE · 2023DOI 10.54941/ahfe1004199

Next / Contact

Complex systems, made understandable.

All work

Next projectPhangan Horizon SchoolCase study in progress