Human Centered Design for AI in Healthcare

What I learned about designing for health care with health data with the help from my data scientists, engineers and SMEs.

Image created by a designer on my team: Braz De Piña

I am a firm believer in technology and that if used in a smart way, putting humans first, we can create some incredibly, impactful experiences. I work in the Technology and Research space in MSR, (Microsoft Research Organization.) On our team, we believe that focusing on the patient/caregiver will create a human impact flywheel of collaborative Healthcare solutions that can be cross-applicable to other important healthcare customers; the payor, the clinic systems, the researchers and clinicians. We designed a POC (proof of concept) and I worked with our engineering team to push the limits of what we could do with WebGPT and our health data model, we were already using for one of our products. After a 2 week experiment, I sat down with one of our data scientists, engineers and SME, Katie Claveau, to ask questions about the technical aspects of what we were building. This is what I learned: Healthcare not only needs AI to be driven by what is technically feasible and humanly desirable but it needs to be compassionate, safe, accurate and valuable.

We fundamentally do not understand how LLM’s do what they do. They are still very opaque and it’s early days. Unexpected capabilities are expected to emerge. This is critically important in healthcare and us as designers have to understand how GPT paired with data models work and how powerful this tool can be when building healthcare solutions.

GPT is an incredibly powerful type of large language model that can do a wide range of language related tasks, including generating original content that looks like a human wrote it. But how do we get it to deliver something with a tone-of-voice in the context of good product design for healthcare. We want to pair accurate AI responses with good UX writing standards. For example, we wanted the AI to respond at a 6th grade reading level, use a compassionate, optimistic, , realistic voice and return a short enough messages that the user can easily comprehend. We quickly realized we then needed to alter our approach to ask the AI to be not overly optimistic. When talking to Katie, about the importance of content guardrails, she mentions,

"When I went through some of the data science and I haven't done it all, I found that it was overly optimistic. If I didn't tell the AI to be realistic it it would spit out something like oh, you can live a long and meaningful life with congestive heart failure."

It is amazing how important traditional UX writing is here. It certainly applied to responses to prompts in products, whether its conversational or not. Applying standards to AI outputs could be greatly beneficial. Good product design writing meets a certain grade level depending on the product, applying passive voice, (in healthcare realistic, compassionate and mildly optimistic) formatting text and simplistic words for optimal readability.

When we talk about "safe" AI, I am talking about things that do not harm humanity. But to be human in itself, we are all individuals with different gauges on what what we consider harmful to us. Biases and discrimination in AI is a huge talking point, especially with tools like mid-journey returning biased images based off prompts from human input. I could go down a whole tangent on whether this makes the human bias in themselves, or just ignorant to good prompt writing.

Prompt writing in itself is an art form. As designers of an experience, we need to recognize good patterns, but also patterns of failure. Just like we all learned how to design with UI states, default, hover, and error states and messages. With prompts, we aren't just dealing with UI solution responses to text inputs. We need to deal with good content patterns with code. We can either adjust prompts to deal with those content failure patterns , or catch them in some sort of moderation endpoint. For example, if a human interacting with an AI writes a prompt asking about diabetes, and that AI happens to be biased against people with diabetes than it is our responsibility to build in moderations to our solutions to provide safety for humans.

In addition to moderation, we need to build trust through accuracy. Not in the way that every answer that AI outputs is 100% right all the time, but we need to provide the human with the power and ability to do something with the resources that they are provided. Just like in traditional UI you can typically edit or delete a Facebook post, the same should apply to AI experience. Humans need a way to check answers (ex. citations where we found the answer or links to an expert resource on the topic) or there needs to be a validation step (ex. do you want to add this to x,y,z?) or a way to dismiss suggestions. It is also very important to have AI admit when it is wrong or doesn't have the confidence to respond accurately. Trust is built when keep our use cases safe.

Here is my framework for working with LLMs and how it impacts the design process:

My final thought is that AI in healthcare products and services should be valuable. Daily, I hear "AI will take my job." I don't believe that, at least not for everyone. I have long believed that to deliver innovation to people you need to innovate with them and not for them. We must provide value in how humans use AI as a trusted collaborator and build that seamlessly into their workflows of our products and services. I take an active role partnering with my data scientists and engineers and SMEs to experiment with AI-enabled experiences that will positively impact humans in healthcare by continuing to design with people first. I am excited to to use traditional UX knowledge and apply it to the future of how we design for humans in healthcare.

---

Special thanks to the members on my team: Katie Claveau, Braz De Piña, Alex Garcia, Daniel Byrne, Daniel Tsirulnikov, Ernie Booth, Melanie Kneisel, Rich Guion, Michael Ricker, Leo Dominguez, Trevor Gruby, Megan Saunders, Suff Syed

Additionally : Sally Todd, Christopher Carter, Steve Leonard, Juan Valencia, Yara El-Tawil, Sapna Bafna, Jon Campbell, Paul Payne, Adam Glass, Pete Ansell, Jatin Sharma, Radu Trandafir