Why Build an App for Blind Therapists?

Today is Global Accessibility Awareness Day #GAAD #gbla11yday and I am pleased to announce that my app the Patient is in provides the best accessibility features in a real time, HIPAA-compliant messaging app for the Apple Watch and Siri. Built over 15 months with leading doctors, therapists, and practitioners to support the needs of blind therapists, the Patient is in made its public debut at last week’s Digital Health Demo Day.

Many people ask me why did I first implement accessibility support in this ground breaking real time, HIPAA-compliant messaging app rather than add a localized version of the app for French, German, or Spanish speaking doctors and nurses. They correctly note that there is a larger market for real time medical messaging for French, German, and Spanish speaking doctors and nurses than there is for English speaking blind therapists and their nurses. But the fact is that most app developers struggle with just creating a useful and usable app that they just don’t care about such a small group of potential users. Unfortunately, the imagined profits from the many outweigh the needs of the few.

Here’s a secret that only a handful of app developers know about and the real secret why Apple has invested billions of dollars in accessibility to support the needs of a small group of its users: if you design accessible products both your underlying technology and product will be more usable by the majority of your users.

This does not discount the obvious compassion which has driven Apple engineers and managers to support all types of users with both accessible features as well as foreign language support since the 1980s. As Apple watcher Rene Ritchie from iMore.com often says: “Apple engineers create products which their families use so they try to build the best and usable products”. This is why Apple products have soul and many products from Google and Microsoft lack soul. How many CEOs have you seen brought to near tears when discussing the empowerment of blind users of their products? Watch the end of the then new CEO Tim Cook’s first WWDC keynote in 2012 and you will witness a soulful man.

If you are a press person, query your contacts to see how many of Apple’s top human factors/user experience engineers worked on the vibrations and sounds used on the Apple Watch. And their requirements were supported by great hardware engineers who worked on the Taptic engine. By the way, notification of a new or urgent patient assignment through “discrete vibrations felt on the doctor’s wrist” continues to be the most useful app feature for most doctors demoing the Patient is in and that feature was originally built to support notifying a blind therapist that he had a new patient assignment.

Beyond the engineering benefits in building accessible apps, here is another reason why we should think about blind therapists on Global Accessibility Awareness Day 2017: blind therapists support our blind children as they start learning about the world and assist them as they transition into adulthood. Blind therapists also help our parents during the transition brought on by age-related macular degeneration.

 

The Patient is in accessibility features

The Patient is in provides access to patient assignments and updates in 4 ways, all of which have been designed to support low-vision and blind doctors and therapists:

  • The watch app supports VoiceOver and provides a great user experience.
  • Patient assignments from the charge nurse can be automatically announced over  AirPlay compatible speakers such as Apple’s HomePod. This removes the step in which the user must first access the iPhone notification to find out about the assignment details
  • Conversations with the charge nurse using Siri on the iPhone: “Hey Siri, read my Patient messages” allows the therapist to hear her patient assignments and “Hey Siri, send a Patient message saying I’ll go to Therapy Room 2 in 10 minutes” allows the therapist to respond to assignments from either her Apple Watch or iPhone with an estimated time of arrival. Notice of assignment completion is supported from both the Apple Watch and iPhone with “Hey Siri, send a Patient message saying I’ve completed my assignment in Recovery Room 2”. All of these statements are processed by the Patient is in natural language processing (NLP) engine fixing homophones (“Exam Room 2″ vsexam room to” vsexam room too” vsexam room two”) and other linguistic and transcription impediments to create a structured message upon which the charge nurse’s iPad app can visually display and drive real world processes such as cleaning the room so that the next patient’s wait time is significantly reduced.
  • FaceTime support is integrated directly into all of the apps allowing the charge nurse, therapist, and other doctors to instantly connect. This includes initiating a FaceTime audio call directly from the watch app.

… just my bit to inspire other app developers for Global Accessibility Awareness Day 2018

Accessibility and Blind Therapists

Support for low-vision and blind therapists

The Patient is in provides access to patient assignments and updates in 4 ways, all of which have been designed to support low-vision and blind doctors and therapists:

  • The watch app supports VoiceOver and provides a great user experience. Click here for more information about the app for Apple Watch.
  • Patient assignments from the charge nurse can be automatically announced over AirPlay compatible speakers such as Apple’s HomePod. This removes the step in which the user must first access the iPhone notification to find out about the assignment details
  • Conversations with the charge nurse using Siri on the iPhone: “Hey Siri, read my Patient messages” allows the therapist to hear her patient assignments and “Hey Siri, send a Patient message saying I’ll go to Therapy Room 2 in 10 minutes” allows the therapist to respond to assignments from either her Apple Watch or iPhone with estimated time of arrival and notice of assignment completion. Click here for more information about Siri integration with the app.
  • FaceTime support is integrated directly into all of the apps allowing the charge nurse, therapist, and other doctors to instantly connect. This includes initiating a FaceTime audio call directly from the watch app.

An important note about the important role of blind therapists in society of which many of us are unaware: they support our blind children as they start learning about the world and assist them as they transition into adulthood. Blind therapists also help our parents during the transition of age-related macular degeneration. 

An Apple Watch App for Doctors

The Patient is in watch app notifies the doctor when a patient is ready to be treated with a discrete vibration felt on her wrist and allows her to provide an estimated time of arrival to see the patient. When the doctor has completed her assignment, she can notify the charge nurse that she has completed the assignment with a tap of a button or with Siri. The doctor may also decline the assignment if she is unable to attend.

Apple Watch Series 3 with Cellular: Patient assignments on the go

The Patient is in watch app supports the cellular capabilities of the Apple Watch Series 3 ensuring that the doctor can receive and respond to patient assignments:

watch-cellular-recoveryRoom2-complication-1-case

Notifications and Alerts

The charge nurse uses the iPad app to send a patient assignment to the doctor consisting of

  • The name of the room in which the patient is waiting
  • Important notes about the patient such as if he has elevated vital signs

and the watch app will alert the doctor with

  • A popup notification
  • A discrete vibration felt on her wrist
  • An optional, audible sound

Glanceable Information

Since the doctor may receive a new notification while she is treating another patient, the Patient is in watch app uses haptic feedback in the form of a silent vibration to alert her that a new notification was received from the charge nurse. The Patient is in watch app will display the new room name and abbreviated notes directly on the watch face affording the doctor a quick glance without disrupting her current patient’s treatment.

The following screen shots show how the app supports complications to implement glanceable information:

watch-complication-modular-assignment-1-case

 

watch-complication-utility-assignment-1-case

Actionable Information

When the doctor is able to respond to the assignment, the Patient is in watch app allows the following:

  • Review any patient notes provided by the charge nurse by tapping on the name of the room
  • Inform the charge nurse of the estimated time of arrival
  • Decline the assignment
  • Inform the charge nurse that the assignment is complete

The following screen shots show how the doctor uses the Patient is in watch app. First, a discrete vibration is felt on her wrist, optionally with a sound. A popup alert is also displayed and allows the doctor to quickly respond directly from the notification:

watch-notification-recoveryRoom2-assignment-1-casewatch-notification-recoveryRoom2-assignment-2-case

If the doctor is busy and misses the notification, she can access it by swiping down from the watch face. This will display all of her missed notifications as seen in this screen shot:

watch-notification-recoveryRoom2-assignment-3-case

 

With a tap on the notification, the Patient is in watch app is launched and visually displays the assignment allowing the doctor to quickly accept the assignment and provide an estimated time of arrival or decline the assignment. A timer displays how long has it been since the charge nurse sent the assignment. The doctor may tap the room name to review the patient notes and start an FaceTime audio call with the charge nurse if she needs clarification. Finally, when the doctor has completed the assignment, she can notify the charge nurse that she is done with a tap of a button as seen in the following screen shots:

 

 

If the doctor prefers, she can use Siri to perform theses tasks.

Siri support on Apple Watch

Using Siri, doctors can converse with the charge nurse to respond to patient assignments using the following phrases:

  • “Hey Siri, send a Patient message saying I’ll go to Exam Room 1 in 15 minutes”
  • “Hey Siri, send a Patient message saying I’ve completed my assignment in Recovery Room 2”
  • “Hey Siri, send a Patient message saying I cannot go to Operating Room 3”

And the following screen shots show the doctor using Siri to acknowledge a patient assignment and then notify the charge nurse that she has completed the assignment:

watch-siri-both-3-case

Click her for more information about creating a voice interface with Siri in the Patient is in.

Click here for more information about integrating Siri with an AI engine to reduce patient wait times in the Patient is in.

Audio and Video Conference Calls

Enhancing the convenience of mobile conference calls with advanced presence and geolocation rules, Apple’s FaceTime is deeply integrated throughout the app allowing doctors to initiate FaceTime calls directly from the watch app. Work-only FaceTime addresses may be defined in the iOS app to protect the privacy of personal FaceTime addresses and to support on-call doctors and other shift-based staff members.

To start an audio FaceTime call with another doctor, nurse, or charge nurse who is working today, the doctor use a Force Touch gesture on list of patient assignments which pops up a menu and then taps on the “Today’s Users” button. This is seen in the following screen shots:

 

 

 

The doctor may also initiate a FaceTime audio call to the charge nurse who sent the patient assignment when clarification about the patient assignment is needed. By tapping on the patient assignment room name, the doctor has access to the patient notes and the option to begin a FaceTime audio call as seen in this screen shot:

watch-notes-facetime-available-case

Support for low-vision and blind therapists

The Patient is in provides access to patient assignments and updates, all of which have been designed to support low-vision and blind therapists. For example, VoiceOver is comprehensively supported as well as Siri for voice access and FaceTime for audio conferencing with both the charge nurse as well as other available staff members as seen in following screen shots:

watch-4-voiceover-screens-1

 

Click here to learn more about the accessibility features of the Patient is in and why the app supports blind therapists.

 

A video of the Patient is in app for Apple Watch

The following video shows the Patient is in in action on Apple Watch:

 

Siri integration for a voice interface

The Patient is in apps are integrated with Siri on both the iPhone and Apple Watch to provide a voice interface for messaging.

Conversations with the charge nurse using Siri on the iPhone: “Hey Siri, read my Patient messages” allows the doctor to hear her patient assignments and “Hey Siri, send a Patient message saying I’ll go to Therapy Room 2 in 10 minutes” allows the doctor to respond to assignments from either her Apple Watch or iPhone with an estimated time of arrival. Notice of assignment completion is supported from both the Apple Watch and iPhone with “Hey Siri, send a Patient message saying I’ve completed my assignment in Recovery Room 2”.

All of these statements are processed by the Patient is in natural language processing (NLP) engine fixing homophones (“Exam Room 2″ vsexam room to” vsexam room too” vsexam room two”) and other linguistic and transcription impediments to create a structured message upon which the charge nurse’s iPad app can visually display and drive real world processes such as cleaning the room so that the next patient’s wait time is significantly reduced.

For a truly hands-free experience on the iPhone, the doctor needs only to use AirPods or another bluetooth headset and use the phrase “Read my Patient messages” after activating Siri either with a double tap on either AirPod or with the “Hey Siri” trigger phrase.

And then the doctor may respond to the assignment using the phrases: “Hey Siri, send a Patient message…” as seen in these Apple Watch screen shot:

watch-siri-both-3-case

and on the iPhone as seen in this screen shot we see the natural language processing (NLP) engine of the Patient is in fixing homophones (“Exam Room 2″ was transcribed by Siri as exam room to” ) and other linguistic and transcription impediments to understand the doctor’s intent:

iphone-siri-done-3-bezel

To ensure the best experience, the internal NLP engine used in the Patient is in apps has been optimized for Siri in a medical environment.

The following are some sample phrases which the doctor may use with Siri on both the iPhone, Apple Watch, and HomePod:

  • Send a Patient message saying I’ll go to Exam Room 1 in 15 minutes
  • Send a Patient message saying I’ve completed my assignment in Recovery Room 2
  • Send a Patient message saying I cannot go to Exam Room 2

Exclusive to the iPhone and HomePod, the doctor may ask Siri the following:

  • Read my Patient messages
  • What are my messages on Patient?

Understanding the AI integration with Siri

Click below to learn more about the technology powering the integration with Siri:

Videos of using Siri with the Patient is in app

In the following HomePod video, a group of anesthesiologists use Siri to respond to patient assignments:

In the following iPhone video, the doctor uses Siri to listen to her patient assignments: