patientisin

February 12, 2018February 13, 2018 patientisin Technology

HomePod and Multi-User Siri in Health Care: It’s how to significantly reduce patient wait times

Since the release of version 2.0 of the Patient is in app, Siri has been supported in both the Patient is in iPhone app and the Patient is in Apple Watch app providing doctors with a voice first user interface. With version 2.5 of the Patient is in, multi-user Siri is now supported on Apple’s HomePod wireless speaker.

Wireless patient announcements with AirPlay speakers

If you deploy an AirPlay-compatible speaker such as Apple’s HomePod on your network, the Patient is in iPad app can wirelessly announce patient assignments. As seen in these screen shots, the charge nurse selects the AirPlay control from the navigation bar:

ipad-navbar-accepted-1

and a pop up dialog will display available AirPlay-compatible speakers such as the HomePod in the Anesthesiologists’ Lounge and the Apple TV in Dr. Quinn’s office:

airplay-controls-anesthesiologist-lounge-selected

Once an AirPlay-compatible speaker is selected as in the above example, then whenever the charge nurse assigns a doctor to treat a patient in a room, the HomePod in the Anesthesiologists’ Lounge will announce the patient assignment. As we will see in the video at the end of this blog post, the charge nurse assigns an anesthesiologist to Recovery Room 1 and the HomePod will announce: “The charge nurse says that the anesthesiologist has an assignment in Recovery Room 1”. And to ensure HIPAA compliance, any patient notes provided by the charge nurse will not be read aloud.

Multi-user Siri and HomePod

Much like on the Patient is in iPhone app, the doctor may use Siri to hear her patient assignments with the phrase: “Hey Siri, read my patient messages” and on both Patient is in apps for iPhone and Apple Watch, the doctor may respond to the patient assignment with “Hey Siri, send a patient message saying I’ll go to the exam room in 10 minutes” or “Hey Siri, send a patient message saying I’ve finished surgery”; now doctors can listen to their assignments on HomePod and respond to the charge nurse with Siri. As we see in this video, the charge nurse assigns the on-duty anesthesiologist to different recovery rooms and different anesthesiologists respond using Siri on the same HomePod:

For more information about Siri support in the Patient is in app, visit these links:
Siri integration for a voice interface and Et tu Apple Watch: Integrating Siri with an AI engine to reduce patient wait time

February 12, 2018February 12, 2018 patientisin Technology

Apple Watch Series 3 with Cellular: Patient Assignments on the Go

Since its introduction, the Patient is in watch app supported WiFi connections allowing the doctor to use her Apple Watch while away from her iPhone. With the release of version 2.5 of the Patient is in, the doctor will now receive patient assignments over a cellular connection. The doctor may respond to the patient assignment directly from the notification as seen in theses screen shots:

watch-notification-recoveryRoom2-assignment-1-case

or directly from the main watch app as she normally would as seen in these screen shots:

watch-mainApp-new-assignment-OR1-1-case watch-mainApp-done-assignment-OR1-1-case

As in previous versions of the Patient is in watch app, Siri as well as FaceTime audio conference calls are also supported over cellular as seen in these screen shots:

watch-siri-both-3-case watch-notes-facetime-available-case watch-todayStaff-1-case

For more information on the the Patient is in watch app, please visit:
An Apple Watch App for Doctors

February 8, 2018February 8, 2018 patientisin Technology

Reducing Patient Wait Times with AI and Microsoft’s Azure Cloud: Integrating Virtual Assistants, Wearables, and IoT

The Patient is in reduces patient wait times by integrating a series of technologies including AI and natural language processing, the Siri virtual assistant, the Apple Watch wearable, and the iBeacon IoT micro-location technology.

In this blog post, we will discuss two features of the Patient is in app which use Microsoft’s Azure cloud: Apple Watch notifications and Skype for Business. The Azure cloud was selected over Amazon’s AWS cloud as well as Google’s and IBM’s cloud for many reasons but the most important is that Azure supports the latest iOS and Apple Watch technologies and provides greater flexibility in managing devices.

Realtime patient assignments and updates

From the perspective of your doctor, the charge nurse sends a patient assignment from the Patient is in iPad app to the doctor’s Apple Watch and iPhone. On her Apple Watch, the doctor can accept the assignment and provide an estimated time of arrival, reject the assignment, and update the charge nurse that she has completed her assignment and thereby allowing the cleaning staff to prepare the room and consequently reduce the wait time for the next patient.

In this screenshot, we see the charge nurse using the Patient is in iPad app to visualize doctor availability and room status:

ipad-assignments

and here we see the charge nurse creating a patient assignment by assigning a doctor to treat a patient in Recovery Room 3:

The patient assignment is encrypted and securely sent from the Patient is in iPad app to the Microsoft Azure cloud which notifies the doctor on her Apple Watch of the assignment as we see here:

watch-notification-OR1-assignment-1-case
The Patient is in watch app will alert the doctor with:

A discrete vibration felt on the doctor’s wrist
An optional, audible sound

The doctor can then update the charge nurse with her estimated time of arrival and likewise updated the charge nurse after she completes her assignment as seen here:

From the main watch app, the doctor can reject the assignment or accept it and provide an estimated time of arrival

When the assignment is complete, the doctor can notify the charge nurse which allows for a quicker room turn over

In large clinics and hospitals multiple iPads may be deployed to administrators and charge nurses and of course, a new iPad may be running side-by-side an older iPad during repairs. The Patient is in iPad app will securely synchronize its patient assignments to all other Patient is in iPad apps via iCloud and thereby provide a realtime global view of:

Doctor availability
The status of each doctor assignment
The status of rooms
Location of off-site doctors with live map and traffic view
Location of on-site doctors using iBeacon micro-location technology

In this screenshot, we can see that the doctor is onsite and near Nurse Station 6. The doctor’s onsite location is provided by the doctor’s iPhone proximity to a charge nurse’s iPad all of which use iBeacon micro-location technology to determine proximity. A doctor may enable or disable this feature from the Patient is in iPhone app.

Group conference calls with Skype for Business

The Patient is in iPhone app is integrated with Skype for Business which is Microsoft’s HIPAA compliant audio and video conferencing solution for groups. Unlike Microsoft’s consumer version named Skype, Skype for Business enables medical staff to record their conference calls and comply with the various HIPAA requirements regarding conference calls. While hospitals may choose to install their own Skype for Business servers on their own premises, smaller organization can choose the Microsoft Azure cloud-based solution which costs a few dollars a month per staff member.

Much like how the charge nurse sent a patient assignment with Patient is in iPad app, she can also send a conference call request to the doctor and the doctor can join the conference call directly from the Patient is in iPhone app as seen here:

iphone-sfb-notification-bezel

and here is a clip showing the integration between the Patient is in and the Skype for Business iPhone apps:

May 23, 2017February 6, 2018 patientisin Technology

Et tu Apple Watch: Integrating Siri with an AI engine to reduce patient wait time

The Patient is in apps integrate with Siri to provide a voice interface for messaging between the doctor and the charge nurse. This includes listening to patient assignments, rejecting an assignment, accepting an assignment and providing an estimated time of arrival, and lastly notifying the charge nurse of assignment completion which allows the staff to start the cleaning process to reduce the wait time for the next patient.

On both the iPhone and Apple Watch, this enables the doctor to use AirPods and a few other Bluetooth headsets to remotely manage patient assignments exclusively with her voice. For shorter distances, the “Hey Siri” voice trigger also works well and that is the exclusive technique for hands-free voice control on Apple Watch as of watchOS 3.2. Click on this link to learn about Siri support in the Patient is in.

In this article, we will look at the underlying natural language processing (NLP) engine built for Siri integration with the Patient is in messaging features.

Natural Language Processing Theory

NLP, is one of the core AI speech technologies along with text-to-speech (TTS) and speech-to-text which is also known as speech recognition. Advances in text-to-speech has led to more natural sounding computer voices and advances in speech recognition has led to better audio transcription.

NLP attempts to understand the meaning or intent of a sentence and to do that, an NLP engine must first be able to determine in which language is the sentence. Next, the part of speech of the words used in the sentence, as well as other components of a language such as word stems and contractions must be identified. For example, the doctor’s sentence: “I’ll go to the Exam Room” would be decomposed or parsed into the pronoun “I”, the verb “will”, the verb “go”, the preposition “to”, the determiner “the”, the adjective “front”, and the noun “office”. So even though the doctor spoke the word “I’ll” the NLP engine had to understand the concept of American English language contractions and process the two words “I” and “will”. When the doctor annunciates correctly, then everything works well as we see in these screen shots:

watch-siri-articulated-assignment-2-case

watch-siri-articulated-assignment-1-case

Siri in Healthcare: Combinators and the Patient is in

Beyond contractions, advanced natural language processing algorithms must also consider the accent with which a user speaks. From the NLP engine’s point of view, everyone has an accent whether its the slow-taking southerner’s accent, the neutral mid-atlantic accent, the fast-taking New Yorker’s accent, a Bostonian accent, or perhaps the accent of a partial deaf therapist. And of course, when we are tired, we all tend to mumble a bit making us sound inarticulate. This also happens to surgeons and anesthesiologist after a middle-of-the-night emergency surgery. So, it should not be surprising that the intended word “I’ll” can be misspoken just enough to be transcribed as the word “all” as we see in the following screen shot:

watch-siri-all-go-1-case

The NLP engine used in the Patient is in apps on both Apple Watch and iPhone compensates for these common transcription errors because the algorithms were tailored for a doctor’s use of Siri.

Other transcription errors can seem insurmountable to deduce the doctor’s intent. For example, can you fix the phrase “go to low or one”? The app’s NLP engine was able to correctly determine that the doctor intended “I’ll go to OR 1” as we see in these screen shots:

watch-siri-transcription-lowORone-1-case

You should also notice that the NLP engine also understands American Language homophones as in “8 and eight and ate”, “4 and for and four”, et tu “2 and to and too and two”.

If we reexamine the classic NLP pipeline advocated in AI research as mentioned at the start of this article, we learned that it started with first identifying the context language and then parses the text into parts of speech. When we applied this to the doctor’s message we saw that the phrase “Exam Room” was decomposed and identified as the adjective “front” and the noun “office”. While linguistically correct, that does not help us because the app has to notify the charge nurse that the doctor has accepted the assignment to go to the specific room named Exam Room to treat a patient or perform surgery.

Unlike the Siri messaging support in apps such WhatsApp or Messages which simply relay the transcription provided by Siri’s speech recognition, the Patient is in must determine which room is the doctor discussing, if the doctor is rejecting an assignment, completing an assignment, or accepting an assignment and providing an estimated time of arrival which may be provided in hours or minutes.

The NLP engine uses multiple algorithms and text processing techniques to best ensure that the doctor’s intent is correctly captured even if the doctor’s words were incorrectly transcribed. The approach to use multiple algorithms and extraction techniques is relatively new and is called parser combinators. The NLP engine used in the Patient is in apps on both the iPhone and Apple Watch adds many Siri-specific and doctor-specific algorithms to the classic NLP approach.

Due to the uncertainty in processing speech, a new user interface idiom has evolved. Called the conversational user interface, it enables Siri to mediate a conversation between the user and the app and more abstractly between the doctor and the charge nurse.

Introducing Conversational Interfaces

Like in real life, rarely is information unambiguously clear. If you ask a taxi driver to drive you from the John Wayne Airport in Santa Ana, California to your office “on Main and MacArthur in the next town over” which is a few blocks away in the city of Irvine, he may instead drive you a few miles further to Main and MacArthur in the city of Costa Mesa as both cities are adjacent to Santa Ana and not only have the same street names but are actually the exact same streets which intersect in two different cities a few miles apart.

With voice interfaces, the app has to support a conversation to clarify the user’s words by asking for more information from the user either because the user has not provided enough information or has provided ambiguous information. The app must also confirm its understanding before taking action on behalf of the user.

Siri mediates this conversation between the user and the app and handles the speech recognition and passes the text transcript to the app which processes that with natural language algorithms and other text processing technologies. If the app needs additional information, it asks Siri to prompt the user by providing Siri with text to read to the user. Siri uses text-to-speech to ask those questions. After the app is satisfied that it understands the user’s request, the app asks Siri to ask the user to confirm or reject those assumptions and with the user’s permission, the app finally processes the user’s request.

What’s next in Conversational Interfaces?

Voice technologies and conversational interfaces empower app developers to use AI to provide alternative ways for users to access their app. These technologies, also extend access to a larger set of users. Through a voice interface, perhaps app developers will finally realize that accessibility and usability are fundamentally intertwined or as Tim Cook recently said in an interview marking Global Accessibility Awareness Day 2017, accessibility is a human right.

May 18, 2017February 6, 2018 patientisin Technology

Why Build an App for Blind Therapists?

Today is Global Accessibility Awareness Day #GAAD #gbla11yday and I am pleased to announce that my app the Patient is in provides the best accessibility features in a real time, HIPAA-compliant messaging app for the Apple Watch and Siri. Built over 15 months with leading doctors, therapists, and practitioners to support the needs of blind therapists, the Patient is in made its public debut at last week’s Digital Health Demo Day.

Many people ask me why did I first implement accessibility support in this ground breaking real time, HIPAA-compliant messaging app rather than add a localized version of the app for French, German, or Spanish speaking doctors and nurses. They correctly note that there is a larger market for real time medical messaging for French, German, and Spanish speaking doctors and nurses than there is for English speaking blind therapists and their nurses. But the fact is that most app developers struggle with just creating a useful and usable app that they just don’t care about such a small group of potential users. Unfortunately, the imagined profits from the many outweigh the needs of the few.

Here’s a secret that only a handful of app developers know about and the real secret why Apple has invested billions of dollars in accessibility to support the needs of a small group of its users: if you design accessible products both your underlying technology and product will be more usable by the majority of your users.

This does not discount the obvious compassion which has driven Apple engineers and managers to support all types of users with both accessible features as well as foreign language support since the 1980s. As Apple watcher Rene Ritchie from iMore.com often says: “Apple engineers create products which their families use so they try to build the best and usable products”. This is why Apple products have soul and many products from Google and Microsoft lack soul. How many CEOs have you seen brought to near tears when discussing the empowerment of blind users of their products? Watch the end of the then new CEO Tim Cook’s first WWDC keynote in 2012 and you will witness a soulful man.

If you are a press person, query your contacts to see how many of Apple’s top human factors/user experience engineers worked on the vibrations and sounds used on the Apple Watch. And their requirements were supported by great hardware engineers who worked on the Taptic engine. By the way, notification of a new or urgent patient assignment through “discrete vibrations felt on the doctor’s wrist” continues to be the most useful app feature for most doctors demoing the Patient is in and that feature was originally built to support notifying a blind therapist that he had a new patient assignment.

Beyond the engineering benefits in building accessible apps, here is another reason why we should think about blind therapists on Global Accessibility Awareness Day 2017: blind therapists support our blind children as they start learning about the world and assist them as they transition into adulthood. Blind therapists also help our parents during the transition brought on by age-related macular degeneration.

The Patient is in accessibility features

The Patient is in provides access to patient assignments and updates in 4 ways, all of which have been designed to support low-vision and blind doctors and therapists:

The watch app supports VoiceOver and provides a great user experience.
Patient assignments from the charge nurse can be automatically announced over AirPlay compatible speakers such as Apple’s HomePod. This removes the step in which the user must first access the iPhone notification to find out about the assignment details
Conversations with the charge nurse using Siri on the iPhone: “Hey Siri, read my Patient messages” allows the therapist to hear her patient assignments and “Hey Siri, send a Patient message saying I’ll go to Therapy Room 2 in 10 minutes” allows the therapist to respond to assignments from either her Apple Watch or iPhone with an estimated time of arrival. Notice of assignment completion is supported from both the Apple Watch and iPhone with “Hey Siri, send a Patient message saying I’ve completed my assignment in Recovery Room 2”. All of these statements are processed by the Patient is in natural language processing (NLP) engine fixing homophones (“Exam Room 2″ vs “exam room to” vs “exam room too” vs “exam room two”) and other linguistic and transcription impediments to create a structured message upon which the charge nurse’s iPad app can visually display and drive real world processes such as cleaning the room so that the next patient’s wait time is significantly reduced.
FaceTime support is integrated directly into all of the apps allowing the charge nurse, therapist, and other doctors to instantly connect. This includes initiating a FaceTime audio call directly from the watch app.

… just my bit to inspire other app developers for Global Accessibility Awareness Day 2018

May 17, 2017February 6, 2018 patientisin Product

Accessibility and Blind Therapists

Support for low-vision and blind therapists

The Patient is in provides access to patient assignments and updates in 4 ways, all of which have been designed to support low-vision and blind doctors and therapists:

The watch app supports VoiceOver and provides a great user experience. Click here for more information about the app for Apple Watch.
Patient assignments from the charge nurse can be automatically announced over AirPlay compatible speakers such as Apple’s HomePod. This removes the step in which the user must first access the iPhone notification to find out about the assignment details
Conversations with the charge nurse using Siri on the iPhone: “Hey Siri, read my Patient messages” allows the therapist to hear her patient assignments and “Hey Siri, send a Patient message saying I’ll go to Therapy Room 2 in 10 minutes” allows the therapist to respond to assignments from either her Apple Watch or iPhone with estimated time of arrival and notice of assignment completion. Click here for more information about Siri integration with the app.
FaceTime support is integrated directly into all of the apps allowing the charge nurse, therapist, and other doctors to instantly connect. This includes initiating a FaceTime audio call directly from the watch app.

An important note about the important role of blind therapists in society of which many of us are unaware: they support our blind children as they start learning about the world and assist them as they transition into adulthood. Blind therapists also help our parents during the transition of age-related macular degeneration.

May 16, 2017February 8, 2018 patientisin Product

An iPhone App for Doctors

The Patient is in iPhone app notifies the doctor when a patient is ready to be treated allowing her to provide an estimated time of arrival to see the patient. When the doctor has completed her assignment, she can notify the charge nurse that she has completed the assignment with a tap of a button or with Siri. The doctor may also decline the assignment if she is unable to attend.

Notifications and Alerts

The charge nurse uses the iPad app to send a patient assignment to the doctor consisting of two pieces of information:

The name of the room in which the patient is waiting
Important notes about the patient such as if he has elevated vital signs, or perhaps that the patient is ready for discharge

and the iPhone app will alert the doctor with

A popup notification will be displayed on the iPhone as seen in these screen shots:
If the doctor is not wearing her Apple Watch, she can quickly respond to the patient assignment directly from the iPhone notification providing her estimated time of arrival

If the charge nurse has assigned the doctor to a Skype for Business conference call, she can start the call directly from the notification

And the iPhone app provides a dashboard displaying the doctor’s assignments and because many doctors do not have easy access to tech support staff, the iPhone app displays diagnostic information to solve common configuration problems such as the iPhone’s WiFi connection to the real time peer-to-peer network; iCloud status for redundant and offsite message delivery; Apple Watch connection status if available; iOS app permissions for notifications, location, and Siri; and the status of Guided Access which allows the Doctor to lock the app into the foreground and prevents access to other apps or manually turning off the iPhone. This feature allows the app to run at peak performance to ensure that the patient assignments from the charge nurse and doctor responses are delivered near instantaneously.

Doctors may respond to assignments from the dashboard as seen in this video of joining a Skype for Business conference call directly from the app dashboard:

and more generally in the following screen shots:

iphone-dashboard-assignments-1-bezel — The dashboard displays patient assignments and their status

iphone-dashboard-assignments-facetime-slide-1-bezel — If the doctor needs clarification about the assignment, she may place a FaceTime call directly from the dashboard to the charge nurse who sent the assignment. The app manages the FaceTime information providing flexibility and privacy. A business address rather than a personal FaceTime address may be used which is best for shift-based, on-call doctors.

The iPhone app provides a list of today’s doctor and allows instant FaceTime calls to be placed.

iphone-2dayStaff-facetime-1-bezel — Today’s staff is listed and their availability to receive FaceTime calls. Tap to instantly connect via FaceTime.

Siri support and a voice interface

The Patient is in apps for the iPhone and Apple Watch provide a voice interface through its integration with Siri. When used with AirPods, the doctor can have a truly wireless experience. In the following screen shot, the doctor used Siri to notify the charge nurse that she has completed her assignment in Exam Room 2:

iphone-siri-done-3-bezel

Click here to learn more about the Patient is in Siri support on both the iPhone and Apple Watch.

May 16, 2017February 12, 2018 patientisin Product

An Apple Watch App for Doctors

The Patient is in watch app notifies the doctor when a patient is ready to be treated with a discrete vibration felt on her wrist and allows her to provide an estimated time of arrival to see the patient. When the doctor has completed her assignment, she can notify the charge nurse that she has completed the assignment with a tap of a button or with Siri. The doctor may also decline the assignment if she is unable to attend.

Apple Watch Series 3 with Cellular: Patient assignments on the go

The Patient is in watch app supports the cellular capabilities of the Apple Watch Series 3 ensuring that the doctor can receive and respond to patient assignments:

watch-cellular-recoveryRoom2-complication-1-case

Notifications and Alerts

The charge nurse uses the iPad app to send a patient assignment to the doctor consisting of

The name of the room in which the patient is waiting
Important notes about the patient such as if he has elevated vital signs

and the watch app will alert the doctor with

A popup notification
A discrete vibration felt on her wrist
An optional, audible sound

Glanceable Information

Since the doctor may receive a new notification while she is treating another patient, the Patient is in watch app uses haptic feedback in the form of a silent vibration to alert her that a new notification was received from the charge nurse. The Patient is in watch app will display the new room name and abbreviated notes directly on the watch face affording the doctor a quick glance without disrupting her current patient’s treatment.

The following screen shots show how the app supports complications to implement glanceable information:

watch-complication-modular-assignment-1-case

watch-complication-utility-assignment-1-case

Actionable Information

When the doctor is able to respond to the assignment, the Patient is in watch app allows the following:

Review any patient notes provided by the charge nurse by tapping on the name of the room
Inform the charge nurse of the estimated time of arrival
Decline the assignment
Inform the charge nurse that the assignment is complete

The following screen shots show how the doctor uses the Patient is in watch app. First, a discrete vibration is felt on her wrist, optionally with a sound. A popup alert is also displayed and allows the doctor to quickly respond directly from the notification:

watch-notification-recoveryRoom2-assignment-1-case

If the doctor is busy and misses the notification, she can access it by swiping down from the watch face. This will display all of her missed notifications as seen in this screen shot:

watch-notification-recoveryRoom2-assignment-3-case

With a tap on the notification, the Patient is in watch app is launched and visually displays the assignment allowing the doctor to quickly accept the assignment and provide an estimated time of arrival or decline the assignment. A timer displays how long has it been since the charge nurse sent the assignment. The doctor may tap the room name to review the patient notes and start an FaceTime audio call with the charge nurse if she needs clarification. Finally, when the doctor has completed the assignment, she can notify the charge nurse that she is done with a tap of a button as seen in the following screen shots:

From the main watch app, the doctor can reject the assignment or accept it and provide an estimated time of arrival

When the assignment is complete, the doctor can notify the charge nurse which allows for a quicker room turn over

If the doctor prefers, she can use Siri to perform theses tasks.

Siri support on Apple Watch

Using Siri, doctors can converse with the charge nurse to respond to patient assignments using the following phrases:

“Hey Siri, send a Patient message saying I’ll go to Exam Room 1 in 15 minutes”
“Hey Siri, send a Patient message saying I’ve completed my assignment in Recovery Room 2”
“Hey Siri, send a Patient message saying I cannot go to Operating Room 3”

And the following screen shots show the doctor using Siri to acknowledge a patient assignment and then notify the charge nurse that she has completed the assignment:

watch-siri-both-3-case

Click her for more information about creating a voice interface with Siri in the Patient is in.

Click here for more information about integrating Siri with an AI engine to reduce patient wait times in the Patient is in.

Audio and Video Conference Calls

Enhancing the convenience of mobile conference calls with advanced presence and geolocation rules, Apple’s FaceTime is deeply integrated throughout the app allowing doctors to initiate FaceTime calls directly from the watch app. Work-only FaceTime addresses may be defined in the iOS app to protect the privacy of personal FaceTime addresses and to support on-call doctors and other shift-based staff members.

To start an audio FaceTime call with another doctor, nurse, or charge nurse who is working today, the doctor use a Force Touch gesture on list of patient assignments which pops up a menu and then taps on the “Today’s Users” button. This is seen in the following screen shots:

The doctor may also initiate a FaceTime audio call to the charge nurse who sent the patient assignment when clarification about the patient assignment is needed. By tapping on the patient assignment room name, the doctor has access to the patient notes and the option to begin a FaceTime audio call as seen in this screen shot:

watch-notes-facetime-available-case

Support for low-vision and blind therapists

The Patient is in provides access to patient assignments and updates, all of which have been designed to support low-vision and blind therapists. For example, VoiceOver is comprehensively supported as well as Siri for voice access and FaceTime for audio conferencing with both the charge nurse as well as other available staff members as seen in following screen shots:

watch-4-voiceover-screens-1

Click here to learn more about the accessibility features of the Patient is in and why the app supports blind therapists.

A video of the Patient is in app for Apple Watch

The following video shows the Patient is in in action on Apple Watch:

May 16, 2017May 31, 2017 patientisin Product

Siri integration for a voice interface

The Patient is in apps are integrated with Siri on both the iPhone and Apple Watch to provide a voice interface for messaging.

Conversations with the charge nurse using Siri on the iPhone: “Hey Siri, read my Patient messages” allows the doctor to hear her patient assignments and “Hey Siri, send a Patient message saying I’ll go to Therapy Room 2 in 10 minutes” allows the doctor to respond to assignments from either her Apple Watch or iPhone with an estimated time of arrival. Notice of assignment completion is supported from both the Apple Watch and iPhone with “Hey Siri, send a Patient message saying I’ve completed my assignment in Recovery Room 2”.

All of these statements are processed by the Patient is in natural language processing (NLP) engine fixing homophones (“Exam Room 2″ vs “exam room to” vs “exam room too” vs “exam room two”) and other linguistic and transcription impediments to create a structured message upon which the charge nurse’s iPad app can visually display and drive real world processes such as cleaning the room so that the next patient’s wait time is significantly reduced.

For a truly hands-free experience on the iPhone, the doctor needs only to use AirPods or another bluetooth headset and use the phrase “Read my Patient messages” after activating Siri either with a double tap on either AirPod or with the “Hey Siri” trigger phrase.

And then the doctor may respond to the assignment using the phrases: “Hey Siri, send a Patient message…” as seen in these Apple Watch screen shot:

watch-siri-both-3-case

and on the iPhone as seen in this screen shot we see the natural language processing (NLP) engine of the Patient is in fixing homophones (“Exam Room 2″ was transcribed by Siri as “exam room to” ) and other linguistic and transcription impediments to understand the doctor’s intent:

iphone-siri-done-3-bezel

To ensure the best experience, the internal NLP engine used in the Patient is in apps has been optimized for Siri in a medical environment.

The following are some sample phrases which the doctor may use with Siri on both the iPhone and Apple Watch:

Send a Patient message saying I’ll go to Exam Room 1 in 15 minutes
Send a Patient message saying I’ve completed my assignment in Recovery Room 2
Send a Patient message saying I cannot go to Exam Room 2

Exclusive to the iPhone, the doctor may ask Siri the following:

Read my Patient messages
What are my messages on Patient?

Understanding the AI integration with Siri

Click below to learn more about the technology powering the integration with Siri:

Integrating Siri with an AI engine to reduce patient wait times

A video of using Siri with the Patient is in app

In the following iPhone video, the doctor uses Siri to listen to her patient assignments:

May 15, 2017February 12, 2018 patientisin Product

An iPad App for the Charge Nurse

Doctor assignments, room status, and doctor location

With the Patient Is In iPad app you can notify the doctor on her Apple Watch or iPhone that a patient is ready to be treated. Also, the Patient is in iPad app allows the charge nurse to quickly:

View doctor availability
Visualize the status of each doctor assignment
Visualize the status of rooms
Location of off-site doctors with live map
Location of on-site doctors using iBeacon technology
Start an audio or video FaceTime call with the doctor (without knowing the doctor’s FaceTime information)
Receive an audio or video FaceTime call from the doctor (without the doctor knowing your FaceTime information)
Schedule a Skype for Business conference call
Announce a patient assignment on any AirPlay-compatible speaker such as Apple’s HomePod wireless speaker
Broadcast a general announcement to any AirPlay-compatible speaker such as Apple’s HomePod wireless speaker
Synchronize doctor assignments and room status with other charge nurses and administrators

Sending an assignment to the doctor

You send a notification to the doctor’s iPhone or Apple Watch containing

The name of the room in which the patient is waiting
Important notes about the patient such as if he has elevated vital signs

and the Patient Is In watch app will alert the doctor with

A discrete vibration felt on the doctor’s wrist
An optional, audible sound

If an AirPlay compatible speaker such as Apple’s HomePod wireless speaker is set up in a doctor’s office and authorized to be used by the charge nurse’s iPad, then the doctor can hear a wireless announcement similar to “The charge nurse says that Dr. Quinn has an assignment in Exam Room 2”. Here is a link to learn more about using multi-user Siri support and HomePod: HomePod and Multi-User Siri in Health Care: It’s how to reduce patient wait times

Additionally, this assignment is synchronized with the Patient is in iPad app managed by other charge nurses and administrators. This provides large clinics and hospitals with a global, real-time view of doctor availability, patient assignments, and room status.

In this video, we see how the charge nurse is able to quickly send a patient assignment to the doctor, visualizing doctor availability and see which doctor is onsite and connected to the real time peer-to-peer network or through iCloud, if the doctor is receiving patient assignments on her Apple Watch or iPhone, if the doctor is available for a FaceTime conference call, the status of assignments (accepted with an estimated time of arrival, decline, or completed), and the status of each room.

Locating an onsite doctor with iBeacon technology

In large hospitals, it is sometimes critical to locate a doctor who may be busy attending to another patient. The Patient is in app allows you to locate an onsite doctor. First tap the icon representing the doctor and then tap the magnifying glass icon as seen in the following video:

You will then be presented with a map view displaying the general location of the doctor as seen here:

The Patient is in app implements Apple’s iBeacon technology to automatically register each time a doctor’s iPhone passes near a charge nurse’s iPad running the app. In the above screen shot, a map pin shows that at 9:21 PM, the doctor was “Near Nurse Station 6”. And likewise, a few minutes after the doctor leaves the area of the iPad named “Nurse Station 6”, the map pin will be updated to show that the doctor has “Left Nurse Station 6”.

iBeacon uses Bluetooth, so for this feature to work both the iPad and the iPhone must enable Bluetooth.

If you are a fan of TV science fiction and specifically Star Trek, you may recognize this feature as a real world example of the Enterprise computer knowing the location of a crew member by passively tracking the location of the communicator badge as seen in following snippet from the first episode of Star Trek: The Next Generation

iBeacon technology and your privacy

It is important to note that the Patient is in iPhone app is only able to report on the last time the iPhone was in the Bluetooth proximity of a charge nurse’s iPad which was at the time both running the app and had Bluetooth enabled. The Patient is in iPhone app is unable to determine its proximity to other iPhones.

Locating an offsite doctor

If the doctor allows, the charge nurse may see the doctor’s offsite location on a live map. This map will also display current traffic conditions and is similar to apps such as Uber, Lyft, and Glympse. This feature helps the charge nurse estimate the doctor’s time of arrival for emergency surgery and other time sensitive activities.

Offsite location data and your privacy

It is important to note that only the doctor can decide if her offsite location should be disclosed to the charge nurse. The doctor can at anytime enable or disable the setting “Update charge nurse” in the section “OFFSITE AND ONSITE LOCATION UPDATES” as seen in the following screen shot:

iphone-dashboard-permissions-1-bezel

If this setting is enabled and due to the imprecise nature of mobile location technologies, the app may update the charge nurse with the doctor’s location up to 1 mile or more from the location of the charge nurse’s iPad. Consequently, if the doctor is using her personal iPhone and not a hospital issued device, the doctor may choose to manually disable this feature when leaving work.

This setting is disabled by default.