Augmented Reality for Healthcare

Vidya Murali and Stan Birchfield

Augmented reality (AR) is a rapidly developing field of computer research that superimposes virtual reality onto the real world. Augmented reality software can be used to project 3D data onto a display so that the data appears to be located on the surface containing an identifying marker or tag. Our research deals with harnessing this power for healthcare solutions using a PDA / smart phone. The tags/markers can be used by clinicians in a hospital to pull up data (such as statistics, CXRs, CTs, etc.) on patients using their personal PDAs, with the data associated either with the outside door to the room or with the actual patient. Extended applications could involve using the PDA as a data lens for AR-based navigation support in hospitals.

Augmented Reality Hospital Videos

Below are some preliminary demo videos that depict the capability and scope of the system. These are common data acquisition scenarios in a hospital using our augmented reality system based on Studierstube (a commercially available AR toolkit) on a typical PDA (HTC Touch Pro).

Augmented Reality Hospital Corridor (2 min. 16 sec., 11.4 MB)

A clinician walks down a corridor, and in front of each patient's door data is displayed on the PDA based on the identifying marker on the door.

Augmented Reality Patient (with anatomical data overlay) (48 sec., 2.2 MB)

Patient wears a tag necklace that hangs in the middle of the chest. When approaching the patient, the PDA shows all of the patient data geospatially and anatomically arranged.

Augmented Reality Patient (with data on the wall) (22 sec., 1.0 MB)

The tag is now next to the patient on the wall. When approaching the patient, the PDA shows the patient data on the wall.

Additional Videos

Augmented Reality Hospital Room (Tag-wall) (1 min., 9 sec., 6.8 MB)	All patient data including CXRs, CT scans, angiograms, etc. are displayed together on the wall of the room, based on a single identifying tag/marker.
Augmented Reality Patient (with anatomical data overlay, top view) (34 sec., 2.1 MB)	Patient wears a tag necklace that hangs in the middle of the chest. When approaching the patient (top view), the PDA shows all of the patient data geospatially and anatomically arranged.
Augmented Reality Hospital Corridor (using the Motorola Q) (2 min. 28 sec., 14.7 MB)	A clinician walks down a corridor, this time with a Motorola Q and in front of each patient's door data is displayed on the PDA based on the identifying marker on the door. The Motorola Q is faster than the HTC Touch Pro, but has a lower screen resolution.

* Thanks to Microsoft for supporting this project.

Augmented Reality Hospital Corridor (2 min. 16 sec., 11.4 MB)	A clinician walks down a corridor, and in front of each patient's door data is displayed on the PDA based on the identifying marker on the door.
Augmented Reality Patient (with anatomical data overlay) (48 sec., 2.2 MB)	Patient wears a tag necklace that hangs in the middle of the chest. When approaching the patient, the PDA shows all of the patient data geospatially and anatomically arranged.
Augmented Reality Patient (with data on the wall) (22 sec., 1.0 MB)	The tag is now next to the patient on the wall. When approaching the patient, the PDA shows the patient data on the wall.