Augmented Reality AR is a relatively new research branch of
Augmented Reality (AR) is a relatively new research branch of virtual reality (VR) technology. The ultimate aim of this promising technology is to connect particulars, add value to a real object and enhance the user’s experience. The biggest difference between an augmented reality and virtual reality is augmented reality does not create a simulation of reality. Virtual reality technology can create a virtual world, while augmented reality superimposes computer-generated virtual objects, figures or information to the real world environment. The computer generated information can be both real and virtual objects.
In order to enhance the experience, audio narration, location data, historical background are added to the augmented reality system. From the user’s perspective, AR has linked the virtual reality to the real world. A correct three dimensional (3D) placement and orientation of the virtual data is required to superimpose the virtual data to real environment accurately. Meanwhile, an AR system also requires to achieve 3I characteristics: Immersion, Imagination and Interaction. The crucial concerns are real time registration and render problem.
This involves the precision of the object to be rendered which determines the immersion effect of the system directly. Therefore, real time and accuracy improvements are the most essential factors. Augmented reality can be determined in two main groups: an optical see-through systems and a video see through AR which based on the mix of live stream with graphical scene elements. AR technology has been used in many medical based applications and it has been developed rapidly to improve medical diagnosis and treatment.
For example, computer assisted medical intervention (CAMI), Microsoft HoloLens/ Google Glass, Accuvein, treatment for phobia.
Computer-aided surgery (CAS)/ CAMI is a modern medical imaging technology which has significantly improved and simplified the complexity of diagnosis and intervention treatment. The key role of a CAS system is to enhance the physical world by laying out the pre-operative information. For example, AR can be used as a collaborative tool in an intra-operative planning of a liver resection. This involved two augmented reality techniques which are a see- through head-mounted display (HMD) and a video based AR for the radiologist. Surgeon is provided with a headset combined with computer interface that portray data on the patient. This allows them to have a comprehensive visualization of the intrahepatic vascular structure of the liver. Employment the AR system in the surgery greatly shorten the operation time, assists the surgeon to evaluate different resection planes, and introduces exact resection margins.
Visual inspection is the stage where the cross sectional images are evaluated for visual exploration. A surface reconstruction technology established on the foundation of an algorithm related to the treading cubes algorithm is applied to convert initial segmentation to a surface illustration. The surface representation is then converted to a deformable simplex mesh. A radiologist would be able to view the reconstructed structure from different viewpoints and distances by moving it using tracked input devices. Moreover, the topological relations can be clarified by modifying the transparency of the object. The utilization of the tracked input devices provides an intuitive and natural way for visual evaluation of the segmentation data. On the other hand, the visual inspection is followed by refinement of segmentation. Various tools are developed to interact and edit the 3D surface representation from the generic, mesh based methods or a higher level shape information. These tools support the radiologists to alter any deformations or flaws in segmentation in an uncomplicated way without spending lots of time on it. Several hardware is setup in a medical AR system, including camera, tracked input device, stereoscopic HMD, and rendering workstation.
An ARAS is a combined effort between a surgeon and an off-site radiologist. The preoperative computer-generated data proposed in a surgery are achieved by magnetic resonance imaging (MRI), computerized tomography scan (CT Scan), or X ray. These techniques have allowed the medical professionals to identify the anatomical structures of liver, including the vasculature, liver segments or tumour. The image data obtained from 3D scanning devices is stored according to DICOM standard , and generated into the 3D model. The main challenges for a radiologist are producing of a standard 3D data based on segmented image sets, identifying the target tissues, and measurements. The whole process of sorting and producing the data is time consuming and requires meticulousness. Hence, a developed AR system is able to facilitate the surgery planning stage. AR systems also provides an intuitive way of interaction with 3D objects along with the usage of HMD. Furthermore, radiologist and surgeons can take advantage of the AR system to tailor and inspect the 3D segmentations to the individual needs. The cross sectional images obtained are then processed in various stages. For example, visual inspection, interactive segmentation refinement , and resection planning
Traditionally, CT scan and MRI are used to scan images in order to figure out the details in a targeted object. However, radiology techniques could be harmful to a human body by releasing a massive amount of radiation. Therefore, scientists have revised the current techniques and aimed to improve the quality of images. A complex 3D models are proposed by embracing the use of Stereo X-Ray that uses synchrotron radiation. This technology could reconstruct and visualize blood vessels and display it through augmented reality with the usage of Microsoft HoloLens. Despite that stereo projections cannot provide a complete 3D model, it can be detected to achieve real time imaging and details of dynamical phenomena in a time scale of milliseconds. A stereo imaging experiment on a living frog and six twisted metal wires were conducted by Masato et al to exhibit the 3D analysis and derivation of depth of the object. It is shown that stereo X-ray imaging works satisfactorily in vascular surgery that demands pre-operative strategy, such as aneurysms.
Google Glass is a smart glass developed by Google X with the purpose to display information in hands-free format. Hardware devices used for AR have built in camera, processor, GPS, sensors and/or a compass. Software developers had developed an app for this head mounted device which allows live- stream during patient visit. The aim is to eliminate electronic medical record (EMR), a collection of records regarding patients’ health information in digital form. These records can be accessed across several healthcare settings. The greatest benefits gain from Google Glass instead of EMR are time-saving and better record quality. On the other hand, Google Glass has been used in lactation consultation in Melbourne, Australia. The Australian Breastfeeding Association has developed a Google Glass Breastfeeding app trial in order to help new mothers in breastfeeding. Mothers are able to breastfeed their infants by instructions or call a lactation consultant through the built in audio feature. This app and implementation of Google Glass based on AR system has enhanced a new mother’s experience and boosted the quality of consultation. The advantages of these hardware devices are non-handheld, and the display remains in front of the users ‘eyes.
Accuvein is a near- infrared vein finder used to portray a map of peripheral veins on the skin surface aiming to improve , and other vascular access procedures. Accuvein as a handheld device and non-invasive is a proven visualization technology. Nurses or medical professionals may face difficulties in establishing intravenous (IV) access, if patients are elderly, obese or dark skin tone. IV access can be a skill that requires practices, and Accuvein supports medical staff by locating and facilitate a safe and efficient IV access. By using Accuvein, it is shown that patient satisfaction and patient care are highly improved.
AR has a huge potential to deliver contextual and power learning experiences to medical students. It is shown to be reliable, useful, and applicable to healthcare education. Anatomy is an essential subject and has to be mastered by medical students; however, it is a tedious and complex subject. Medical studies involves the understanding of physiological, and possessing skills in multidisciplinary practice. AR system could support the students in exploration of the complex vascular, nerves, or muscles interconnections and broaden their understanding in anatomy based the interactions with virtual information. Medical students are able to study the internal view of a human body without the need of cadaver lab. The purpose of a cadaver lab is to elaborate knowledge obtained from lectures and textbooks, and have an overall view of an organism. Moreover, students or a surgeon could do multiple training on their skills through a simulator to avoid errors in a real procedure, the patient’s safety is guaranteed. Laparoscopy skills is required in a novel surgery approach- minimally-invasive surgery (MIS). MIS could eliminate disadvantages of an open surgery and shorten the recovery time from a procedure. In addition, MIS is more skill demanding for a surgeon as it requires more concentration and attention. Surgeon could train themselves handling laparoscopic instruments and able to overcome the peculiarity of this instruments. AR application in laparoscopy training has provided an environment to practice multiple times in execution of psychomotor skill. Researches had been conducted and shown a high acceptance rate of AR as a learning technology and medical students’ learning effect is improved. The implementation of AR in healthcare education can involve several areas and accepted by all levels of student. For example, laparoscopic surgery, forensic medicine, local anaesthesia, intubation, life support training , and other healthcare related subjects. In addition, AR learning technology has decreased the amount of time and practice needed on one subject, improved learning experience with a better insight of spatial relationships.
Unfortunately, there is no proven theories to guide the design of AR available on the market yet. Around 80% of researches did not mention the methods and describe the application of AR in healthcare education was based on which learning theories. Besides, most of the studies were still in a prototype stage and traditional learning methods were applied in practical skills education. At the present, the key role of AR in learning as a feedback tool and guiding system. More studies are needed to explore the potential of AR in medical training and promote its application in education.
Phobia indicates an enormous or illogical fear of or detestation to a specified thing or group. Suffering from phobia could leads to excessive of anxiety when an individual exposed to a trigger stimulus. An increased level of anxiety often linked to an elevated level of heartbeat and sweating. Untreated phobia could further result in more unrealistic fears, as consequence, an individual’s normal life could be affected severely and caused major social issue. Studies have shown that an exposure- based treatment could be an effective treatment to phobia. Exposure-based therapy can be designed based on various theories such as the Emotional Processing Theory, the Perceived Control and Self-Efficacy Theory, and the Bio-informational Theory. The objective of performing exposure- based therapy is to convince the patient that the consequences of fear that he afraid of do not certainly occur. Conventionally, the therapy has been carried out by putting the subject in an actual stimulus environment. However, a patient may refuse to continue the treatment after he aware of the possibility to face the threat. Therefore, advances in technology has promoted ARET, which introduces virtual elements to the user’s visual recognition. ARET has replaced the physical environment with an enhance virtual world, in the form of mixed reality (MR). In fact, ARET has more advantages over virtual reality exposure-based therapy (VRET). ARET requires less virtual element to be designed and the cost to create a virtual environment is less compared to VRET. User of ARET able to do self-training, and has control over the scenario and stimuli. Furthermore, superimposition of virtual element in real environment allows the patient to have more interactions of “own body” with the virtual stimuli. In addition, studies have shown effective results in the treatment of entomophobia (phobia to insects). For example, a study in the treatment phobia to cockroaches has been directed and it was shown that the virtual cockroaches successfully initiated anxiety of the subjects and was able to lessen the panic after a considerable duration of exposure. Besides, the subject was capable to have interaction with the cockroaches in ARET. For instance, killing the living virtual cockroaches. On top of that, Wrzesien and colleagues has directed a the study of phobia to small animal with the application of therapeutic lamp (TL) which is a projection-based AR system. TL does not requires a HMD, and the study has proven that TL can be useful in ARET. Patients show more confidence and belief they could face a cockroach after one session of ARET.
The development of technology has brought us a more convenient way of living. Technology in AR has been demonstrated to improve and modify traditional ways of treatment in medical field. Efficacy of treatment has been increased significantly and have more positive feedbacks from the patients. In spite of blooming technology, guidelines should be abided by the public and inventor in order to ensure that confidentiality of patients are protected. Drawbacks in development of AR technology should be considered and overcome it. First of all, the cost to develop and maintain an AR technology is considerably highly-priced. Developers could associate with the government unit to have more funds in researching and to provide opportunities in accessing the related AR treatment to public. Lastly, the HMD or Google Glass has short battery life, the mobility and weight of the device could bring unpleasant experience. Therefore, future development of this technology needs to be cost-effective, handheld free, and lightweight to be utilized in medical field.
