D Deebika

Master of Engineering, Dept of Computer Science and Engineering, MNM Jain Engineering College, F3,maruthi homes, madipakkam, Chennai. India

DOI : https://dx.doi.org/10.13005/bpj/623

Abstract

The main objective of this research article is to use Augmented Reality for medical imaging for advancement in  Images developed from Ultrasound Scans,and microscopy images   to be combined with videos. This combination of Scan Images mediated with video and audio enables the viewing of fetus moments to be incorporated as an video in the image to be watched by the medical consultant for all its sensory movements. This paper is designed with the intention to develop augmented reality to combine real time images of fetus with its actual videos showing its gestures, movements, and posture inside the uterus. This  showcases the Doctors to examine patients while viewing superimposed medical images. In addition, this augmented reality mobile applications can provide the surgeon with certain essential information, which are usually not visible such as showing heartbeat, blood pressure. This application let a doctor by looking into images such an X-ray combined with the patients photograph, disease name ,symptoms and even videos. For example ,All these helps the specialist in visualizing the position of a tumor in the video of an endoscope or radiation exposure risks from X-ray Imaging devices.

Keywords

Augmented Reality; Sonographer; Ultrasound; superimposed

Download this article as: 

Copy the following to cite this article: Deebika D. Augmented Reality Advancement X-Ray Imaging Medical Reality scanning. Biomed Pharmacol J 2015;8(1)

Copy the following to cite this URL: Deebika D. Augmented Reality Advancement X-Ray Imaging Medical Reality scanning. Biomed Pharmacol J 2015;8(1). Available from: http://biomedpharmajournal.org/?p=734

Introduction

Augmented reality has been nowadays used for more practical purposes such as medicine which is very trivial ,apart from providing entertaining digital content to the users of smart devices. One such important use of AR is in the field of Ultrasound medical imaging of fetus. This technology is going to play an important role in the future of medicine. It can be hoped that similar augmented reality products will be developed and will revolutionize the mode of treatment  used by doctors in the future.  The capabilities that this technology has, can be used to make complex surgical procedures easier. Doctors and other healthcare professionals have lauded this innovative reality and its adaptiveness for this ability. With proper platform, doctors can not only perform surgeries easily but also provide postoperative care.

With this AR technology, the ultrasound medical images of pregnant women can be mediated with videos of fetus to provide  two real time virtual videos. To explain further, this revolutionary wireless technology application in the mobile of a practitioner can be used to display the virtual reality video runs of scanning the womb of a pregnant woman with an ultrasound sensor, generating a 3-D representation of the fetus inside the womb and displaying that in a see-through.The goal is to endow the doctor with the ability to see the moving, kicking fetus lying inside the womb, with the hope that this one day may become a “3-D stethoscope” .More recent efforts have focused on a needle biopsy of a breast tumor. AR can be used to show a mockup of a breast biopsy operation, where the virtual objects identify the location of the tumor and guide the needle to its target

Augmented Reality (AR) is a variation of Virtual Environments (VE), or Virtual Reality as it is more commonly called. VE technologies completely immerse a user inside a synthetic environment. While immersed, the user cannot see the real world around him. In contrast, AR allows the user to see the real world, with virtual objects superimposed upon or composited with the real world[3]. Therefore, AR supplements reality, rather than completely replacing it .A overview of Augmented Reality and its mixed mode of reality with virtual and real world is portrayed in Fig 1.1

Need for AR technology

The “paper-less” office has proven to be a  myth: not only are office workers still inundated with paper, but they must now handle increasing quantities of electronic information. Worse, they are poorly equipped to bridge the gap between these two overlapping, but separate, worlds. Perhaps it is not surprising that computers have had little or no impact on white collar productivity. The problem is not restricted to office work. Medicine has enjoyed major advances in imaging technology and interactive diagnostic tools. Yet most hospitals still rely on paper and pencil for medical charts next to the patient’s bed. A recent study of terminally-ill patients in the U.S. concluded that, despite detailed information in their electronic medical charts, doctors were mostly unaware of their patients’ wishes. Despite its importance, the electronic information all too often remains disconnected from the physical world. Fig 1.1. shows Overview of Augmented Reality.

Historical background of Augmented Reality

According to the Horizon Report, Augmented reality is older than the term coined by Tom Caudwell in 1992.They say that the first applications actually appeared in the late 1960s and 1970s. Early mobile applications began to appear in 2008 and reality mapping and social tools are now on the market. In 2008-2009, Wikitude came out with AR Travel Guide and AR Navigation System for the Android platform Also in 2009, the AR Toolkit was ported to Adobe Flash which essentially brought AR to the web browser

Use Cases Scenarios

AR for ultrasonic Scan Images as motionary videos

Augmented Reality addresses problems of stationary images and traditional Scan pictures obtained from Ultrasound, MRI Scan by reintegrating electronic information back into the real world. The various approaches share a common goal: to enable people to take advantage of their existing skills in interacting with objects the everyday world while benefiting from the power of networked computing by superimposing scan images with from simple text to links with video and audio content.

Importance of AR for surgical enhancement

Because imaging technology is so pervasive throughout the medical field, especially for surgery it is not surprising that this domain is viewed as one of the more important for augmented reality systems. Most of the medical applications deal with Improvised image guided surgery. Pre-operative imaging studies, such as CT or MRI scans, of the patient provide the surgeon with the necessary view of the internal anatomy. From these images the surgery is planned. Visualization of the path through the anatomy to the affected area where, for example, a tumor must be removed is done by first creating a 3D model from the multiple views and slices in the preoperative study. This is most often done mentally though some systems will create 3D volume visualizations from the image study. Augmented reality can be applied so that the surgical team can see the CT or MRI data correctly registered on the patient in the operating theater while the procedure is progressing. Being able to accurately register the images at this point will enhance the performance of the surgical team and eliminate the need for the painful and cumbersome stereotactic frames that are currently used for registration.

AR for X-ray Images as Informative Images to consultant

Augmented Reality can be encumbered with usage of X-rays for medical specialists .X-Rays images, conventional tomography pictures can be mediated with patient photography ,symptoms of the predicted disease, Patient name and necessary audio tracks as shown in Figure 2.1

As per discussion Figure 2.3 shows a normal X-Ray Image against actual Implementation of AR Technology depicting Patient name, Age, Consultant Name embedded in  X-   Ray Image as shown in Figure 2.4 which is a great boon in medical advancement.

AR for 3D holistic view of body as a merger of virtual and physical reality

The goal is a system that shows the surgeon a 3D picture of a patient’s inner body along with the instruments needed during the operation. This image is not visible on a separate display but rather can be seen directly while looking at the patient. To ensure the use of this technology the system should be user friendly.

The technique is sample portrayed in figure which shows keen pinhole look for the surgeon deep inside the victim as shown in Figure 2.5 depicting realistic Overview of holistic Augmented Virtual Reality.

Importance of AR for C-Arm with enhanced cameras

One big step ahead ,yet another development from the chair of Augmented Reality is C-Arm,an enhanced camera for virtual Reality. The researches have enhanced an imaging method, the so-called C-arm, to be used in augmented reality. The technique will be tested in the operation theatre in the yet another few months to come.

Materials And Methods

Importance of AR for fetus Ultrasound scanning

With this AR technology, the ultrasound medical images of pregnant women can be mediated with videos of fetus to provide  two real time virtual videos. To explain further, this revolutionary wireless technology application in the mobile of a practitioner can be used to display the virtual reality video runs of scanning the womb of a pregnant woman with an ultrasound sensor, generating a 3-D representation of the fetus inside the womb and displaying that in a see-through . The goal is to endow the doctor with the ability to see the moving, kicking fetus lying inside the womb.

Reasons  to Use An AR embedded Ultrasound Image for fetus

Depending on your stage of pregnancy, AR Application scan video mediated with the Ultrasound image  can:
Show  that your baby has body movements, limb fluctuations and breathing gestures.

Depict the real number of babies whether you’re pregnant with one baby or twins or more.

Detect an ectopic pregnancy, where the embryo implants outside the womb, usually in the fallopian tube.

Find out size, measurements of your baby.

Assess your baby’s risk of Down’s syndrome by measuring fluid at the back of your baby’s neck at 11 weeks plus two days to 14 weeks plus one day (nuchal translucency (NT) scan).Find out why a blood screening test was abnormal.

Help with diagnostic tests, such as CVS or amniocentesis, by showing the position of the baby and placenta.

Examine you  baby to see if all his organs are normal.

Diagnose most abnormalities, misconceptions such as spinal bifida.

Assess the amount of amniotic fluid you have and find out where the placenta lies.

Measure your baby’s rate of growth over several scans. A scan may show if you are expecting a boy or girl revealing the gender of a baby

Operation of AR Technology

The basic idea of augmented reality is to superimpose graphics, audio and other sensory enhancements over a real-world environment in real time. This concept sound easier.Doctors can examine patients while viewing superimposed medical images children can program their own LEGO constructions; construction engineers can use ordinary paper engineering drawings to communicate with distant colleagues. Rather than immersing people in an artificially created virtual world, the goal is to augment objects in the physical world by enhancing them with a wealth of digital information and communication capabilities.The workflow of fetus Ultrasound Reality scanning  using AR technology as implemented in shown in Figure 3.1

Existing System

Static X-Ray images and ultrasound photos are a thing of the past. With traditional X-Ray, even though, most experts agree that scans performed by a qualified healthcare professional there will  be  a chance to misplace patient’s X-Ray images of one with another patient.
There is no evidence of any Augmented reality mobile application for viewing X-Ray images with patients photograph and symptom names even in multi speciality clinics in our country.

A conventional  ultrasound scan sends sound waves through your womb (uterus). These waves bounce off your baby as echoes which are then turned into an image on a screen that shows your baby’s position and movements.
Hard tissues, such as bone, reflect the most sound waves and so make the biggest echoes. These appear white in the image, and soft tissues appear grey. Fluids, such as the amniotic fluid surrounding your baby, appear black. This is because the sound waves go through them with no echoes.
The person performing the scan (sonographer) will look at these different shades to interpret the images.
But this can be viewed only for the current movement by the sonographer who captures the scanning video. This motionary pictures and videos are neither available for the consultant doctor nor the parents for future reference.

Proposed System

In Our Proposed System, we have used Augmented Reality (AR),a new wireless communication mobile application virtual technology which allows the combination of multiple images, audios, videos to be incorporated together in  a single scan Ultrasound Image .Such virtual reality motionary  videos of fetus inside mother’s womb provides improved medical enhancement by allowing the fetus movements weaved to be watched  later  by specialist and parents. The first scan can be very exciting because it gives you a first glimpse and live videos of your baby. Your sonographer may even print out the image of your baby embedded in the mediated motionary video of the baby and give it to you as a keepsake. Suddenly this ultrasound advancement is not just a visual offer but it actually allows you to gain parental affection by watching gestures of the baby when desired ,practitioner loyalty and even instant motionary predictions !This has all the benefits of the traditional Ultrasound imaging but it is a dramatic enhancement that offers much more.The System Architecture of placement of augmented reality in shown in Figure 4.1.This Figure 4.1 shown absolute positioning of medical reality scanning.

Challenges and Discussion

In general, one can summarize that AR is not more insecure than other related technologies. At present, the AR technology has reached a level where commercial launch preparation can begin and should be established. Reliability and usability of AR applications are probably the most important determinants of the user experience in everyday use of this augmented technology and therefore important keys to its success.

Results and Discussions

The next generation Augmented reality technology is designed to make life much easier for medical filed for patients as well as medical consultants .Figure 4.1 clearly portrays how developed application using augmented reality shows all the details of patients including name, diagnosis, age and medical  record. For trend setters everywhere, AR technology is, without doubt, the next big thing[3]..Practitioners don’t need to install any apps or commit to any system to engage with AR. It’s all built in as an application. Touch your phone to something, consume content. With these kind of medical advancement the specialists would say in future “I sometimes forget my keys, my ID, my watch, my wallet… but I NEVER forget my phone which has AR application”

References

Journals

1. T. P. Caudell and D. W. Mizell, “Augmented reality: An application of heads-up display technology to manual manufacturing processes,” in System Sciences, 1992. Proceedings of the Twenty-Fifth Hawaii International Conference on, vol. 2. IEEE, 1992, pp. 659–669
2. H.-Y. Chang, H.-K. Wu, and Y.-S. Hs, “Integrating a mobile augmented reality activity to contextualize student learning of a socioscientific issue,” British Journal of Educational Technology, vol. 44,2014.
3. M. de Sa and E. Churchill, “Mobile augmented reality: explor- ´ ing design and prototyping techniques,” in Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services. ACM, 2012, pp. 221–230.
4. S. M. Land and H. T. Zimmerman, “Synthesizing perspectives on augmented reality and mobile learning,” TechTrends, vol. 58, no. 1, p. 3, 2014.
5. D. Van Krevelen and R. Poelman, “A survey of augmented reality technologies, applications and limitations,” International Journal of Virtual Reality, vol. 9, no. 2, p. 1, 2010

Chapter In Books

1. R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, and B. MacIntyre, “Recent advances in augmented reality,” Computer Graphics and Applications, IEEE, vol. 21, no. 6, pp. 34– 47, 2001.
2. R. T. Azuma et al., “A survey of augmented reality,” Presence, vol. 6, no. 4, pp. 355–385, 1997.
3. O. Bimber, R. Raskar, and M. Inami, Spatial augmented reality. AK Peters Wellesley, 2005
4.  A. L. Janin, D. W. Mizell, and T. P. Caudell, “Calibration of head-mounted displays for augmented reality applications,” in Virtual Reality Annual International Symposium, 1993., 1993 IEEE. IEEE, 1993, pp. 246–255.
5. P. Milgram and F. Kishino, “A taxonomy of mixed reality visual displays,” IEICE TRANSACTIONS on Information and Systems, vol. 77, no. 12, pp. 1321–1329, 1994.
6. W. Piekarski, B. Gunther, and B. Thomas, “Integrating virtual and augmented realities in an outdoor application,” in Augmented Reality, 1999.(IWAR’99) Proceedings. 2nd IEEE and ACM International Workshop on. IEEE, 1999, pp. 45–54. [12] S. K. Ong, A. Y. Nee, and S. K. Ong, Virtual Reality and Augmented Reality Applications in Manufacturing. SpringerVerlag, 2004.
7.  I. E. Sutherland and C. A. Mead, “Microelectronics and computer science,” Scientific American, vol. 237, pp. 210–228, 1977.
8.  A. Shatte, J. Holdsworth, and I. Lee, “Mobile augmented reality based context-aware library management system,” Expert Systems with Applications, vol. 41, no. 5, pp. 2174–2185, 2014.
9. F. Zhou, H. B.-L. Duh, and M. Billinghurst, “Trends in augmented reality tracking, interaction and display: A review of ten years of ismar,” in Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE Computer Society, 2008, pp. 193–202.