Skip to main content

Application of holography and augmented reality based technology to visualize the internal structure of the dental root – a proof of concept

Abstract

Background

The Augmented Reality (AR) blends digital information with the real world. Thanks to cameras, sensors, and displays it can supplement the physical world with holographic images. Nowadays, the applications of AR range from navigated surgery to vehicle navigation.

Development

The purpose of this feasibility study was to develop an AR holographic system implementing Vertucci’s classification of dental root morphology to facilitate the study of tooth anatomy. It was tailored to run on the AR HoloLens 2 (Microsoft) glasses. The 3D tooth models were created in Autodesk Maya and exported to Unity software. The holograms of dental roots can be projected in a natural setting of the dental office. The application allowed to display 3D objects in such a way that they could be rotated, zoomed in/out, and penetrated. The advantage of the proposed approach was that students could learn a 3D internal anatomy of the teeth without environmental visual restrictions.

Conclusions

It is feasible to visualize internal dental root anatomy with AR holographic system. AR holograms seem to be attractive adjunct for learning of root anatomy.

Peer Review reports

Introduction

Dental students have to learn dental anatomy for the successful diagnosis and treatment of different pathological processes. Currently, students have a wide range of books, videos, lectures, and seminars from which they can study. However, recent technological advancements offer new possibilities for improvement of the effectiveness of the learning process [1, 2].

In 2016, Microsoft released a product called “HoloLens” (www.microsoft.com/en-us/hololens). It is a device resembling goggles with a built-in battery, microcomputer, cameras, microphone, speakers, and holographic projectors. Installed depth cameras analyze the entire environment around the user (e.g. position and shape of tables, chairs, walls, etc.) and the microcomputer decides about a way of displaying a hologram so that it is compatible with the environment. This enhancement of the real world by computer-generated information is called augmented reality (AR). Virtual reality (VR), as opposed to AR, replaces the surrounding reality with the digital world so that users’ perception of reality is entirely based on virtual information. With continuous improvement of the performance of head-mounted displays, the potential of VR and AR is increasing - they are more and more often used in medicine [3, 4], dentistry [5,6,7,8], and education in health sciences [9,10,11,12].

In endodontics, knowledge of the internal anatomy of teeth is instrumental. Students and clinicians must have a well-developed spatial vision to imagine the course of root canals when they start endodontic treatment. Attempts to utilize VR [13] or HoloHuman application [14] for teaching head anatomy demonstrated potential of holographic technologies as an adjunct tool in this field. Moreover, Song et al. [15] introduced the general idea of endodontic treatment assisted by AR-guided technologies. Thus, the purpose of this paper was to describe the development of a Microsoft HoloLens 2-based application enabling 3D display of the internal anatomy of dental roots for facilitation of learning process.

Development process

Holography

First reports about early holographic technology date back to 19th century [16]. Later, hologram was defined as an interference pattern between a coherent reference beam and the wave, which has been scattered by the real object [17]. Recent rapid and versatile development of holographic technology [18] holds great promise for medical education because it enables to visualize complex structures or procedures taught during classes.

Augmented reality

AR is an enhancement of the real world by computer-generated information. Using AR glasses, the user sees the environment he/she is in and the glasses display holograms harmonizing with the setting. A considerable benefit of AR in comparison to VR is that the user never loses orientation in the environment (frequent for someone using VR devices).

The HoloLens 2 (Microsoft, Redmond, USA) (Fig. 1) is a commercially available system of AR. It is an independent device and does not require a separate operation space and manual controllers. It is fully integrated with Microsoft Enterprise systems, which resembles other Microsoft operating systems (e.g. Windows 10). The disadvantage of HoloLens 2 is that the commercial license is currently rather expensive [19]. Some educational trainings were already developed [20,21,22].

Fig. 1
figure 1

HoloLens 2 glasses

Development of the application

We planned to implement all holographic features offered by the HoloLens 2 device: (a) ability to display images truly 3-dimensionally – they are projected as 3D objects in front of the user so that he/she can “feel” them. It is a significant progress in comparison to today’s standard according to which 3D models are displayed only in 2D views. PC’s laptops, tablets, and books offer the option to view the 3D model but only in a two-dimensional perspective. One can rotate or view models from different angles, but these are only “3D-like” views because they are shown on a flat screen/page; (b) display of 3D models in such a way that they look like they are displayed in a specific area of the space. For example, one can force a device to display a model above the table and then walk around the table but with the model always “hanging” in the same place above the table. This feature of HoloLens 2 allows watching the 3D models from any perspective, angle, or place; (c) change of view perspective depending on the angle of observation; (d) automatic scanning of the environment to determine where the 3D model(s) would be placed in the space; (e) possibility that a user can look into the 3D object just by moving his/her head into it.

The development process included several steps (Fig. 2):

Fig. 2
figure 2

 A scheme of the HoloLens-based application development

Fig. 3
figure 3

The root canal configurations from the pulp chamber to the root apex according to Vertucci [23] displayed with our application utilizing the HoloLens device

a) development of the 3D model of a root of the tooth (maxillary 1st. premolar) with a different pattern of course of the canal. The 3D model had to define connections, separations, or change in the direction of the root canal(s). It was prepared accordingly to Vertucci’s classification [23] - (Fig. 3). The entire work was done using Autodesk Maya – software used to create 3D models for games and animations. When the model was done, it was exported to Unity software - an application used to create video games and AR software for different devices and systems. A large benefit of this application is that it allows creating AR applications/games for the HoloLens 2 device. Thanks to it, the model created in Maya could be imported to the project;

b) creation of scripts. The scripts were needed to:

  • place a model (root) in the exact position in the user-visible area (anchoring the model in the environment).

  • animate action for “clicked”/selected object (root) in the way that it moved to the middle of the screen, increased its size, and started to rotate.

  • animate action for returning the model (root) to its original place and to mark it as already used/selected.

  • perform ‘main management’, i.e., the task of identifying which root was selected and determining where it should return;

c) testing the application in Unity emulator twitch export ready project with Microsoft Visual Studio. This development software is responsible for the final compilation process in the way that the application could be installed on the Microsoft HoloLens 2 device.

We created the 3D model of the tooth using the above-described steps. Then, the model was extended to include several types of root canal morphology according to [23]. Then, the 3D models were exported into the Unity application. Each model was filled with a script responsible for doing the programmed action. As a result, when the user starts the application, he/she can see 8 root models, each with different morphology and course of the canal. After pointing and triggering a click with the finger, the selected root model moves to the front of the user, changes its size by around 3 times, and starts to rotate slowly. In the background, the user can hear the lector gives a short definition of the selected model. The user has also an option to point to the model and to trigger a click to stop the rotation of the model. Another click would force the model to go back to its original size and to return to its original place in the row and column. This way the user can feel as if he/she took a root of the tooth from the shelf, watched, returned, and took possibly another one if need be (Fig. 4).

Fig. 4
figure 4

Experimental holograms projected in the operating room of the dental office

Initial evaluation

Holograms of dental roots projected in a natural setting of the dental office (Fig. 4) and showed to dental school instructors and dental students provoked great interest. All participants of the presentation of the application were amazed by the holograms of the root models. Everyone agreed that holography and AR technologies could offer a significant opportunity to facilitate the learning process during dental studies.

The formal part of the initial evaluation of the application consisted of 2 components: (1) demonstration of the application and (2) answering a questionnaire (Table 1). During the demonstration, each evaluator was required to wear a HoloLens 2 headset and manipulate the displayed dental anatomy models themselves. During the demonstration, a member of the research team was always present to assist the raters, answer questions, and resolve technical issues if they arose. The demonstration was conducted once. The characteristics of the evaluation panel are shown in Table 2.

Table 1 The questionnaire given to the evaluators to complete along with the scores for each question
Table 2 The demographic data of the evaluators

The questionnaire included 8 questions-5 of which were related to the prepared application, and the others described the status of the rater (student vs. physician, gender, and age). The questionnaire was available online at the demonstration site, and evaluators were asked to complete it immediately after the demonstration. The results of the assessment are shown in Table 1.

Discussion and conclusions

In this paper, we present the application of holography and AR for visualizing the internal structure of the dental root. The use of this technology in medicine, especially dentistry, is a relatively new approach. The first commercial course using AR for teaching purposes was published in 2016 by Case Western Reserve University School of Medicine (Cleveland, OH, USA) in collaboration with Cleveland Clinic (Cleveland, Ohio) [24]. Their HoloAnatomy allowed the body and its structures, systems, and organs to be rotated and virtually dissected. Compared to traditional cadaver dissection, this is an effective and time-saving method for teaching anatomy [25]. Our team has also actively used the HoloAnatomy application in teaching medical and dental students. Another mixed reality (MR)-based application, HoloPatient, is a learning tool for nurses [26]. Recently, advances have also been made in the field of applying immersive technologies (VR, AR, and MR) to dentistry [27]. Jiang et al. [28] presented an AR-based application that allowed interactive display of the planned preparation curves of adjacent teeth. An interesting study on a three-dimensional AR system with integrated videography for displaying oral and maxillary regions was presented by Suenaga et al. [29]. It allowed stereoscopic visualization of 3D CT-IV images in situ superimposed on the surgical area with the naked eye. Dental education using VR in the field of dental anatomy was presented by Libermann and Erdelt [30]. It was found that the proposed solutions helped over 90% of the students to better understand the presented subject. In turn, Morales-Vadillo et al. [31] described the use of the online virtual world with a social environment to further explore the topic of dental clinical situations. Their study suggests that this helps students to understand the anatomical interaction better than in the traditional model of dental education.

Our results show that AR- and holography-based technology can be successfully used in student education, especially in the area of dental root anatomy. It allows consultations with experts from different fields and sharing 3D images in real time. Students have the opportunity to learn dental morphology and pathology not only by reading or watching a video, but also by immersing themselves in the enhanced transparency. Although our application refers to a part of dental anatomy, the high prevalence of endodontic problems in many countries and the difficulty of learning root canal morphology suggest that the use of our invention may be justified. Moreover, dental morphology is only one possible application. The future development of holography and AR-based training system may help to create an operating room with unlimited perspective that can be used in the training system for students, especially in the field of dental implant surgery. The application of immersive technologies can have a positive impact on the results of dynamic navigation.

Despite the potentially promising and broad application of holographic technology in medicine and dentistry, it is also important to mention its drawbacks such as its relatively high price and the challenges of ensuring proper hygiene when using the HoloLens2 goggles. Regarding the high price, it can be expected to drop as it is commonly seen with the spread of the technology or device. Disinfecting with liquids every time and using disposable gloves could provide an adequate level of hygiene. Other potential disadvantages include development of mild eyestrain, headaches, or motion sickness in some students [32]. Of course, there is no substitute for face-to-face contact with the teacher, but especially in times of a pandemic, this technology offers reasonable alternative.

It is also important to note that this study only tested to a limited extent the effectiveness of the developed application as an adjunct in the dental education process.

Availability of data and materials

Not applicable.

References

  1. Singal A, Bansal A, Chaudhary P, Singh H, Patra A. Anatomy education of medical and dental students during COVID-19 pandemic: a reality check. Surg Radiol Anat. 2021;43(4):515–21. doi:https://doi.org/10.1007/s00276-020-02615-3.

    Article  PubMed  Google Scholar 

  2. Conte DB, Zancanaro M, Guollo A, Schneider LR, Lund RG, Rodrigues-Junior SA. Educational Interventions to Improve Dental Anatomy Carving Ability of Dental Students: A Systematic Review. Anat Sci Educ. 2021;14(1):99–109. doi:https://doi.org/10.1002/ase.2004.

    Article  PubMed  Google Scholar 

  3. Andrews C, Southworth MK, Silva JNA, Silva JR. Extended Reality in Medical Practice. Curr Treat Options Cardiovasc Med. 2019;21(4):18. doi:https://doi.org/10.1007/s11936-019-0722-7.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Eckert M, Volmerg JS, Friedrich CM. Augmented Reality in Medicine: Systematic and Bibliographic Review. JMIR Mhealth Uhealth. 2019;7(4):e10967. doi:https://doi.org/10.2196/10967.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Ma L, Jiang W, Zhang B, Qu X, Ning G, Zhang X, et al. Augmented reality surgical navigation with accurate CBCT-patient. Med Biol Eng Comput. 2019;57(1):47–57. doi:https://doi.org/10.1007/s11517-018-1861-9.

    Article  PubMed  Google Scholar 

  6. Suenaga H, Hoang Tran H, Liao H, Masamune K, Dohi T, Hoshi K, et al. Real-time in situ three-dimensional integral videography and surgical navigation using augmented reality: a pilot study. Int J Oral Sci. 2013;5(2):98–102. doi:https://doi.org/10.1038/ijos.2013.26.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Bartella AK, Kamal M, Scholl I, Schiffer S, Steegmann J, Ketelsen D. et al. Virtual reality in preoperative imaging in maxillofacial surgery: implementation of “the next level”? Br J Oral Maxillofac Surg. 2019;57(7):644–8. doi:https://doi.org/10.1016/j.bjoms.2019.02.014.

  8. Wang J, Suenaga H, Yang L, Kobayashi E, Sakuma I. Video see-through augmented reality for oral and maxillofacial surgery. Int J Med Robot. 2017;13(2). doi:https://doi.org/10.1002/rcs.1754.

  9. Kim-Berman H, Karl E, Sherbel J, Sytek L, Ramaswamy V. Validity and User Experience in an Augmented Reality Virtual Tooth Identification Test. J Dent Edu. 2019;83(11):1345–52. doi:https://doi.org/10.21815/JDE.019.139.

    Article  Google Scholar 

  10. Talaat S, Ghoneima A, Kaboudan A, Talaat W, Ragy N, Bourauel C. Three-dimensional evaluation of the holographic projection in digital dental model superimposition using HoloLens device. Orthod Craniofac Res. 2019;22 (Suppl 1):62-8. doi: https://doi.org/10.1111/ocr.12286. PMID: 31074156.

  11. Thoma J, Havlena M, Stalder S, Van Goll L. Augmented Reality for User-Friendly Intra-Oral Scanning. 2017 IEEE International Symposium on Mixed and Augmented Reality Adjunct Proceedings. Conference Poster 2017.

  12. Cihak DF, Moore EJ, Wright RE, McMahon DD, Gibbons MM, Smith C. Evaluating Augmented Reality to Complete a Chain Task for Elementary Students With Autism. J Spec Edu Technol. 2016;31(2):99–108. doi:https://doi.org/10.1177/0162643416651724.

    Article  Google Scholar 

  13. Reymus M, Liebermann A, Diegritz C. Virtual reality: an effective tool for teaching root canal anatomy to undergraduate dental students – a preliminary study. Int Endo J. 2020;53(11):1581–7. doi:https://doi.org/10.1111/iej.13380.

    CAS  Article  Google Scholar 

  14. Zafar S, Zachar JJ. Evaluation of HoloHuman augmented reality application as a novel educational tool in dentistry. Eur J Dent Edu. 2020;24(2):259–65. doi:https://doi.org/10.1111/eje.12492.

    Article  Google Scholar 

  15. Song T, Yang C, Diant O, Azimi E. Endodontic Guided Treatment Using Augmented Reality on a Head-Mounted Display System. Healthc Technol Lett. 2018;5(5):201–7. doi:https://doi.org/10.1049/htl.2018.5062.

    Article  Google Scholar 

  16. Johnston SF. Channeling Dreams. Holograms: A Cultural History. 1st ed.; London: Oxford University Press, 2015.

    Chapter  Google Scholar 

  17. Gabor DA. New microscopic principle. Nature. 1948;161:777–8. doi:https://doi.org/10.1038/161777a0.

    CAS  Article  PubMed  Google Scholar 

  18. Corda R, Giusto D, Liotta A, Song W, Perra C. Recent Advances in the Processing and Rendering Algorithms for Computer-Generated Holography. Electronics. 2019;8(5):556. doi:https://doi.org/10.3390/electronics8050556.

    Article  Google Scholar 

  19. Ogdon DC. HoloLens And ViVE Pro: Virtual Reality Headsets. J Med Lib Assoc. 2019;107(1):118–21. doi:https://doi.org/10.5195/jmla.2019.602.

    Article  Google Scholar 

  20. https://3d4medical.com/apps/holohuman.

  21. www.pearson.com/uk/web/pearsontq/news-and-blogs/2017/04/introducing-holopatient.html.

  22. www.microsoft.com/en-ca/p/holo-chemistry/9p25hnkpmh4n?activetab=pivot:overviewtab.

  23. Vertucci FJ. Root canal anatomy of the human permanent teeth. Oral Surg Oral Med Oral Pathol. 1984;58(5):589–99. doi:https://doi.org/10.1016/0030-4220(84)90085-9.

    CAS  Article  PubMed  Google Scholar 

  24. Digital Trends. Available online: https://www.digitaltrends.com/virtual-reality/hololens-holoanatomy-award-jackson-hole-science-media-awards/ (accessed on October 12, 2021).

  25. Ruthberg JS, Quereshy HA, Ahmadmehrabi S, Trudeau S, Chaudry E, et al. A Multimodal Multi-Institutional Solution to Remote Medical Student Education for Otolaryngology During COVID-19. Otolaryngol Head Neck Surg. 2020 Oct;163(4):707–9. doi: https://doi.org/10.1177/0194599820933599.

  26. Bin S, Masood S, Jung Y. Chapter Twenty - virtual and augmented reality in medicine. In: Dagan Feng D, editor. Biomedical Engineering. Academic Press Elsevier; 2020. pp. 673–86.

  27. Moussa R, Alghazaly A, Althagafi N, Eshky R, Borzangy S. Effectiveness of Virtual Reality and Interactive Simulators on Dental Education Outcomes: Systematic Review. Eur J Dent. 2021 Aug 24. doi:https://doi.org/10.1055/s-0041-1731837. Epub ahead of print.

  28. Jiang J, Guo Y, Huang Z, Zhang Y, Wu D, Liu Y. Adjacent surface trajectory planning of robot-assisted tooth preparation based on augmented reality. Eng Sci Technol. 2021. doi:https://doi.org/10.1016/j.jestch.2021.05.005. Epub ahead of print.

    Article  Google Scholar 

  29. Liebermann A, Erdelt K. Virtual education: Dental morphologies in a virtual teaching environment. J Dent Educ. 2020;84(10):1143–50. doi:https://doi.org/10.1002/jdd.12235.

    Article  PubMed  Google Scholar 

  30. Suenaga H, Hoang Tran H, Liao H, Masamune K, Dohi T, Hoshi K, et al. Real-time in situ three-dimensional integral videography and surgical navigation using augmented reality: a pilot study. Int J Oral Sci. 2013;5(2):98–102. doi:https://doi.org/10.1038/ijos.2013.26.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Morales-Vadillo R, Guevara-Canales JO, Flores-Luján VC, Robello-Malatto JM, Bazán-Asencios RH, Cava-Vergiú CE. Use of virtual reality as a learning environment in dentistry. Gen Dent. 2019;67(4):21–7.

    PubMed  Google Scholar 

  32. Kuehn BM. Virtual and Augmented Reality Put a Twist on Medical Education. JAMA. 2018;319(8):756–8. doi:https://doi.org/10.1001/jama.2017.20800.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This study was supported and financed by the National Centre for Research and Development under Grant Lider No. LIDER/17/0064/L-11/19/NCBR/2020.

Funding

Jagiellonian University Medical College in Kraków.

Author information

Authors and Affiliations

Authors

Contributions

DDD- performed programming, drafted the manuscript. KP- supervised the study, prepared the concept of the study. MDD - drafted the manuscript. AP – processing and analysing data, drafted the manuscript. JHB – drafted the manuscript. MT – processing and analysing datacritically assessed the manuscript. JC – drafted the manuscript. PW – critically assessed the manuscript. PSF - critically assessed the manuscript, supervised the study. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Piotr S. Fudalej.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

All authors give their consent for the publication of the short report.

Competing interests

Authors have stated explicitly that there are no competing interests in connection with this article.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Dolega-Dolegowski, D., Proniewska, K., Dolega-Dolegowska, M. et al. Application of holography and augmented reality based technology to visualize the internal structure of the dental root – a proof of concept. Head Face Med 18, 12 (2022). https://doi.org/10.1186/s13005-022-00307-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13005-022-00307-4

Keywords

  • Mixed reality
  • Augmented reality
  • Holography
  • Tooth
  • Dental root
  • Root canal
  • Visualization