Agnostic Extended Reality Development Strategies and Policies: A Systematic Literature Review
Keywords:
PRISMA systematic review, XR development strategy and policy framework, XR literature sources, Title-focused Boolean search, Co-occurrence phrase searchAbstract
This work performed two PRISMA studies on literature from four academic databases, two search platforms, ten non-indexed journals, and eight XR websites from March to May 2023. The first study used a title-focused Boolean keyword search. The second study used a co-occurrence phrase search. The keywords and phrases relate to finding literature containing an XR development strategy or policy. The exclusion criteria involved non-English records, publications before 2019, and titles omitting search keywords or literature without co-occurrence phrases. The studies produced one eligible record. It is the authors’ 2023 publication of an agnostic XR development reference strategy used for the research questions, inclusion and exclusion criteria, and codebook. This research creates a systematic exclusion structure that researchers and stakeholders can adopt to identify potential XR development strategies or policies. The quantified breakdown of excluded and included literature from various sources and search strategies can guide future research work. This paper addresses the challenges of finding suitable literature and recommends removing data limitations, increasing literature sources, utilising non-academic search engines and AI LLMs, and implementing real-time alerts for new publications. The single eligible record highlights the research gap. It justifies the research focus on stakeholders' opinions and perspectives. The study identifies existing XR development literature with the potential to contribute to a partial strategy. It emphasises the need for more publications on clearly defined agnostic XR development strategies and policies. This paper demonstrates the value of reporting and publishing PRISMA studies that contain one or fewer records in the inclusion dataset.
Downloads
References
Alsanea, Z. M., & Alawaidi, A. H. (2021). The reality of social studies Teachers Awareness regarding the notion of augmented reality technology and the strategy of its usage in teaching to develop geographical phenomena from the social studies teachers perspectives. Journal of Educational and Psychological Sciences, 5(41), 57–39. https://doi.org/10.26389/AJSRP.N280421
AREA. (n.d.). Research Reports. Retrieved May 24, 2023, from https://thearea.org/research-reports-and-case-studies/
Barteit, S., Lanfermann, L., Bärnighausen, T., Neuhann, F., & Beiersmann, C. (2021). Augmented, Mixed, and Virtual Reality-Based Head-Mounted Devices for Medical Education: Systematic Review. JMIR Serious Games, 9(3), e29080. https://doi.org/10.2196/29080
Berlemont, K. (2022, September 2). Using AI To Improve Your Literature Review. Medium. https://pub.towardsai.net/using-ai-to-improve-your-literature-review-bb2d53348778
Créquit, P., Trinquart, L., Yavchitz, A., & Ravaud, P. (2016). Wasted research when systematic reviews fail to provide a complete and up-to-date evidence synthesis: The example of lung cancer. BMC Medicine, 14(1), 1–15. https://doi.org/10.1186/S12916-016-0555-0/TABLES/2
de Winter, J. C. F., Zadpoor, A. A., & Dodou, D. (2013). The expansion of Google Scholar versus Web of Science: a longitudinal study. Scientometrics 2013 98:2, 98(2), 1547–1565. https://doi.org/10.1007/S11192-013-1089-2
Diaba-Nuhoho, P., & Amponsah-Offeh, M. (2021). Reproducibility and research integrity: the role of scientists and institutions. BMC Research Notes, 14(1), 451. https://doi.org/10.1186/s13104-021-05875-3
Elliott, J. H., Turner, T., Clavisi, O., Thomas, J., Higgins, J. P. T., Mavergames, C., & Gruen, R. L. (2014). Living Systematic Reviews: An Emerging Opportunity to Narrow the Evidence-Practice Gap. PLoS Medicine, 11(2), e1001603. https://doi.org/10.1371/journal.pmed.1001603
EON Reality. (n.d.). EON Reality. Retrieved May 24, 2023, from https://eonreality.com/
EuroXR. (n.d.). Research. Retrieved May 24, 2023, from https://www.euroxr-association.org/research/
EuroXR Association. (n.d.). EuroXR. Retrieved January 13, 2023, from https://www.euroxr-association.org/
Fitter, K., & Sinha, S. (2022). Development of analytical solution for Photoacoustic Imaging of an Extended line source with acoustic lens based reconstruction strategy. Proceedings of PCEMS 2022 - 1st International Conference on the Paradigm Shifts in Communication, Embedded Systems, Machine Learning and Signal Processing, 124–129. https://doi.org/10.1109/PCEMS55161.2022.9808014
Ghobakhloo, M., Iranmanesh, M., Morales, M. E., Nilashi, M., & Amran, A. (2023). Actions and approaches for enabling Industry 5.0-driven sustainable industrial transformation: A strategy roadmap. Corporate Social Responsibility and Environmental Management, 30(3), 1473–1494. https://doi.org/10.1002/CSR.2431
Giustini, D., & Kamel Boulos, M. N. (2013). Google Scholar is not enough to be used alone for systematic reviews. Online Journal of Public Health Informatics, 5(2), 214. https://doi.org/10.5210/ojphi.v5i2.4623
Gray, R. (2021). Empty systematic reviews: Identifying gaps in knowledge or a waste of time and effort? Nurse Author & Editor, 31(2), 42–44. https://doi.org/10.1111/nae2.23
Gusenbauer, M. (2019). Google Scholar to overshadow them all? Comparing the sizes of 12 academic search engines and bibliographic databases. Scientometrics, 118(1), 177–214. https://doi.org/10.1007/S11192-018-2958-5/TABLES/4
Haddaway, N. R., Collins, A. M., Coughlin, D., & Kirk, S. (2015). The role of google scholar in evidence reviews and its applicability to grey literature searching. PLoS ONE, 10(9). https://doi.org/10.1371/journal.pone.0138237
Haddaway, N. R., Page, M. J., Pritchard, C. C., & McGuinness, L. A. (2022). PRISMA2020: An R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis. Campbell Systematic Reviews, 18(2), e1230. https://doi.org/10.1002/CL2.1230
Hizam-Hanafiah, M., Soomro, M., & Abdullah, N. (2020). Industry 4.0 Readiness Models: A Systematic Literature Review of Model Dimensions. Information, 11(7), 364. https://doi.org/10.3390/info11070364
IDB Lab. (2022). Metaverse Communities Challenge. https://convocatorias.iadb.org/en/bid-lab/metaverse-communities
Ikhsan, M., Guntur, S., Setyaningrum, W., Retnawati, H., Marsigit, M., Saragih, N. A., & Kahir Bin Noordin, M. (2019). Developing augmented reality in mathematics learning: The challenges and strategies. Jurnal Riset Pendidikan Matematika, 6(2), 211–221. https://doi.org/10.21831/JRPM.V6I2.28454
International Journal of Virtual Reality. (n.d.). Retrieved January 10, 2023, from https://ijvr.eu/about
International Journal on Stereo & Immersive Media. (n.d.). Retrieved January 30, 2023, from https://revistas.ulusofona.pt/index.php/stereo/about
Iqbal, S. A., Wallach, J. D., Khoury, M. J., Schully, S. D., & Ioannidis, J. P. A. (2016). Reproducible Research Practices and Transparency across the Biomedical Literature. PLOS Biology, 14(1), e1002333. https://doi.org/10.1371/journal.pbio.1002333
Ishamuddin Mustapha, Masroor Ali, Nohman Khan, & Huma Sikandar. (2023). The Impact of Industry 4.0 on Innovative Organisations, A Thematic Review using the PRISMA Statement 2020. International Journal of Interactive Mobile Technologies (IJIM), 17(09), 88–105. https://doi.org/10.3991/ijim.v17i09.39465
IVRPA. (n.d.). International Virtual Reality Professionals Association. Retrieved January 13, 2023, from https://ivrpa.org/
Journal of Immersive Media and Creative Arts. (n.d.). Retrieved January 30, 2023, from https://jimca.org/index.php/1/about
Journal of the Virtual Reality Society of Japan. (n.d.). Retrieved January 10, 2023, from https://www.jstage.jst.go.jp/browse/jvrsj/_pubinfo/-char/en
Kiani, S., Rezaei, I., Abasi, S., Zakerabasali, S., & Yazdani, A. (2023). Technical aspects of virtual augmented reality-based rehabilitation systems for musculoskeletal disorders of the lower limbs: a systematic review. BMC Musculoskeletal Disorders, 24(1), 4. https://doi.org/10.1186/S12891-022-06062-6/TABLES/3
Lee, E.-T., & Im, S. (2022). A Development Study of Instructional Design Strategies for Metaverse Based on Goal-Directed Design Methodology. Korean Association for Educational Information and Media, 28(4), 983–1010. https://doi.org/10.15833/KAFEIAM.28.4.983
Liao, Y., Deschamps, F., Loures, E. de F. R., & Ramos, L. F. P. (2017). Past, present and future of Industry 4.0 - a systematic literature review and research agenda proposal. International Journal of Production Research, 55(12), 3609–3629. https://doi.org/10.1080/00207543.2017.1308576
Liu, H. X. (2022). Building the “Complete Game”: An Overview Study of a Development Strategy for Geo AR Mobile Games. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 13517 LNCS, 604–622. https://doi.org/10.1007/978-3-031-22131-6_45/COVER
Loutoo, J. (2021, April 5). UWI secures US$25 million technology grant. Trinidad and Tobago Newsday. https://newsday.co.tt/2021/04/05/uwi-secures-us25-million-technology-grant/
Matthews, B., See, Z. S., & Day, J. (2021). Crisis and extended realities: remote presence in the time of COVID-19. Media International Australia, 178(1), 198–209. https://doi.org/10.1177/1329878X20967165
McGowan, A., Gui, Y., Dobbs, M., Shuster, S., Cotter, M., Selloni, A., Goodman, M., Srivastava, A., Cecchi, G. A., & Corcoran, C. M. (2023). ChatGPT and Bard exhibit spontaneous citation fabrication during psychiatry literature search. Psychiatry Research, 326, 115334. https://doi.org/10.1016/j.psychres.2023.115334
McGuirt, J. T., Cooke, N. K., Burgermaster, M., Enahora, B., Huebner, G., Meng, Y., Tripicchio, G., Dyson, O., Stage, V. C., & Wong, S. S. (2020). Extended Reality Technologies in Nutrition Education and Behavior: Comprehensive Scoping Review and Future Directions. Nutrients, 12(9), 2899. https://doi.org/10.3390/nu12092899
Meta. (n.d.). Meta. Retrieved May 24, 2023, from https://www.meta.com/
Metaverse. (n.d.). Retrieved January 10, 2023, from https://aber.apacsci.com/index.php/met/index
Metaverse Standards Forum. (n.d.). Active Domain Groups. Retrieved May 24, 2023, from https://metaverse-standards.org/domain-groups/
Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4(1), 1. https://doi.org/10.1186/2046-4053-4-1
Montgomery, P. (n.d.). How empty are empty reviews? University of Birmingham. Retrieved April 27, 2023, from https://www.birmingham.ac.uk/schools/social-policy/departments/social-policy-sociology-criminology/research/projects/2017/Empty-reviews.aspx
Naseem, M. H., & Yang, J. (2021). Role of Industry 4.0 in Supply Chains Sustainability: A Systematic Literature Review. Sustainability, 13(17), 9544. https://doi.org/10.3390/su13179544
Nassereddine, H. M. (2019). Design, Development and Validation of an Augmented Reality-Enabled Production Strategy Process for the Construction Industry [UNIVERSITY OF WISCONSIN–MADISON]. https://www.proquest.com/central/docview/2231630646/fulltextPDF/EA7029A0B6F14E94PQ/1?accountid=45039
Nassereddine, H., Veeramani, D., & Hanna, A. S. (2022). Design, Development, and Validation of an Augmented Reality-Enabled Production Strategy Process. Frontiers in Built Environment, 8, 5. https://doi.org/10.3389/FBUIL.2022.730098/BIBTEX
OAS. (2021, December 15). OAS and Meta Partner to Train Thousands of Latin American and Caribbean Creators in Augmented Reality. https://www.oas.org/en/media_center/press_release.asp?sCodigo=E-116/21
Object Management Group. (n.d.). AREA. Retrieved January 13, 2023, from https://thearea.org/
Orduna-Malea, E., Martín-Martín, A., & López-Cózar, E. D. (2017). Google Scholar as a source for scholarly evaluation: a bibliographic review of database errors. Revista Española de Documentación Científica, 40(4), 1–33. https://doi.org/10.3989/REDC.2017.4.1500
Oyekan, J. O., Hutabarat, W., Tiwari, A., Grech, R., Aung, M. H., Mariani, M. P., López-Dávalos, L., Ricaud, T., Singh, S., & Dupuis, C. (2019). The effectiveness of virtual environments in developing collaborative strategies between industrial robots and humans. Robotics and Computer-Integrated Manufacturing, 55, 41–54. https://doi.org/10.1016/j.rcim.2018.07.006
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Systematic Reviews, 10(1), 1–11. https://doi.org/10.1186/S13643-021-01626-4/FIGURES/1
Park, S.-M., & Kim, Y.-G. (2022). A Metaverse: Taxonomy, Components, Applications, and Open Challenges. IEEE Access, 10, 4209–4251. https://doi.org/10.1109/ACCESS.2021.3140175
Phaal, R., Farrukh, C. J. P., & Probert, D. R. (2004). Technology roadmapping—A planning framework for evolution and revolution. Technological Forecasting and Social Change, 71(1–2), 5–26. https://doi.org/10.1016/S0040-1625(03)00072-6
Phaal, R., Farrukh, C. J. P., & Probert, D. R. (2009). Visualising strategy: a classification of graphical roadmap forms. International Journal of Technology Management, 47(4), 286. https://doi.org/10.1504/IJTM.2009.024431
Proceedings of Annual Conference of Japan Society for Medical Virtual Reality. (n.d.). Retrieved January 10, 2023, from https://www.jstage.jst.go.jp/browse/jsmvr/_pubinfo/-char/en
Qureshi, R., Shaughnessy, D., Gill, K. A. R., Robinson, K. A., Li, T., & Agai, E. (2023). Are ChatGPT and large language models “the answer” to bringing us closer to systematic review automation? Systematic Reviews, 12(1), 72. https://doi.org/10.1186/S13643-023-02243-Z/PEER-REVIEW
Rameshwar, J. R., & King, G. S. (2020). Case Studies in Engineering and Technology Innovation in the Caribbean: A focus on EduColCom and BevCom. In B. V. Chowdary (Ed.), Proceedings of the International Conference on Emerging Trends in Engineering & Technology (IConETech-2020) (pp. 574–584). Faculty of Engineering, The University of the West Indies, St. Augustine. https://doi.org/10.47412/EBWB6632
Rameshwar, J. R., & King, G. S. (2022). Caribbean Metaverse Development: A Literature Review Perspective. Journal of Metaverse, 2(2), 83–99. https://dergipark.org.tr/en/pub/jmv/issue/71114/1120470
Rameshwar, J. R., & King, G. S. (2023a). An Explanation of the Codebook for Analysing XR Development. International Journal of Advanced Virtual Reality, 1(2), 52–69. https://doi.org/10.5281/ZENODO.10406387
Rameshwar, J. R., & King, G. S. (2023b). Analysis of Caribbean XR Survey Creates an XR Development Strategy as a Path to the Regional Metaverse Evolution. Journal of Metaverse, 3(1), 43–65. https://doi.org/10.57019/jmv.1179104
Rauschnabel, P. A., Felix, R., Hinsch, C., Shahab, H., & Alt, F. (2022). What is XR? Towards a Framework for Augmented and Virtual Reality. Computers in Human Behavior, 133, 107289. https://doi.org/10.1016/j.chb.2022.107289
Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., Blunt, H., Brigham, T., Chang, S., Clark, J., Conway, A., Couban, R., de Kock, S., Farrah, K., Fehrmann, P., Foster, M., Fowler, S. A., Glanville, J., … Young, S. (2021). PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Systematic Reviews, 10(1), 1–19. https://doi.org/10.1186/S13643-020-01542-Z/TABLES/1
Sanderson, K. (2023). AI science search engines are exploding in number — are they any good? Nature, 616(7958), 639–640. https://doi.org/10.1038/d41586-023-01273-w
Sun, Y., Li, H., & Khan, Y. A. (2022). Impact of monetary policy on China’s tourism market development: An application of factor –augmented vector auto-regression (FAVAR) model. Current Psychology, 1–20. https://doi.org/10.1007/S12144-021-02562-1/TABLES/6
The Alma Jordan Library. (n.d.). UWIlinC. Retrieved January 7, 2023, from https://uwin-primo.hosted.exlibrisgroup.com/primo-explore/search?vid=STA
The Japanese Journal for Medical Virtual Reality. (n.d.). Retrieved January 10, 2023, from https://www.jstage.jst.go.jp/browse/jmvr/_pubinfo/-char/en
The Khronos Group. (n.d.). The Metaverse Standards Forum. Retrieved January 13, 2023, from https://metaverse-standards.org/
The VR/AR Association. (n.d.-a). Case Studies & White Papers. Retrieved May 24, 2023, from https://www.thevrara.com/casestudiesandwhitepapers
The VR/AR Association. (n.d.-b). VR/AR Association. Retrieved January 13, 2023, from https://www.thevrara.com/
The XR Association. (n.d.). XR Association. Retrieved January 13, 2023, from https://xra.org/
Transactions of the Virtual Reality Society of Japan. (n.d.). Retrieved January 10, 2023, from https://www.jstage.jst.go.jp/browse/tvrsj/_pubinfo/-char/en
Vasarainen, M., Paavola, S., & Vetoshkina, L. (2021). A Systematic Literature Review on Extended Reality: Virtual, Augmented and Mixed Reality in Working Life. International Journal of Virtual Reality, 21(2), 1–28. https://doi.org/10.20870/IJVR.2021.21.2.4620
VAST: International Symposium on Virtual Reality. (n.d.). Retrieved January 1, 2023, from https://fatcat.wiki/container/e2adeuzd2zcotj2f6hxoab5wfq
Virtual Reality & Intelligent Hardware. (n.d.). Retrieved May 5, 2023, from https://www.sciencedirect.com/journal/virtual-reality-and-intelligent-hardware
Welch, V., Petticrew, M., Tugwell, P., Moher, D., O’Neill, J., Waters, E., White, H., Atun, R., Awasthi, S., Barbour, V., Bhutta, Z., Cuervo, L. G., Grimshaw, J., Groves, T., Koehlmoos-Perez, T., Kristjansson, E., O’Neill, J., Oxman, A., Pantoja, T., … Wells, G. (2012). PRISMA-Equity 2012 Extension: Reporting Guidelines for Systematic Reviews with a Focus on Health Equity. PLOS Medicine, 9(10), e1001333. https://doi.org/10.1371/JOURNAL.PMED.1001333
XR Association. (2018, October). XRA’S DEVELOPERS GUIDE, CHAPTER ONE: Fundamental Design Principles for Immersive Experiences. https://xra.org/research/xr-primer-1-0-a-starter-guide-for-developers/
XR Association. (2019, December). XRA’S DEVELOPERS GUIDE, CHAPTER TWO: Creating Safe, Inclusive, and Respectful Immersive Experiences. https://xra.org/research/xr-primer-2-0-a-starter-guide-for-developers/
XR Association. (2020, October). XRA’S DEVELOPERS GUIDE, CHAPTER THREE: Accessibility & Inclusive Design in Immersive Experiences. https://xra.org/research/xra-developers-guide-accessibility-and-inclusive-design/
XR Association. (2022, November). XRA’S DEVELOPERS GUIDE, CHAPTER FOUR: Designing Immersive Learning for Secondary Education. https://xra.org/research/xras-developers-guide-chapter-four-designing-immersive-learning-for-secondary-education/
Yaffe, J., Montgomery, P., Hopewell, S., & Shepard, L. D. (2012). Empty reviews: A description and consideration of cochrane systematic reviews with no included studies. PLoS ONE, 7(5). https://doi.org/10.1371/JOURNAL.PONE.0036626
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Jason Robert Rameshwar, Graham S. King
This work is licensed under a Creative Commons Attribution 4.0 International License.
Accepted 2024-03-29
Published 2025-10-03
