Teaching WebAR development with integrated machine learning: a methodology for immersive and intelligent educational experiences

- Semerikov, Serhiy O. (orcid.org/0000-0003-0789-0272), Foki, Mykhailo V., Shepiliev, Dmytro S. (orcid.org/0000-0001-6913-8073), Mintii, Mykhailo (orcid.org/0000-0002-0488-5569), Mintii, I.S. (orcid.org/0000-0003-3586-4311) and Kuzminska, Olena (orcid.org/0000-0002-8849-9648) (2024) Teaching WebAR development with integrated machine learning: a methodology for immersive and intelligent educational experiences Educational Dimension (10). pp. 198-234. ISSN 2708-4604

Text ED_660_Semerikov_et_al.pdf - Published Version Download (2MB)

Abstract

Augmented reality (AR) and machine learning (ML) are rapidly growing technologies with immense potential for transforming education. Web-based augmented reality (WebAR) provides a promising approach to delivering immersive learning experiences on mobile devices. Integrating machine learning models into WebAR applications can enable advanced interactive effects by responding to user actions, thus enhancing the educational content. However, there is a lack of effective methodologies to teach students WebAR development with integrated machine learning. This paper proposes a methodology with three main steps: (1) Integrating standard TensorFlow.js models like handpose into WebAR scenes for gestures and interactions; (2) Developing custom image classification models with Teachable Machine and exporting to TensorFlow.js; (3) Modifying WebAR applications to load and use exported custom models, displaying model outputs as augmented reality content. The proposed methodology is designed to incrementally introduce machine learning integration, build an understanding of model training and usage, and spark ideas for using machine learning to augment educational content. The methodology provides a starting point for further research into pedagogical frameworks, assessments, and empirical studies on teaching WebAR development with embedded intelligence.

Item Type:	Article
Keywords:	web-based augmented reality, WebAR, machine learning, TensorFlow.js, Teachable Machine, educational technology
Subjects:	Science and knowledge. Organization. Computer science. Information. Documentation. Librarianship. Institutions. Publications > 00 Prolegomena. Fundamentals of knowledge and culture. Propaedeutics > 004 Computer science and technology. Computing. Data processing Science and knowledge. Organization. Computer science. Information. Documentation. Librarianship. Institutions. Publications > 3 Social Sciences > 37 Education > 37.01/.09 Special auxiliary table for theory, principles, methods and organization of education > 37.09 Organization of instruction
Divisions:	Institute for Digitalisation of Education > Department of Cloud-Oriented Systems and Artificial Intelligence in Education
Depositing User:	Сергій Олексійович Семеріков
Date Deposited:	21 Jul 2024 10:32
Last Modified:	21 Jul 2024 10:32
URI:	https://lib.iitta.gov.ua/id/eprint/741781

Downloads

Actions (login required)

View Item