Methodology for teaching development of web-based augmented reality with integrated machine learning models

- Semerikov, Serhiy O. (orcid.org/0000-0003-0789-0272), Foki, Mykhailo V., Shepiliev, Dmytro S. (orcid.org/0000-0001-6913-8073), Mintii, Mykhailo (orcid.org/0000-0002-0488-5569), Mintii, I.S. (orcid.org/0000-0003-3586-4311) and Kuzminska, Olena (orcid.org/0000-0002-8849-9648) (2024) Methodology for teaching development of web-based augmented reality with integrated machine learning models Proceedings of the 11th Illia O. Teplytskyi Workshop on Computer Simulation in Education (CoSinE 2024) co-located with XVI International Conference on Mathematics, Science and Technology Education (ICon-MaSTEd 2024). Kryvyi Rih, Ukraine, May 15, 2024 (3820). pp. 118-145. ISSN 1613-0073

Text paper249.pdf Download (3MB)

Abstract

Augmented reality (AR) is an emerging technology with many applications in education. Web-based augmented reality (WebAR) provides a cross-platform approach to deliver immersive learning experiences on mobile devices. Integrating machine learning models into WebAR applications can enable advanced interactive effects by responding to user actions. However, little research exists on effective methodologies to teach students WebAR development with integrated machine learning. This paper proposes a methodology with three main steps: (1) Integrating standard TensorFlow.js models like handpose into WebAR scenes for gestures and interactions; (2) Developing custom image classification models with Teachable Machine and exporting to TensorFlow.js; (3) Modifying WebAR applications to load and use exported custom models, displaying model outputs as augmented reality content. The methodology is designed to incrementally introduce machine learning integration, build an understanding of model training and usage, and spark ideas for using machine learning to augment educational content. The methodology provides a starting point for further research into pedagogical frameworks, assessments, and empirical studies on teaching WebAR development with embedded intelligence.

Item Type:	Article
Keywords:	web-based augmented reality, WebAR, machine learning, TensorFlow.js, Teachable Machine, educational technology
Subjects:	Science and knowledge. Organization. Computer science. Information. Documentation. Librarianship. Institutions. Publications > 00 Prolegomena. Fundamentals of knowledge and culture. Propaedeutics > 004 Computer science and technology. Computing. Data processing > 004.9 Application-oriented computer-based techniques Science and knowledge. Organization. Computer science. Information. Documentation. Librarianship. Institutions. Publications > 3 Social Sciences > 37 Education > 37.01/.09 Special auxiliary table for theory, principles, methods and organization of education > 37.02 General questions of didactics and method
Divisions:	Institute for Digitalisation of Education > Department of Open Education and Scientific Information Systems
Depositing User:	Сергій Олексійович Семеріков
Date Deposited:	08 Jul 2026 11:00
Last Modified:	08 Jul 2026 11:00
URI:	https://lib.iitta.gov.ua/id/eprint/749829

Downloads

Actions (login required)

View Item