Evaluating the Effectiveness of AI Integration in Educational Research: Criteria and Indicators

- Спірін, О.М. (orcid.org/0000-0002-9594-6602), Олексюк, Василь Петрович (orcid.org/0000-0003-2206-8447), Осадчий, В. В. (orcid.org/0000-0001-5659-4774), Вакалюк, Тетяна Анатоліївна (orcid.org/0000-0001-6825-4697) and Семеріков, С.О. (orcid.org/0000-0003-0789-0272) (2026) Evaluating the Effectiveness of AI Integration in Educational Research: Criteria and Indicators CEUR Workshop Proceedings. ISSN 1613-0073 (Submitted)

Text Criteria_work.pdf - Draft Version Restricted to Repository staff only until 31 December 2026. Download (1MB)

Abstract

The rapid spread of artificial intelligence (AI) in educational research has brought to the fore the problem of evaluating the effectiveness of integration AI in educational research. Existing approaches are predominantly focused on productivity indicators and technical aspects; however, they insufficiently account for the specific nature of educational research, as well as issues of epistemic reliability, transparency, ethics, and human–AI interaction. The aim of this article is to develop and substantiate a system of criteria and indicators for evaluating the effectiveness of AI integration in educational research. To achieve this aim, the study involves: (1) conducting a systematic analysis of the scientific literature over the last ten years on the evaluation of AI use, including generative AI and large language models, in educational research; and (2) developing a structured system of criteria and corresponding indicators for assessing AI integration in educational research. The research methodology is based on a systematic analysis of the scientific literature published between 2016 and 2026, a bibliometric analysis of publications indexed in the Scopus and Web of Science databases, a contextual analysis of approaches to evaluating AI use, and a theoretical synthesis of contemporary concepts of digital transformation, socio-technical systems, human–AI interaction, epistemology of science, responsible AI, and research evaluation. As a result, the study develops a system of criteria and indicators for evaluating AI integration in educational research. The system comprises seven criteria and twenty-two indicators and covers the methodological justification of AI integration, the process alignment of AI use with the stages of educational research, research productivity and resource efficiency, the quality of human–AI interaction, epistemic reliability and evidential validity of results, ethical, legal, and academic responsibility, and institutional capacity for sustainable AI integration. For each indicator, evaluation descriptors and key sources of evidence are proposed. The findings provide a theoretically grounded basis for the comprehensive evaluation of AI integration in educational research and may be used for the further development of a practical assessment methodology and expert validation of the proposed system.

Item Type:	Article
Keywords:	generative artificial intelligence, evaluation of AI integration, criteria, indicators, educational research
Subjects:	Science and knowledge. Organization. Computer science. Information. Documentation. Librarianship. Institutions. Publications > 00 Prolegomena. Fundamentals of knowledge and culture. Propaedeutics > 001 Science and knowledge in general. Organization of intellectual work > 001.8 Methodology > 001.89 Organization of science and scientific work Science and knowledge. Organization. Computer science. Information. Documentation. Librarianship. Institutions. Publications > 00 Prolegomena. Fundamentals of knowledge and culture. Propaedeutics > 004 Computer science and technology. Computing. Data processing > 004.5 Human-computer interaction. Man-machine interface. User interface. User environment
Divisions:	Institute for Digitalisation of Education > Department of Open Education and Scientific Information Systems
Depositing User:	проф. Василь Петрович Олексюк
Date Deposited:	19 Sep 2026 11:58
Last Modified:	19 Sep 2026 11:58
URI:	https://lib.iitta.gov.ua/id/eprint/750331

Downloads

Actions (login required)

View Item