- Дудко, А.Ф. (orcid.org/0000-0003-3858-7313) (2019) Computer-oriented methodology of assessing the quality of tests in higher mathematics by teachers of higher education establishments Doctoral thesis, Національний технічний університет України «Київський політехнічний інститут імені Ігоря Сікорського».
![[thumbnail of Дисертація Дудко А.Ф.]](https://lib.iitta.gov.ua/style/images/fileicons/text.png) |
Text (Дисертація Дудко А.Ф.)
diss Дудко А.Ф..pdf Download (11MB) |
|
Text (Відгук Триуса Ю.В. на дисертацію Дудко А.В.)
Відгук_Триус Ю.В..pdf Download (455kB) |
|
Text (Відгук Кислової М.А на дисертацію Дудко А.Ф.)
Відгук Кислової М.А..pdf Download (5MB) |
Abstract
The thesis for a Candidate Degree in Pedagogical Sciences (Philosophy Doctor) specialty 13.00.10. “Information and communication technology in education” (011 – Educational, pedagogical science). - National Technical University of Ukraine " Igor Sikorsky Kyiv Polytechnic Institute", Institute of Information Technologies and Learning Tools NAES of Ukraine, Kyiv, 2019. The paper studies the problem of assessing the quality of tests in higher mathematics by teachers of higher education establishments. The international and domestic experience on the research issue is analyzed and the concepts of "test", "test item", "quality of the test", "test quality assessment", "computer-oriented methodology of assessing the quality of tests" are specified. The necessity of combining methods of Classical Test Theory and Item Response Theory for assessing the quality of tests in higher mathematics is substantiated. The basic methods for assessing the quality of tests in higher mathematics in the context of CTT are determined. They include analysis of individual test items and the test in general. The test analysis consists in analysis of distribution of the results’ sample, estimation of the test reliability, validity and efficiency. The analysis of test items consists in analysis of the correlation matrix of items, analysis of the ensemble of characteristic curves of items, the analysis of polytomous items, the analysis of the distractors of the multiple choice items. The IRT-models for assessing the quality of tests in higher mathematics are determined: Rash and Birnbaum for dichotomous items, Andersen for polytomous items and Thyssen-Steinberg for multiple choice items. The IRT analysis of test items and the test in general is realized with the use of determined models and includes: evaluation of the latent parameters of IRT-models; analysis of the ensemble of characteristic curves of the test items; analysis of characteristic curves of levels of political test items; analysis of characteristic curves of distractors of multiple choice items. The issue of estimating of latent parameters of IRT-models is studied. The maximum likelihood is chosen for estimating of latent parameters. Its application is demonstrated by the example of the partial credit model. The following problems of practical parameter estimation of IRT-models are solved: the choice of initial values; the shift of estimated parameter; the presence of trivial levels of polytomous test items. The author's computer-based system of quality analysis of test items is developed. The fundamental functionality of this system: flexible choice of algorithm of statistical analysis of test results; visualization of graphical presentation of results; generation and support of the base of calibrated items. Statistical analysis of the system is based on the methods of CTT and IRT. The computer-oriented methodology of assessing the quality of the tests in higher mathematics by teachers of higher education establishments is developed. The purpose of assessing the quality of the tests is to improve the control of students' knowledge of higher mathematics. The main means of assessing the quality of tests is computer-based system of quality analysis of test items. Methods of assessing the quality of tests in higher mathematics are selected methods of probability theory and mathematical statistics, namely, methods of CTT and IRT. The general scheme reflects the content component of assessing. It includes the following steps: generation of test matrix; analysis of test and individual test items; final decision making. The generation of test matrix is the data preparation for follow-up analysis by computer-based system of quality analysis of test items. The analysis of the test in general includes: analysis of distribution of the results’ sample, estimation of the test reliability, validity and efficiency. Analysis of test items includes: analysis of the correlation matrix of items, analysis of the ensemble of characteristic curves of items, the analysis of polytomous items, and the analysis of the distractors of the multiple choice items. The final decisions are made on the basis of a consistent analysis of the complementary characteristics of IRT and CTT. At the same time, conclusions on the quality of both individual test items and the test in general are drawn. It is expedient to realize the introduction of a computer-based methodology of assessing the quality of tests in higher mathematics step-by-step: the development of teachers' competence in assessing the quality of tests and the application of a computer-oriented methodology for assessing the quality of tests in higher mathematics by teachers in their professional activities. It is expedient to realize the use of computer-oriented methodology for assessing the quality of tests on higher mathematics by teachers in accordance with the general scheme of the teacher's work with tests in higher mathematics. This scheme provides: solution of the problem of determining the approach to creation of tests; creation the test; estimation of content validity; test loading into the computer test system (Moodle); trial test administration; importation of test results and quality assessment of tests based on the developed methodology; decision making about the test quality and further actions with it; retention of test and test items together with their statistical characteristics in the base of calibrated items. The use of tests requires high level teachers’ competence concerning the quality of tests in higher mathematics. The teacher's competence concerning the quality of tests in higher mathematics is defined as teacher’s readiness and ability to calculate the characteristics of the test, test items with the use of ICT and on the basis of these characteristics to evaluate the quality of single item and objectively draw conclusions about the quality of the test, its improvement and the expediency of using in the learning process for knowledge control of higher mathematics. The basic criteria of teachers’ competence concerning the quality of tests in higher mathematics are defined and described: value-motivational, cognitive, and operational-activity. The characteristics of the defined criteria for basic, advanced and professional levels are presented. The development model of teachers' competence concerning the quality of tests in higher mathematics is proposed. It consists of the following components: motivational, content, technological, diagnostic and resultative. The development methodology of teachers' competence concerning the quality of tests in higher mathematics is based on this model. The general results of the pedagogical experiment gives evidences to confirm that the introduction of the computer-oriented methodology for assessing the quality of tests in higher mathematics will promote the development of teachers' competence concerning the quality of tests in higher mathematics and improvment of knowledge control. The result of introduction of the developed computer-oriented methodology of assessing the quality of tests in higher mathematics by teachers of higher education establishments confirms its effectiveness. Scientific novelty and theoretical meaning of the research: for the first time the computer-oriented methodology of assessing the quality of tests in higher mathematics by teachers of higher education establishments has been substantiated based on interaction of the methods of KTT and IRT; the criteria and levels of teachers’ competence concerning the quality of tests in higher mathematics has been defined; the development model of teachers' competence concerning the quality of tests in higher mathematics has been developed; the concepts of test, test item, quality of the test, test quality assessment and computer-oriented methodology of assessing the quality of tests have been specified; methods of test quality improvement and methods of estimation of latent parameters of test items have been improved; future developments of theoretical and methodical foundations of the creating and using of computer-based systems and tutorials, theoretical and methodical problems of development and use of information and communication technologies for measurement and evaluation of academic performance and monitoring of educational activities, the theory and methods of training, retraining and advanced training of informatization of education have been suggested. Practical meaning of the received results: the main components of the computer-oriented methodology of assessing the quality of tests in higher mathematics by teachers of higher education establishments has been developed; the computer-based system of quality analysis of test items has been developed; the development thechnology of teachers' competence concerning the quality of tests in higher mathematics has been developed; the curriculum "Development of the teachers’ competence concerning the quality of tests in higher mathematics" for the advanced training course of scientific, scientific and pedagogical staff has been developed and placed on the site of the Educational methodical complex "Institute of Postgraduate Education" of Igor Sikorsky Kyiv Polytechnic Institute (http://ipo.kpi.ua/ua/pidvishchennja-kvalifikaciji.html); the distance course "Development of the teachers’ competence concerning the quality of tests in higher mathematics" has been developed and placed on the site of the Educational methodical complex "Institute of Postgraduate Education" of Igor Sikorsky Kyiv Polytechnic Institute (http://ipo.kpi.ua/ua/pidvishchennja-kvalifikaciji.html); the base of calibrated items in higher mathematics (1647 items); the test package in higher mathematics for students of technical specialties higher education establishments, used by teachers of Faculty of Physics and Mathematics of Igor Sikorsky Kyiv Polytechnic Institute, has been developed. The materials and the results of the research can be used by teachers of mathematics in higher education establishments for analysis and improvement of the quality of tests in higher mathematics, in the system of advanced training of scientific, pedagogical and scientific staff for development of teachers' competence concerning the quality of tests in higher mathematics and in the process of developing educational and methodological manuals for the assessment of the quality of tests in higher mathematics. The research does not exhaust the completeness of the problem and can be proceeded in the following areas of scientific research: development and introduction of the computer-oriented methodology of assessing the quality of tests of other branches of science; research of innovative methods of IRT for assessing the quality of tests and improving the developed methodology
Downloads
Downloads per month over past year
Actions (login required)
 |
View Item |