Optimization of the structure of test tasks for online training courses based on a probabilistic model

Author(s):

DOI: https://doi.org/10.30929/2307-9770.2022.10.02.03

Paper Language: UKR

Download full-pdf

Abstract

The authors consider the question of substantiation of quantitative parameters of test tasks: the number of questions in the test, the number of test questions of a certain type, the number of answer options in the test task in terms of minimizing the probability of passing the test by random guessing correct answers. The obtained probability models based on the classical definition of probability involving the Bernoulli scheme and the hypergeometric distribution allow to solve the so-called direct and inverse problems. The direct task is to assess the probability of complete or partial passing the test by the method of random guessing of answers. The reverse one is in determining the minimum values of the above quantitative test parameters in terms of minimizing the probability of passing the test by guessing the correct answers. The study showed that the creation of tests with a large number of questions in terms of reducing the likelihood of passing it by guessing the correct answers is not appropriate, because it is enough, for example, to contain mixed questions in equal proportions totaling no more than ten, as at the level of individual topics and in the final tests. This approach makes it possible to significantly save the resource of the database of test questions and use it optimally. It should be noted that to build the model, the authors relied on the probabilistic approach, which is based on the deductive paradigm, and not on simulation, which is based on the inductive approach, which allowed to obtain accurate rather than approximate estimates of model parameters. The proposed models do not cover all existing types of test tasks, but the idea embedded in them can be developed for other, more complex, cases.

Keywords

probabilistic model, deductive approach, optimization of test tasks structure, distance learning

References

  1. Hwang, Dae-Yeop. (2003). Classical Test Theory and Item Response Theory: Analytical and Empirical Comparisons.
  2. Mudruk, S. (2014). Praktychnyi posibnyk dlia rozrobky testovykh zavdan. URL: https://newjustice.org.ua/wp-content/uploads/2018/05/Manual_for_test_writers.pdf (дата звернення 28.06.2022) [In Russian]
  3. Chornyi, О.P., Serhiienko, S.A. (2017). A virtual complex with the parametric adjustment to electromechanical system parameters. Institute of Electrodynamics National Academy of Science of Ukraine. Issue No 1, 2019 (January/February) Pages 38 – 41 https://doi.org/10.15407/techned2019.01.038
  4. Geroimenko, V. (2020). Augmented reality in Education. A new technology for teaching and learning.. Springer Nature Switzerland AG 2020. DOI:10.1007/978-3-030-42156-4
  5. Chornyi O., Herasymenko L. Tytiuk V., Bigdan M., Busher V. Visualisation of the Maturity of Future Electrical Engineers Professional Competencies. Proceedings of the IEEE 20th International Conference on Modern Electrical and Energy System (MEES), MEES 2021 (2021) 254-259. DOI: 10.1109/MEES52427.2021.9598577
  6. The analysis of the process of the laboratory practicum fulfillment and the assessment of its efficiency on the basis of the distance function Chornyi, O., Serhiienko, S., Yudyna, A., Sydorenko, V. Proceedings of the International Conference on Modern Electrical and Energy Systems, MEES 2017, 2017, 2018-January, pp. 328–331
  7. Chornyi O., Herasymenko L., Zelenska L. Diagnostic Assessment Of The Competency Maturity Of Electrical Engineers Through Profession-oriented problems. Proceedings of the International Conference on 2020 IEEE 25th International Conference on Problems of Automated Electrodrive. Theory and Practice (PAEP), PAEP 2020 (2020) 423-426. DOI: 10.1109/PAEP49887.2020.9240864
  8. Chornyi O., Herasymenko L., Zelenska L. Diagnostic Assessment Of The Competency Maturity Of Electrical Engineers Through Profession-oriented problems. Proceedings of the International Conference on 2020 IEEE 25th International Conference on Problems of Automated Electrodrive. Theory and Practice (PAEP), PAEP 2020 (2020) 423-426. DOI: 10.1109/PAEP49887.2020.9240864
  9. Chornyi, O.P., Herasymenko, L.V., Busher, V.V. The learning process simulation based on differential equations of fractional orders. CEUR Workshop Proceedings, 8th Workshop on Cloud Technologies in Education, CTE 2020, 2879, pp. 473–483. DOI: ISSN: 16130073
  10. 10. Gmurman V. Teoriya veroyatnostej i matematicheskaya statistika: Ucheb. Posobie dlya vuzov. – 10-e izd., ster. – Moskva, Vyssh. shk., 2004. 479 с. [In Russian]
  11. Hwang, Dae-Yeop. (2003). Classical Test Theory and Item Response Theory: Analytical and Empirical Comparisons.
  12. Anderson, Lorin W.; Krathwohl, David R., ред. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman. ISBN 978-0-8013-1903-7.
  13. Guchenko, M., Sydorenko, V., Belska, V., Liutenko, M., Fesenko, N. (2021). DComFra Project Learning Module M20 Advanced Spreadsheets in Mathematical Modeling Tasks of Electrical and Computer Engineers Education. Proceedings of the 20th IEEE International Conference on Modern Electrical and Energy Systems, MEES 2021, 2021
  14. Shikhnabieva, T.Sh. (2009). Methodological basis for the submission and control of knowledge in the field of computer science using an adaptive semantic models. Diss… d-RA PED. Sciences. M., p. 365
  15. Shikhnabieva, T.Sh. (2012). On the development of modern educational systems. Information environment education and science. Information environment education and science – 2012, No. 10 (2012).