Predicting Physics Students’ Achievement Using In-Class Assessment Data: A Comparison of Two Machine Learning Models

Purwoko Haryadi Santoso*    -  Universitas Negeri Yogyakarta, Indonesia
Hayang Sugeng Santosa  -  Universitas Muhammadiyah Prof. Dr. HAMKA, Indonesia
Edi Istiyono  -  Universitas Negeri Yogyakarta, Indonesia
Haryanto Haryanto  -  Universitas Negeri Yogyakarta, Indonesia
Heri Retnawati  -  Universitas Negeri Yogyakarta, Indonesia

(*) Corresponding Author
DOI: 10.21580/perj.2023.5.2.14217
Data is the primary source to scaffold physics teaching and learning for teachers and students, mainly reported through in-class assessment. Machine learning (ML) is an axis of artificial intelligence (AI) study that immensely attracts the development of physics education research (PER). ML is built to predict students’ learning that can support students’ success in an effective physics achievement. In this paper, two ML algorithms, logistic regression and random forest, were trained and compared to predict students’ achievement in high school physics (N = 197). Data on students’ achievement was harvested from in-class assessments administered by a physics teacher regarding knowledge (cognitive) and psychomotor during the 2020/2021 academic year. Three assessment points of knowledge and psychomotor were employed to predict students’ achievement on a dichotomous scale on the final term examination. Combining in-class assessment of knowledge and psychomotor, we could discover the plausible performance of students’ achievement prediction using the two algorithms. Knowledge assessment was a determinant in predicting high school physics students’ achievement. Findings reported by this paper recommended open room for the implementation of ML for educational practice and its potential contribution to supporting physics teaching and learning.

Keywords: Achievement prediction; assessment; machine learning; physics education; students

  1. Aikenhead, G. S. (2023). Humanistic school science: Research, policy, politics and classrooms. Science Education, 107(2), 237–260. https://doi.org/https://doi.org/10.1002/sce.21774
  2. Albreiki, B., Zaki, N., & Alashwal, H. (2021). A Systematic Literature Review of Student’ Performance Prediction Using Machine Learning Techniques. Education Sciences, 11(9). https://doi.org/10.3390/educsci11090552
  3. Atmam, P. L., & Mufit, F. (2023). Using Adobe Animated CC in Designing Interactive Multimedia Based on Cognitive Conflict on Parabolic Motion Materials. 8(1), 64–74.
  4. Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01
  5. Bloor, T., & Santini, J. (2023). Modeling the Epistemic Value of Classroom Practice in the Investigation of Effective Learning. Science & Education, 32(1), 169–197. https://doi.org/10.1007/s11191-021-00298-9
  6. Chapman, P. (2000). CRISP-DM 1.0: Step-by-step data mining guide. https://api.semanticscholar.org/CorpusID:59777418
  7. Chen, J., Wang, M., Grotzer, T., & Dede, C. (2018). Using a three-dimensional thinking graph to support inquiry learning. Journal of Research in Science Teaching, 55. https://doi.org/10.1002/tea.21450
  8. Darling-Hammond, L., Flook, L., Cook-Harvey, C., Barron, B., & Osher, D. (2020). Implications for educational practice of the science of learning and development. Applied Developmental Science, 24(2), 97–140. https://doi.org/10.1080/10888691.2018.1537791
  9. Domínguez Romero, E., & Bobkina, J. (2021). Exploring critical and visual literacy needs in digital learning environments: The use of memes in the EFL/ESL university classroom. Thinking Skills and Creativity, 40, 100783. https://doi.org/https://doi.org/10.1016/j.tsc.2020.100783
  10. Fynn, A., Adamiak, J., & Young, K. (2022). A Global South Perspective on Learning Analytics in an Open Distance E-learning (ODeL) Institution. In P. Prinsloo, S. Slade, & M. Khalil (Eds.), Learning Analytics in Open and Distributed Learning: Potential and Challenges (pp. 31–45). Springer Nature Singapore. https://doi.org/10.1007/978-981-19-0786-9_3
  11. Hochberg, K., Kuhn, J., & Müller, A. (2018). Using Smartphones as Experimental Tools – Effects on Interest, Curiosity, and Learning in Physics Education. Journal of Science Education and Technology, 27. https://doi.org/10.1007/s10956-018-9731-7
  12. Jeong, J. S., González-Gómez, D., & Cañada-Cañada, F. (2021). How does a flipped classroom course affect the affective domain toward science course? Interactive Learning Environments, 29(5), 707–719. https://doi.org/10.1080/10494820.2019.1636079
  13. Kind, P. M. (2013). Establishing Assessment Scales Using a Novel Disciplinary Rationale for Scientific Reasoning. Journal of Research in Science Teaching, 50(5), 530–560.
  14. Le, B., Lawrie, G. A., & Wang, J. T. H. (2022). Student Self-perception on Digital Literacy in STEM Blended Learning Environments. Journal of Science Education and Technology, 31(3), 303–321. https://doi.org/10.1007/s10956-022-09956-1
  15. Leitner, P., Khalil, M., & Ebner, M. (2017). Learning Analytics in Higher Education---A Literature Review. In A. Peña-Ayala (Ed.), Learning Analytics: Fundaments, Applications, and Trends: A View of the Current State of the Art to Enhance e-Learning (pp. 1–23). Springer International Publishing. https://doi.org/10.1007/978-3-319-52977-6_1
  16. Lin, R., Yang, J., Jiang, F., & Li, J. (2023). Does teacher’s data literacy and digital teaching competence influence empowering students in the classroom? Evidence from China. Education and Information Technologies, 28(3), 2845–2867. https://doi.org/10.1007/s10639-022-11274-3
  17. Lu, K., Yang, H. H., Shi, Y., & Wang, X. (2021). Examining the key influencing factors on college students’ higher-order thinking skills in the smart classroom environment. International Journal of Educational Technology in Higher Education, 18, 1. https://doi.org/10.1186/s41239-020-00238-7
  18. Luo, M., Sun, D., Zhu, L., & Yang, Y. (2021). Evaluating scientific reasoning ability: Student performance and the interaction effects between grade level, gender, and academic achievement level. Thinking Skills and Creativity, 41, 100899. https://doi.org/https://doi.org/10.1016/j.tsc.2021.100899
  19. Narvaez Rojas, C., Alomia Peñafiel, G. A., Loaiza Buitrago, D. F., & Tavera Romero, C. A. (2021). Society 5.0: A Japanese Concept for a Superintelligent Society. Sustainability, 13(12). https://doi.org/10.3390/su13126567
  20. Ndoa, Y. A. A., & Anastasia, D. P. (2022). Development of An Android-Based Physics E-Book with A Scientific Approach to Improve The Learning Outcomes of Class X High School Students on Impulse and Momentum Materials. 18(December), 107–121. https://doi.org/10.15294/jpfi.v18i2.30824
  21. Romero, C., & Ventura, S. (2020). Educational data mining and learning analytics: An updated survey. WIREs Data Mining and Knowledge Discovery, 10(3), e1355. https://doi.org/https://doi.org/10.1002/widm.1355
  22. Rubie-Davies, C. M. (2006). Teacher expectations and student self-perceptions: Exploring relationships. Psychology in the Schools, 43(5), 537–552. https://doi.org/https://doi.org/10.1002/pits.20169
  23. Santoso, P. H., Istiyono, E., & Haryanto. (2022). Physics Teachers’ Perceptions about Their Judgments within Differentiated Learning Environments: A Case for the Implementation of Technology. Education Sciences, 12(9). https://doi.org/10.3390/educsci12090582
  24. Santoso, P. H., & Munawanto, N. (2020). Approaching Electrical Circuit Understanding with Circuit Builder Virtual Laboratory. Jurnal Ilmiah Pendidikan Fisika Al-Biruni, 9(2), 259–269. https://doi.org/10.24042/jipfalbiruni.v9i2.5976
  25. Sari, D., Herlina, K., Viyanti, V., Andra, D., & Safitri, I. (2023). E-module Newton's Law of Gravity based Guided Inquiry to Train Critical Thinking Skills. Physics Education Research Journal, 5(1), 13-20. doi:https://doi.org/10.21580/perj.2023.5.1.11657
  26. Sekeroglu, B., Abiyev, R., Ilhan, A., Arslan, M., & Idoko, J. B. (2021). Systematic Literature Review on Machine Learning and Student Performance Prediction: Critical Gaps and Possible Remedies. Applied Sciences, 11(22). https://doi.org/10.3390/app112210907
  27. Semenikhina, O., Yurchenko, A., & Udovychenko, O. (2020). Formation of skills to visualize of future physics teacher: results of the pedagogical experiment. Physical and Mathematical Education, 23(1), 122–128. https://doi.org/10.31110/2413-1571-2020-023-1-020
  28. Shafiq, D. A., Marjani, M., Habeeb, R. A. A., & Asirvatham, D. (2022). Student Retention Using Educational Data Mining and Predictive Analytics: A Systematic Literature Review. IEEE Access, 10, 72480–72503. https://doi.org/10.1109/ACCESS.2022.3188767
  29. Shengnan Wang, C. M. R.-D., & Meissel, K. (2018). A systematic review of the teacher expectation literature over the past 30 years. Educational Research and Evaluation, 24(3–5), 124–179. https://doi.org/10.1080/13803611.2018.1548798
  30. Starr, C., Hunter, L., Dunkin, R., Honig, S., Palomino, R., & Leaper, C. (2020). Engaging in science practices in classrooms predicts increases in undergraduates’ STEM motivation, identity, and achievement: A short-term longitudinal study. Journal of Research in Science Teaching, 57. https://doi.org/10.1002/tea.21623
  31. Susilawati, S., Azizah, N. A. N., & Kusuma, H. H. (2021). Investigating differences in project activities and student digital literacy between learning through electronic workbench and PhET Simulation. Jurnal Ilmiah Pendidikan Fisika Al-Biruni, 10(2), 299–311. https://doi.org/10.24042/jipfalbiruni.v10i2.10008
  32. Vasalou, A., Benton, L., Ibrahim, S., Sumner, E., Joye, N., & Herbert, E. (2021). Do children with reading difficulties benefit from instructional game supports? Exploring children’s attention and understanding of feedback. British Journal of Educational Technology, 52(6), 2359–2373. https://doi.org/https://doi.org/10.1111/bjet.13145
  33. Yang, C., Lan, S., Shen, W., Huang, G. Q., Wang, X., & Lin, T. (2017). Towards product customization and personalization in IoT-enabled cloud manufacturing. Cluster Computing, 20. https://doi.org/10.1007/s10586-017-0767-x
  34. Zimmerman, L., Spillane, S., Reiff, P., & Sumners, C. (2014). Comparison of Student Learning about Space in Immersive and Computer Environments. Journal Review Astronomical Education and Outreach, 1(1), A5–A20.

Open Access Copyright (c) 2023 Physics Education Research Journal
Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

 

 
apps