Learner Perceptions of AI-Powered Learning Portfolios and Personalized Material Recommendation Mechanisms in Reinforcement Learning Algorithms Author: Jian-Wei Tzeng(Department of Information Management, National Taichung University of Science and Technology), Tien-Chi Huang (Department of Computer Science, National Taichung University of Science and Technology), Cheng-Yu Hsueh (Department of Information Management, National Taichung University of Science and Technology), Ying-Song Liao (Department of Information Management, National Taichung University of Science and Technology)

Vol.&No.：Vol. 69, No. 3
Date：September 2024
Pages：73-96
DOI：https://doi.org/10.6209/JORIES.202409_69(3).0003

Abstract：
The COVID-19 pandemic necessitated alternative pedagogical approaches, with online autonomous learning courses emerging as a viable method for compiling learning portfolios. Consequently, online autonomous learning has garnered increasing scholarly attention. Embodying principles of openness and transcending temporal and spatial constraints, online courses afforded global learners opportunities for continued education during the pandemic. Online courses facilitate enhanced online interaction among students and teachers and allow students to control their learning experience (learner autonomy) and pace. Nevertheless, online autonomous learning presents fundamental challenges. Notably, in the absence of direct teacher and teaching assistant supervision, online autonomous learning tends to lead to lower completion rates and higher dropout rates, concerns currently under investigation by numerous researchers. In contrast to traditional teacher-centered models, online autonomous learning courses prioritize self-directed learning. Learners independently establish learning objectives and strategies commensurate with their personal learning levels to master course content. Through a series of instructional videos, in-class exercises, discussion forums, and other interactive features, an appropriate self-regulated learning mechanism was developed to guide learners toward effective autonomous learning.
The exponential growth of big data in recent years has positioned artificial intelligence as a focal point of inquiry across various fields. Machine learning has catalyzed substantial advancements in the field of data science. The accumulation of extensive learning generates substantial volumes of structured and unstructured data, including the personal information of learners and various learning metrics. A growing body of research advocates for the use of data analytics as a viable method to optimize online and adaptive learning processes.
Learning diagnosis entail learners’ self-assessment of requisite capabilities for learning tasks and comparative analyses of capabilities against domain expert-established concept structures by employing relevant question parameters, such as difficulty and discrimination. To facilitate this, an automated artificial intelligence material recommendation mechanism was developed, underpinned by several machine learning models. By observing online user learning behavior patterns, learning data and indicators were formulated, enabling the analysis of various online learning behaviors (e.g., watching videos and answering practice questions) and the generation of learning processes that can be viewed by learners. A practice question recommendation mechanism combined with an instant messaging application (LINE) was designed, leveraging teacher-created knowledge maps to assess students’ mastery of concepts. Zimmerman’s cyclical model of self-regulation served as the foundational framework for the recommendation mechanism.
A quasiexperimental research design was employed. Participants were recruited from a calculus course taught at a university in northern Taiwan. An experimental group used reinforcement learning–recommended practice questions for self-evaluation, and a control group received randomly assigned questions. Significant improvements in scores were observed in the experimental group, and greater learning stickiness was observed compared with the control group. Consistent percentile rank increases following practice question completion suggest the system’s capacity to deliver personalized recommendations on the basis of individual differences, thereby facilitating concept-specific feedback and adaptive learning. This, in turn, fostered increased teacher–student interaction, mitigated learner isolation, and increased learning motivation, thereby strengthening self-regulated learning abilities.
Upon course completion, the participants could autonomously generate artificial intelligence learning portfolios through the system on the basis of diagnostic results, creating a comprehensive record of their learning performance. These portfolios facilitated the elucidation of learner mastery levels through the accumulation of extensive learning data (big data) on the platform. A postcourse self-regulated learning questionnaire survey revealed a positive participant perception of the material recommendation mechanism and generated artificial intelligence learning portfolio. The participants demonstrated strong positive attitudes toward system reliability, learning attitudes, and metacognition but low perceptions of system utility, and low overall usage rates. Enhancing usage incentive, continuously refining the accuracy of the recommendation system’s algorithms, and conducting comparative analyses with existing systems are essential to improve the recommendation system’s perceived utility

Keywords：AI learning portfolio, self-regulated learning, reinforcement learning, educational big data, learning diagnosis

《Full Text》