Where Can We Find the Differences Between Experts and Novices With Lag Sequential Analysis of Spatial Behavioral Patterns in Digital Pentomino Games
Author: Hi-Lian Jeng (Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology), Chung-Nien Chen (APPLE BUDS, LNC.)
Vol.&No.：Vol. 67, No. 4
Date：December 2022
Pages：105-142
DOI：https://doi.org/10.6209/JORIES.202212_67(4).0004
Abstract：
The rapid advancement of technology has led to vigorous growth in research and the applications of digital learning, including digital game-based learning. Digital game-based learning improves learning achievement (Sung & Hwang, 2013; Sung et al., 2015) and learning motivation (Hao & Lee, 2019; Srisawasdi & Panjaburee, 2019), and it enhances higher-order thinking such as critical thinking (Chang et al., 2019; Hussein et al., 2019) and problem-solving (Hwang et al., 2014; Yang, 2015). Digital game-based learning promotes learning in the form of entertainment, and it has thus attracted considerable attention in learning and instruction recently. Advances in technology have also facilitated research designs and analysis methods. Procedural issues that formerly relied on qualitative research methods can now be fully captured and visualized using learning analytics technology, which facilitates the representation, organization, inspection, comparison, and discussion of procedural data.
Procedural data on learning can be used to better explain or predict end performance. Procedural learning analytics can be used to explain learners’ different end performance; and when learners’ end performances are the same, procedural learning analytics may provide a more detailed and refined explanation of their performances. Procedural learning analytics can also provide useful information for optimizing learning designs and environments to improve learning outcomes (Hwang, Chu et al., 2017). Various learning analytics methods aim for different research purposes and designs. Lag sequential analysis is one such method that has been attended in related research.
Although spatial ability is innate and varies among individuals, it can be enhanced through training and learning (Cherney, 2008; Nazareth et al., 2013; Vander Heyden et al., 2017). Spatial ability is related to mathematics capability (Ke, 2019; Krisztián et al., 2015; Ramirez et al., 2012); future attainment in science, technology, engineering, and mathematics (Kell et al., 2013); and future career choices (Jirout & Newcombe, 2015; Uttal & Cohen, 2012). Spatial ability can be improved through digital game-based learning (Hung et al., 2012; Lin & Chen, 2016; Taylor & Hutton, 2013). Pentomino blocks (referred to as Pentomino) constitute an effective material for spatial ability training. Pentomino jigsaw puzzles promote spatial ability (Yang & Chen, 2010). In Pan and Jeng (2018), players applied problem-solving skills that are related to spatial ability during gameplay; different players (experts and novices) applied different problem-solving thinking and strategies. Consequently, their procedural problem-solving skills and strategies also differed. Experts were more systematic in operation and tended to evaluate their outcomes repeatedly, although necessary actions were quickly completed; therefore, the total task time an expert used would be the same as that of a novice. Jeng et al. (2010) combined Thinking Aloud and Pentomino in a spatial performance test for adult participants and observed that for the average number of operations and average operation time, experts and novices were significantly different in the difficult-and-single- solution tasks only but not in the simple-and-multiple-solution tasks.
Research comparing expert and novice problem-solving has primarily evaluated quantitative data. The procedural differences between these two types of players in digital game-based learning require further research (Loh et al., 2016). Only Pan and Jeng (2018) employed Mining Sequential Patterns with Time Constraints to analyze the spatial operation behaviors of experts and novices in the Digital Pentomino Game for adult participants.
On the basis of the procedures described by Pan and Jeng (2018) and Jeng et al.’s (2010) manipulation of the Digital Pentomino Game, the present study applied a novel and more detailed approach to determine differences in spatial performance between experts and novices. This study used a mixed-methods research design. In the first stage, the independent t-test was used to analyze the spatial ability test scores and the average number of operations of each level in the Digital Pentomino Game. In the second stage, the Lag Sequential Analysis was used to analyze and visualize the procedural operation differences between the two groups in each game level. This study explored the following research questions:
1. Are there significant differences between experts and novices in their scores on three spatial ability tests?
2. Is there a significant difference between experts and novices in the average number of operations for each level of the game?
3. Are there significant differences between experts and novices in the sequential procedural analysis for each level of the game?
This study adopted the Digital Pentomino Game system developed by Pan and Jeng (2018). The game contains six levels. The first to fifth levels involve tasks of two-piece Pentomino combinations, and the sixth level involves a task of three-piece Pentomino combinations. Each level contains single or multiple solutions. The three spatial ability tests used are outlined as follows:
1. Jeng and Li (2014) Computerized Mental Rotation Test
2. Jeng and Liu (2016) Computerized Mental Rotation Test
3. Jeng and Chen (2013) paper-and-pencil standardized spatial ability test
The study participants were 47 fourth- and fifth-grade children (aged between 10 and 11 years). This age range is a critical period for the development of children’s spatial abilities, and it is also a critical period for the emergence of gender spatial differences. With advancements in science, technology, and education, and changes in children nurturing in recent years, the stable age at which children can undertake computerized measures of mental rotation ability (one of the factors of spatial ability) in geometric cubic form is as young as 10 years, in contrast to 13 years as reported by earlier studies.
The study results revealed that the expert group performed significantly better than the novice group on the three spatial ability tests. The expert group had a lower average number of operations per solution in each game level than did the novice group, but the differences were significant only in difficult levels and not in simple levels. The experts and novices exhibited different sequential behavioral patterns in solving every difficult level and simple level as well. The experts continually monitored and evaluated their problem-solving procedures and made quick and appropriate corrections when necessary, thereby reducing the number of operations and improving problem-solving efficiency. This implies that in training programs aimed at cultivating novices into experts, novices must be trained to think systematically so that they can develop the ability to continually monitor task performance and environmental contexts when evaluating solutions. Specifically, novices should be trained to acquire expert-like thinking and strategies so that they can ultimately perform as experts or close to experts.
The results of this study provide design suggestions for related applications and research in teaching intervention, game-based learning at the critical stage of spatial ability development, digital content design, learning analytics methods, and variables of investigative interests in the spatial field and any other fields that involve cultivating novices into experts.
Keywords：artificial intelligence in education, experts and novices, game-based learning, lag sequential analysis, digital Pentomino game