Towards Problematic Design Thinking : Guiding Meaning-driven Curriculum Evaluation Author: Mei-Ju Chen (Department of Education and Learning Technology, National Tsing Hua University) , Pei-Ying Chen (Department of Education, National Taiwan Normal University), Ya-Hui Chang (Department of Education and Learning Technology, National Tsing Hua University) , Chao-Yu Guo (Department of Education, National Chengchi University) , Yuan-Sin Liao (Department of Education and Learning Technology, National Tsing Hua University) 

Vol.&No.：Vol. 69, No. 1
Date：March 2024
Pages：207-239
DOI：https://doi.org/10.6209/JORIES.202403_69(1).0007

Abstract：
Motivation and Purpose
As Taiwan enters the third decade of the twenty-first century, its educational systems must be well-equipped for the future. However, Taiwan’s seemingly rigorous and concrete curriculum evaluations as evidenced by past experience, have shown its limitations. Without a solid foundation in the needs and challenges of educators and the educated, curriculum evaluations become mere formalities incapable of driving self-reflection and improvement. Additionally, such evaluations must prompt critical inquiry into the following questions: “Why do we need curriculum evaluations?” “Why adopt one system of curriculum evaluation over another?” and “What do curriculum evaluations mean to us?” This study sought to utilize methodological and epistemological insights to ground curriculum evaluations in the context of the problems they seek to address; this allows us to expand our horizons, cross boundaries, and, through problem-oriented design thinking, make educators more aware of the systems and groups of which they are a part. Such reflection on curriculum evaluations prompted the creation of a series of meaningful curriculum evaluation measures and courses.
Literature Review
Epistemological reflections informed the present study’s multidisciplinary approach to curriculum evaluation; problem-oriented design thinking served as the engine for this study’s guiding design; and expansive learning and U-theory provided reference strategies, tools, and learning environment support.
Research Questions
The research questions this study sought to answer are as follows:
1. How does the problematization process influence the design of meaning-driven curriculum evaluations?
2. What key themes emerge during the problematization process within teams conducting meaning-driven curriculum evaluations?
3. How do various types of questioning facilitate design thinking in problematization for meaning-driven curriculum evaluation?
4. Through the lens of problematization in design thinking, what innovative guiding designs are developed for conducting meaning-driven curriculum evaluations?
Method
This study employed eight focus group sessions, each lasting 1-2 hours, to guide the design of a “Meaning-driven and Expansive Curriculum Evaluation Workshop.” Keywords and question types from the session transcripts were analyzed and coded; they were supplemented by opinion surveys of the workshop participants and their lesson preparation materials.
Results and Discussion
1. Problematization— which is the process of identifying problems to be addressed and their solutions— is clearly discernible in the chronological evolution of the group discussions, which moved from clarifying key points of curriculum evaluation to designing a workshop on guiding evaluation design.
In chronological order, the key questions of the problematization process were as follows.
Early-stage questions were posed to clarify the problems surrounding curriculum evaluation in Taiwan and its relationship to educators. Beginning with the problems inherent in the current traditional evaluation system, participants questioned the meaning and value of curriculum evaluation, forming the workshop’s goals. Late-stage questions were designed to transform the meaning and goals that emerged during the discussions and incorporate them into the design of the workshop courses. The subsequent process of problematization emphasized creative collaboration and explored more fundamental problems, such as: “How can curriculum evaluation help educators improve their motivation and skills?” and “What must be done next?”
2. Multiple themes emerged from keyword analysis of problematization content, specifically:
(1) The problematization process involves real problems/needs in the context of the curriculum:
All the questions from the group discussion process were imported into a keyword analysis software. “What,” “how,” and “why” ranked as the top three keywords. The high frequency of these question words indicates that the group discussion focused on inquiry and exploration, which are basic functions of the problematization process. The team evaluated problems in the context of the curriculum, using questions to suggest avenues for experimental improvements.
(2) Reflective and intentional language is essential to the design thinking process:
Hypothetical questions typically inspire an epistemological approach of inquiry and verification, or evaluative thinking; this requires the use of reflective language. The keywords, the context of the educators’ questions, and the language used demonstrate that the participants focused their thinking and decision-making on the usability, appropriateness, and improvability of the proposed curriculum and were thus continually optimizing and evaluating ideas. The discussion leaders emphasized the participants’ intentions in curriculum evaluation during the discussion process. Additionally, the keyword “want” occurred frequently (17 times), underscoring the participants’ focus on changing their behaviors and responding to needs.
(3) A learning community is the operational basis for the problematization process:
The fourth-highest ranked keyword was “we,” whereas “they” was ranked 10th, indicating that the educators were focused on solving problems as a community. This community focus is consistent with U-theory’s approach of creating a social field that enables a team to generate collective wisdom and a sense of direction through exploration and reflection. The questions guided participants to consider the relationship between individuals and the team.
(4) Members know that their ultimate goal is benefiting students, prompting critical reflection on and engagement with the evaluation process:
The goal of any curriculum is to educate students. Curriculum evaluation is a crucial component of curriculum development that requires educators to remain flexible. Because the educators did not use traditional terms such as “evaluation” and “performance” in their questions, their meaning-driven curriculum evaluation emphasized the context of education itself, focusing on teachers and students and responding to their needs in the process of meeting the evaluation objectives.
3. Questions serve several functions in guiding the design thinking underpinning the problematization process:
Through question analysis, 258 questions were categorized into 5 types: (1) Value-oriented questions concerning feelings, relationships, and meanings; (2) assessment-oriented questions using logical explorations in the context of a particular problem to reach a solution; (3) strategy-oriented questions focusing on perceiving problems and developing time-sensitive strategies to address them; (4) empathy-oriented questions promoting collaboration and dialogue; (5) task-oriented questions involving action planning, reflective evaluation, and collaboration.
4. The meaning-driven curriculum evaluation guiding design was generated, and a meaning-driven curriculum evaluation was proposed:
The eight group discussions resulted in a 2-day meaning-driven curriculum evaluation guiding design workshop. The design philosophy emphasized participation and cocreation, where learning evolved continually through questioning and the cultivation of shared learning spaces. The process clarified participants’ perspectives on curriculum evaluation, starting from real problems/needs and moving through individual experiences to community dialogue and cocreation. The workshop was based on participants’ questions and encouraged individual reflection, group collaboration, and team cocreation. The guiding design was dynamic, emphasizing participant interaction and responses, with real-time adjustments made as new problems were identified.
Conclusion
1. The themes and types of questions generated from keyword analysis indicate the feasibility of integrating expansive learning and U-theory to guide understanding in problematization thinking and planning.
2. The act of questioning reveals perspectives and options through mutual understanding and shared strategic vision to create a curriculum evaluation that is sensitive to the real needs of teachers and students.
3. The problematization process focuses on real problems/needs in context. Reflective and intentional language is indispensable to the problematization process. Moreover, a group approach to designing a curriculum evaluation establishes meaningful connections between educators’ thoughts and needs and the curriculum they deliver.
4. A diversity of question types enriches and animates the problematization process.
Recommendations
1. Educators must be encouraged to engage in discourse with an open mind, maintain flexibility, and embrace uncertainty.
2. Design thinking must be integrated into curriculum evaluations.
3. A diversity of questions and opinions must be encouraged in curriculum evaluation.

Keywords：problematization, design thinking, meaning-driven, discourse, curriculum evaluation

《Full Text》