從教師自編科學成就測驗之Rasch分析看教與學
作者:陳映孜(國立中山大學教育研究所)、何曉琪(國立中山大學人文創新與社會實踐研究中心)、劉昆夏(國立中山大學教育研究所)、林煥祥(國立中山大學通識教育中心)、鄭英耀(國立中山大學教育研究所)
卷期：62卷第3期
日期：2017年9月
頁碼：1-23
DOI：10.6209/JORIES.2017.62(3).01
摘要：
本研究採Rasch測量模式，分析教師自編科學成就測驗，以檢驗測驗試題的品質，並提供有用的評量回饋訊息，作為教師調整教學與修正評量之參考。研究對象為高雄市8,936位（男生4,617位、女生4,319位）七年級學生。研究工具為教師自編生物科成就測驗。測驗資料採試題反應理論的Rasch模式進行分析。結果發現：整體而言，教師自編科學成就測驗能區分出學生的能力程度。然而，有些試題的內容對女生較不公平。本研究也發現，教師對於部分試題難易度的認知與學生實際的作答反應存有若干差異。最後，本研究透過評量回饋圖表，包含學生能力與試題難度對照圖、學生作答診斷圖等，提出辨識學習基礎發展區和學習可能發展區的範圍之方式，作為教師規劃學生學習進程之參考。
關鍵詞：Rasch測量、教師自編成就測驗、評量回饋、學習可能發展區、學習進程
《詳全文》
參考文獻：
- 王子華、范雅晴、王國華（2008）。數理科在職教師評量素養培育模式之探究。科學教育學刊，16（1），25-51。doi:10.6173/CJSE.2008.1601.04【Wang, T.-H., Fan, Y.-C., & Wang, K.-H. (2008). Implementation of web-based assessment system in improving in-service teachers’ assessment literacy. Chinese Journal of Science Education, 16(1), 25-51 doi:10.6173/ CJSE.2008.1601.04】
- 教育部（2008）。教育部補助辦理精進教學要點修正規定。行政院公報，014（215）。取自https://gazette.nat.gov.tw/EG_FileManager/eguploadpub/eg014215/ch05/type2/gov40/num12/Eg.pdf【Ministry of Education. (2008). Revised regulations governing Ministry of Education subsidies for instruction improvement. The Executive Yuan Gazatte Online, 014(215). Retrieved from https://gazette.nat.gov.tw/EG_ FileManager/eguploadpub/eg014215/ch05/type2/gov40/num12/Eg.pdf】
- Alonzo, A., & Steedle, J. T. (2009). Developing and assessing a force and motion learning progression. Science Education, 93(3), 389-421. doi:10.1002/sce.20303
- Allal, L., & Ducrey, G. P. (2000). Assessment of-or in-the zone of proximal development. Learning and Instruction, 10(2), 137-152. doi:10.1016/S0959-4752(99)00025-0
- Bell, B. (2007). Classroom assessment of science learning. In S. K. Abell & N. G. Lederman (Eds.), Handbook of research on science education (pp. 965-1006). Mahwah, NJ: LEA.
» 展開更多
- 王子華、范雅晴、王國華（2008）。數理科在職教師評量素養培育模式之探究。科學教育學刊，16（1），25-51。doi:10.6173/CJSE.2008.1601.04【Wang, T.-H., Fan, Y.-C., & Wang, K.-H. (2008). Implementation of web-based assessment system in improving in-service teachers’ assessment literacy. Chinese Journal of Science Education, 16(1), 25-51 doi:10.6173/ CJSE.2008.1601.04】
- 教育部（2008）。教育部補助辦理精進教學要點修正規定。行政院公報，014（215）。取自https://gazette.nat.gov.tw/EG_FileManager/eguploadpub/eg014215/ch05/type2/gov40/num12/Eg.pdf【Ministry of Education. (2008). Revised regulations governing Ministry of Education subsidies for instruction improvement. The Executive Yuan Gazatte Online, 014(215). Retrieved from https://gazette.nat.gov.tw/EG_ FileManager/eguploadpub/eg014215/ch05/type2/gov40/num12/Eg.pdf】
- Alonzo, A., & Steedle, J. T. (2009). Developing and assessing a force and motion learning progression. Science Education, 93(3), 389-421. doi:10.1002/sce.20303
- Allal, L., & Ducrey, G. P. (2000). Assessment of-or in-the zone of proximal development. Learning and Instruction, 10(2), 137-152. doi:10.1016/S0959-4752(99)00025-0
- Bell, B. (2007). Classroom assessment of science learning. In S. K. Abell & N. G. Lederman (Eds.), Handbook of research on science education (pp. 965-1006). Mahwah, NJ: LEA.
- Bell, B., & Cowie, B. (2001). The characteristics of formative assessment in science education. Science Education, 85(5), 536-553. doi:10.1002/sce.1022
- Bertenthal, M. W., & Wilson, M. R. (2006). Systems for state science assessment. Washington, DC: National Academy Press.
- Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2004). Working inside the black box: Assessment for learning in the classroom. Phi Delta Kappan, 86(1), 8-21. doi:10.1177/0031 72170408600105
- Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7-74. doi:10.1080/0969595980050102
- Black, P., & Wiliam, D. (2010). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 92(1), 81-90. doi:10.1177/003172171009200119
- Boone, W. J., & Scantlebury, K. (2006). The role of Rasch analysis when conducting science education research utilizing multiple-choice tests. Science Education, 90(2), 253-269. doi:10. 1002/sce.20106
- Boone, W. J., Townsend, J. S., & Staver, J. (2011). Using Rasch theory to guide the practice of survey development and survey data analysis in science education and to inform science reform efforts: An exemplar utilizing STEBI self-efficacy data. Science Education, 95(2), 258-280. doi:10.1002/sce.20413
- Briggs, D. C., Alonzo, A. C., Schwab, C., & Wilson, M. (2006). Diagnostic assessment with ordered multiple-choice items. Educational Assessment, 11(1), 33-63. doi:10.1207/s15326977ea1101_2
- Budoff, M. (1987). The validity of learning potential assessment. In C. S. Lidz (Ed.), Dynamic assessment: An interactional approach to evaluating learning potential (pp. 53-81). New York, NY: Guilford Press.
- Claesgens, J., Scalise, K., Wilson, M., & Stacy, A. (2009). Mapping student understanding in chemistry: The perspectives on chemists. Science Education, 93(1), 56-85. doi:10.1002/sce. 20292
- Duncan, R. G., & Gotwals, A. W. (2015). A tale of two progressions: On the benefits of careful comparisons. Science Education, 99(3), 410-416. doi:10.1002/sce.21167
- Duncan, R. G., & Hmelo-Silver, C. E. (2009). Learning progressions: Aligning curriculum, instruction, and assessment. Journal for Research in Science Teaching, 46(6), 606-609. doi:10. 1002/tea.20316
- Duncan, R. G., & Reiser, B. J. (2007). Reasoning across ontologically distinct levels: Students’ understandings of molecular genetics. Journal of Research in Science Teaching, 44(7), 938-959. doi:10.1002/tea.20186
- Fulmer, G. W. (2014). Validating proposed learning progressions on force and motion using the force concept inventory: Findings from Singapore secondary schools. International Journal of Science and Mathematics Education, 13(6), 1235-1254. doi:10.1007/s10763-014-9553-x
- Furtak, E. M., Morrison, D., & Kroog, H. (2014). Investigating the link between learning progressions and classroom assessment. Science Education, 98(4), 640-673. doi:10.1002/sce.21122
- Gioka, O. (2009). Teacher or examiner? The tensions between formative and summative assessment in the case of science coursework. Research in Science Education, 39(4), 411-428. doi:10.1007/ s11165-008-9086-9
- Graham, P. (2005). Classroom-based assessment: Changing knowledge and practice through preservice teacher education. Teaching and Teacher Education, 21(6), 607-621. doi:10.1016/ j.tate.2005.05.001
- Griffin, P. (2007). The comfort of competence and the uncertainty of assessment. Studies in Educational Evaluation, 33(1), 87-99. doi:10.1016/j.stueduc.2007.01.007
- Guthke, J. (1993). Developments in learning potential assessment. In J. H. M. Hamers & K. Sijtsma (Eds.), Learning potential assessment: Theoretical, methodological and practical issues (pp. 43-67). Amsterdam, The Netherland: Swets & Zeitlinger.
- Hammer, D., & Sikorski, T. R. (2015). Implications of complexity for research on learning progressions. Science Education, 99(3), 424-431. doi:10.1002/sce.21165
- Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. doi:10.3102/003465430298487
- Holland, P. W., & Wainer, H. (Eds.). (1993). Differential item functioning. Hillsdale, NJ: Lawrence Erlbaum Associates.
- Kennedy, C. A., Wilson, M., & Draney, K. (2008). ConstructMap [Computer Program]. UC Berkeley, CA: BEAR Center.
- Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254-284. doi:10.1037/0033-2909.119.2.254
- Lee, H.-S., & Liu, O. L. (2010). Assessing learning progression of energy concepts across middle school graded: The knowledge integration perspective. Science Education, 94(4), 665-688. doi:10.1002/sce.20382
- Linacre, J. M., & Wright, B. D. (1994). Chi-square fit statistics. Rasch Measurement Transactions, 8(2), 350.
- Lipnevich, A. A., & Smith, J. K. (2009). Effects of differential feedback on students’ examination performance. Journal of Experimental Psychology: Applied, 15(4), 319-333. doi:10.1037/ a0017841
- Maclellan, E. (2004). Initial knowledge states about assessment: Novice teachers’ conceptualisations. Teaching and Teacher Education, 20(5), 523-535. doi:10.1016/j.tate.2004.04.008
- Mertler, C. A. (2004). Secondary teachers’ assessment literacy: Does classroom experience make a difference? American Secondary Education, 33(1), 49-64.
- Mertler, C. A. (2009). Teachers’ assessment knowledge and their perceptions of the impact of classroom assessment professional development. Improving Schools, 12(2), 101-113. doi:10. 1177/1365480209105575
- Mohan, L., Chen, J., & Anderson, C. W. (2009). Developing a multi-year learning progression for carbon cycling in socio-ecological systems. Journal of Research in Science Teaching, 46(6), 675-698. doi:10.1002/tea.20314
- Neumann, I., Fulmer, G. W., & Liang, L. L. (2013). Analyzing the FCI based on a force and motion learning progression. Science Education Review Letters, 2013, 8-14. doi:10.18452/8208
- Ogan-Bekiroglu, F. (2009). Assessing assessment: Examination of pre-service physics teachers’ attitudes towards assessment and factors affecting their attitudes. International Journal of Science Education, 31(1), 1-39. doi:10.1080/09500690701630448
- Otero, V. K., & Nathan, M. J. (2008). Preservice elementary teachers’ views of their students’ prior knowledge of science. Journal of Research in Science Teaching, 45(4), 497-523. doi:10.1002/ tea.20229
- Plake, B. S., & Impara, J. C. (1996). Teacher assessment literacy: What do teachers know about assessment? In G. Phye (Ed.), Handbook of classroom assessment (pp. 53-68). San Diego, CA: Academic Press. doi:10.1016/B978-012554155-8/50005-3
- Plummer, J. D., Palma, C., Flarend, A., Rubin, K., Ong, Y. S., Botzer, B., …Furman, T. (2015). Development of a learning progression for the formation of the solar system. International Journal of Science Education, 37(9), 1381-1401. doi:10.1080/09500693.2015.1036386
- Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Danmark: Institute of Educational Research.
- Sato, M., Wei, R. C., & Darling-Hammond, L. (2008). Improving teachers’ assessment practices through professional development: The case of National Board Certification. American Educational Research Journal, 45(3), 669-700. doi:10.3102/0002831208316955
- Shepard, L. A. (2005). Linking formative assessment to scaffolding. Educational Leadership, 63(3), 66-70.
- Shouse, A. W., Schweingruber, H. A., & Duschl, R. A. (2007). Taking science to school: Learning and teaching science in grades K-8. Washington, DC: National Academy Press.
- Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153-189. doi:10.3102/0034654307313795
- Songer, N. B., Kelcey, B., & Gotwals, A. W. (2009). How and when does complex reasoning occur? Empirically driven development of a learning progression focused on complex reasoning about biodiversity. Journal of Research in Science Teaching, 46(6), 610-631. doi:10.1002/tea.20313
- Steedle, J. T., & Shavelson, R. J. (2009). Supporting valid interpretations of learning progression level diagnoses. Journal of Research in Science Teaching, 46(6), 699-715. doi:10.1002/tea.20308
- Taylor, C. S., & Nolen, S. B. (2008). Classroom assessment: Supporting teaching and learning in real classroom (2nd ed.). Upper Saddle River, NJ: Pearson Education.
- Testa, I., Galano, S., Leccia, S., & Puddu, E. (2015). Development and validation of a learning progression for change of seasons, solar and lunar eclipses, and moon phases. Physical Review Special Topics-Physics Education Research, 11(2), 020102-1-020102-22. doi:10.1103/PhysRev STPER.11.020102
- Treagust, D. F., Jacobwitz, R., Gallagher, J. L., & Parker, J. (2001). Using assessment as a guide in teaching for understanding: A case study of a middle school science class learning about sound. Science Education, 85(2), 137-157.
- Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes (M. Cole, V. John-Steiner, S. Scribner, & E. Souberman, Eds. & Trans.). Cambridge, MA: Harvard University Press. (Original works published 1930-1935).
- Wang, J.-R., Kao, H.-L., & Lin, S.-W. (2010). Preservice teachers’ initial conceptions about assessment of science learning: The coherence with their views of learning science. Teaching and Teacher Education, 26(3), 522-529. doi:10.1016/j.tate.2009.06.014
- Wang, T.-H., Wang, K.-H., & Huang. S.-C. (2008). Designing a web-based assessment environment for improving pre-service teacher assessment literacy. Computers & Education, 51(1), 448-462. doi:10.1016/j.compedu.2007.06.010
- Wang, T.-H., Wang, K.-H., Wang, W.-L., Huang, S.-C., & Chen, S.-Y. (2004). Web-based assessment and test analyses (WATA) system: Development and evaluation. Journal of Computer Assisted Learning, 20(1), 59-71. doi:10.1111/j.1365-2729.2004.00066.x
- Wang, W.-C. (2008). Assessment of differential item functioning. Journal of Applied Measurement, 9(4), 387-408.
- Warbruck, F., & Stachelscheid, K. (2010). Adaptive learning environment for chemical reactions. In G. Cakmakci & M. F. Tasar (Eds.), Contemporary science education research: Learning and assessment (pp. 425-429). Ankara, Turkey: Pegem Akademi.
- Wilson, M. (2008). Cognitive diagnosis using item response models. Journal of Psychology, 216(2), 74-88. doi:10.1027/0044-3409.216.2.74
- Wilson, M., & Carstensen, C. (2005). Assessment to improve learning in mathematics: The BEAR assessment system. Journal of Educational Research and Development, 1(3), 27-50.
- Wilson, M., & Scalise, K. (2006). Assessment to improve learning in higher education: The BEAR assessment system. Higher Education, 52(4), 635-663. doi:10.1007/s10734-004-7263-y
- Wilson, M., Scalise, K., Reitze, A. M. A., Bein, E., Bousselot, T., Gochyyev, P., …Vorp, R. (2010, May). Progress monitoring for real classroom contexts: The formative assessment delivery system. Paper presented at the annual meeting of the American Educational Research Association, Denver, CO.
- Wu, M. L., Adams, R. J., & Wilson, M. R. (2007). ConQuest [Computer software and manual]. Camberwell, Australia: Australian Council for Educational Research.
| 中文APA引文格式 | 陳映孜、何曉琪、劉昆夏、林煥祥、鄭英耀(2017) 。從教師自編科學成就測驗之Rasch分析看教與學。教育科學研究期刊，62(3)，1-23。doi:10.6209/JORIES.2017.62(3).01 |
|---|
| APA Format | Chen, Y.-T., Ho, H.-C., Liu, K.-H., Lin, H.-S., & Cheng, Y.-Y. (2017). Glimpse into teaching and learning using Rasch analyses of a teacher-made science achievement test. Journal of Research in Education Sciences, 62(3), 1-23. doi:10.6209/JORIES.2017.62(3).01 |
|---|
Journal directory listing - Volume 62 (2017) - Journal of Research in Education Sciences【62(3)】September
Glimpse Into Teaching and Learning Using Rasch Analyses of a Teacher-Made Science Achievement Test
Author: Ying-Tze Chen(Institute of Education, National Sun Yat-sen University), Hsiao-Chi Ho(Centre for Humanities Innovation and Social Practices, National Sun Yat-sen University), Kun-Hsia Liu(Institute of Education, National Sun Yat-sen University), Huann-Shyang Lin(Center for General Education, National Sun Yat-sen University), Ying-Yao Cheng(Institute of Education, National Sun Yat-sen University)
Vol.&No.：Vol. 62, No.3
Date：September 2017
Pages：1-23
DOI：10.6209/JORIES.2017.62(3).01
Abstract：
This study examined the quality of a teacher-made achievement test using Rasch analyses and provided useful assessment feedback information for improving teaching and testing. The participants were 8,936 seventh-grade students (4,617 boys and 4,319 girls) from the Kaohsiung area in Taiwan. Data were collected through a teacher-made biology achievement test and analyzed using the Rasch model. The results indicated that the test clearly discriminated students’ abilities. Some questions, however, were found to have unfair content for girls. Additionally, some mismatches were identified between teachers’ expectations and students’ assessments of question difficulty. This paper also provides assessment feedback diagrams, including a person–item map and diagnostic map, and a method of identifying “the zone of fundamental learning development” and “the zone of proximal learning development” is proposed to enable teachers to plan students’ learning progression.
Keywords：assessment feedback, learning progression, Rasch model, teacher-made achievement test, zone of proximal learning development