Abstract:
Theoretical framework To provide a concrete description of attained achievement, competency measurement contains the modelling of proficiency levels (König 2009). This option is regularly applied in research studies on student achievement, whereas it has hardly been taken into account by studies measuring pre-service teachers’ pedagogical knowledge yet. However, the current discussion on teacher education standards requires such an application, because proficiency level models enable a connection between a numeric test value and a specific competence.This presentation explores a strategy how proficiency levels of pre-service teachers’ pedagogical knowledge can be modelled. The terminology of test items as well as the complexity of cognitive processes are used to describe test item difficulty and are incorporated into a corresponding measurement model.AimsWe examine the question whether the criteria - terminology of the test items and complexity of cognitive processes - can predict adequately item parameter estimation. We expect that higher item difficulty is accompanied by higher levels of terminology and complexity. In addition we ask for the distribution of different proficiency levels among pre-service teachers during their preparation. Method General pedagogical knowledge (GPK) is assessed via a paper-and-pencil test that was developed and applied in the Teacher Education and Development Study in Mathematics (TEDS-M) (König et al., 2011). Data was collected in the EMW project, an empirical study on teacher training in Germany, Austria and Switzerland. Future teachers who started their bachelor’s program in winter term 2011 were longitudinally sampled at two time points (T1: autumn 2011, T2: autumn 2013). A sub-sample of 1451 pre-service teachers from German and Austrian universities and teacher training colleges is used. Findings Item parameter estimation distributions differentiated by the criteria used to describe test difficulty are nearly as expected and enables the definition of proficiency levels. About 43% of the difference in the difficulties of the individual items can be explained by the proficiency level model (Adjusted R²= .43) (for example of such analyses see, e.g., Gorin & Embretson, 2006). The results of multiple regression were used to construct four proficiency levels. Pre-service teachers in their last year of their bachelor’s program achieve more often higher levels than first-semester students. Frequency distributions of future teachers differentiated by period of training point out the content validity of the modelled proficiency levels. References Gorin, J. S., & Embretson, S. E. (2006). Item difficulty modeling of paragraph comprehension items. Applied Psychological Measurement, 30, 394–411. König, J. (2009). Zur Bildung von Kompetenzniveaus im Pädagogischen Wissen von Lehramtsstudierenden: Terminologie und Komplexität kognitiver Bearbeitungsprozesse als Anforderungsmerkmale von Testaufgaben? Lehrerbildung auf dem Prüfstand, 2 (2), 244- 262. König, J. , Blömeke, S., Paine, L., Schmidt, B. & Hsieh, F-J. (2011). General Pedagogical Knowledge of Future Middle School Teachers. On the Complex Ecology of Teacher Education in the United States, Germany, and Taiwan. Journal of Teacher Education, 62(2), 188-201.Rost, J. (2004). Psychometrische Modelle zur Überprüfung von Bildungsstandards anhand von Kompetenzmodellen. Zeitschrift für Pädagogik, 50 (5), 662-678.