To path indicator To navigation To quick access To footer with other services
To content
IFS-Logo and black lettering of the Center for Research on Education and School Development
Department of Educational Sciences and Psychology
[Translate to English:] © H_Ko​/​adobestock.com
Project duration 01.01.2012 - 31.12.2014

Examination of Test-fairness of Selected Large Scale Assessments for Core Student Sub-populations

The project was focused on systematically testing the test fairness of reading test items from the major school performance surveys for groups of students who differ on the basis of the characteristics (a) gender, (b) social situation, (c) migration status and (d) school form attendance. There was also a special focus on comparing different methods that can be used in this context.

Project description

The central aim of the BMBF-funded project was to re-analyse the tests used in large-scale assessments with regard to their fairness to key student subgroups. The competence tests used in IGLU 2001, IGLU 2006 and IGLU 2011 as well as those used in PISA 2000 and PISA 2009 were considered. This selection allowed a pupil-group-differentiated consideration of reading literacy on the one hand of children in the school year before the important transition from primary to secondary school and on the other hand the consideration of the age group of learners at the end of compulsory schooling. To test fairness to different sub-populations, learners were distinguished on the basis of the following five characteristics: (a) gender, (b) socio-economic status, family migration background in relation to (c) country of birth and (d) language requirements, and (e) school type affiliation. In addition, previously missing analyses on interactions between characteristics were also planned.

Results

The central results are that, in line with the hypothesis, a specific weakness was found in tasks with an open response format for children from socially weaker backgrounds, children with a migration background, boys and children from school types without the Abitur option. This pattern of findings was examined in greater depth by supplementary analyses on the role of motivation. These made it clear that children with low reading motivation show specific weaknesses in open response formats. For the item characteristics of a required demanding reading process and the text genre informative text, however, no or only isolated effects on the relative performance of groups of pupils could be identified within the framework of the project.

Lead researcher at IFS

Project management

  • Dr. Franziska Schwabe

Publication of the results

Lectures

  • Schwabe, F., & McElvany, N. (accepted). DIF for Native and non-Native Speaking Students in Reading Tests with Mixed Item Formats. Paper to be presented at the Annual Meeting of the American Educational Research Association (AERA), 27. April - 1. May in San Francisco, California.
  • Schwabe, F., McElvany, N., El-Khechen, W., & Gebauer, M. M. (2012). Testfairness für Schülerinnen und Schüler mit Migrationshintergrund - Analysen eines Wortschatztests.  Vortrag auf der 77. Tagung der Arbeitsgruppe für Empirische Pädagogische Forschung (AEPF), 10. - 12. September 2012 in Bielefeld.
  • Schwabe, F., Gebauer, M. M., El-Khechen, W., Ünlü, A., & McElvany, N. (2012). Fairness of Language Tests for Students from Various Linguistic Backgrounds. Paper presented at the Annual Meeting of the American Educational Research Association (AERA), 13.-17. April in Vancouver, Canada.

Publications

  • Schwabe, F. & Gebauer, M. M. (2013). (Test-)Fairness - eine Herausforderung an standardisierte Leistungsdiagnostik. In: N. McElvany, M. M. Gebauer, W. Bos & H. G. Holtappels (Hrsg.), Sprachliche, kulturelle und soziale Heterogenität in der Schule als Herausforderung und Chance der Schulentwicklung. IFS-Jahrbuch der Schulentwicklung, Bd. 17 (S. 217-236). Weinheim: Juventa

 

To top of page