Detection of differential item functioning in imbalanced groups. Are INVALSI tests fair among pupils from different academic schools?

Bazoli, Nicola (2019) Detection of differential item functioning in imbalanced groups. Are INVALSI tests fair among pupils from different academic schools?, [Dissertation thesis], Alma Mater Studiorum Università di Bologna. Dottorato di ricerca in Scienze statistiche, 31 Ciclo. DOI 10.6092/unibo/amsdottorato/8937.
Documenti full-text disponibili:
[img] Documento PDF (English) - Richiede un lettore di PDF come Xpdf o Adobe Acrobat Reader
Disponibile con Licenza: Salvo eventuali più ampie autorizzazioni dell'autore, la tesi può essere liberamente consultata e può essere effettuato il salvataggio e la stampa di una copia per fini strettamente personali di studio, di ricerca e di insegnamento, con espresso divieto di qualunque utilizzo direttamente o indirettamente commerciale. Ogni altro diritto sul materiale è riservato.
Download (838kB)


Differential Item Functioning (DIF) and bias measurement are often used as synonyms in standardized tests fairness evaluation between individuals belonging to different groups. Recently, Zumbo et al. (2016, 2017) have provided a redefinition of DIF/bias term and proposed a new methodology for DIF/bias detection analysis. The new definition of bias requires attributional reasoning; therefore, there is a need to find a way to control for possible confounding factors. Only by balancing groups with respect to covariates, it is possible to attribute DIF to group membership. Propensity score matching techniques allow to carry out groups balancing and bias is detected if item is flagged as DIF, after balancing groups. The conditional logistic regression is proposed for DIF detection analysis after matching because it allows to consider the data structure generated by matching. The aim of this work is twofold. Firstly, we assess the efficacy and performance of the new methodology in imbalanced groups, comparing its performance to performance of traditional DIF detection methods (Mantel-Haenszel statistic, logistic regression and Lord's χ2). Our research, through a simulation study, shows that the new methodology outperforms traditional DIF detection methods in imbalanced groups in situations of large samples and DIF items presence. Nevertheless, the new methodology suffers to I error inflation for large samples and simulation results suggest that the use of an effect size measure (ΔR2) reduces significantly this issue. Secondly, the proposal methodology is applied to data coming from the large-scale standardized test administered by the National Evaluation Institute for the School System (INVALSI) to evaluate pupils' Italian language and mathematics competencies. The idea is to detect possible DIF items among pupils from different academic tracks. The results reveal that very few items are flagged as DIF, indicating the fairness of INVALSI tests.

Tipologia del documento
Tesi di dottorato
Bazoli, Nicola
Dottorato di ricerca
Settore disciplinare
Settore concorsuale
Parole chiave
Education Standardized test Test fairness Imbalanced groups Differential item functioning Propensity score Matching Conditional logistic regression
Data di discussione
9 Aprile 2019

Altri metadati

Statistica sui download

Gestione del documento: Visualizza la tesi