A comparative study of reading sections of high-stakes teacher-made tests in Portugal - Pato, Margarida Maria (Cadernos de Linguagem e Sociedade, 2026)
This study investigates the validity of high-stakes, teacher-made tests as parallel measures of achievement of identical curricular goals. Focusing on three reading sections from tests designed in different schools across Portugal, the research assessed a sample of 75 students representative of the intended population. Two complementary approaches were adopted: expert evaluations and psychometric analyses. Findings revealed discrepancies among experts regarding construct coverage and highlighted variations in item scope, format, subcomponents assessed, and comprehension demands across the sections. Technical shortcomings in instructions, item design, and scoring criteria were also identified. Results from a one-way repeated measures ANOVA demonstrated significant differences in mean performance across the three sections, with section two producing notably lower scores compared to sections one and three. These results suggest that the sections are not equivalent in construct or difficulty. The study reinforces concerns about teachers’ assessment literacy, test quality, and fairness in assessment.