The ACA presented its report on the centralized school-leaving examination

Concurrently with many Austrian schools closing for the summer break today, the Austrian Court of Audit (ACA) published its report on the "Centralized School-Leaving Examination“ (in German). The audit was carried out prior to the COVID-19 pandemic. The audited period spanned the school years 2014/15 through 2017/18 or the calendar years 2014 through 2018. The auditors revealed strong fluctuations in the results regarding the subject of mathematics – in particular at the academic secondary schools, but also at the higher technical and vocational schools. The Federal Ministry of Education, Science and Research had not conducted any analysis to identify the factors behind the fluctuations. The opinions thereon differed within the ministry: the units responsible for mathematics and applied mathematics believed that the level of difficulty remained consistent throughout all examinations at the higher technical and vocational schools. The ministry’s website, however, stated that such fluctuations were partly due to deficits in the formulation of the examination questions. The ACA therefore recommends to the ministry to analyse the reasons for the fluctuations in the results and, if need be, to adopt corresponding measures. 

The ACA’s test result does not rule out different levels of difficulty

A major point of criticism in the report: the data on the results of the centralized school-leaving examination, which was available at the ministry and, even more, incomplete – showed, in particular as regards mathematics, marked fluctuations in the audited period. From 2015 to 2018, for example, the failure rates at the academic secondary schools showed, prior to the retake examinations, fluctuations of between 10.5 per cent and 23.2 per cent. A similar picture was observed for applied mathematics at the higher technical and vocational schools, with fluctuations between 9.1 per cent and 19.7 per cent in the period of 2016 and 2018. The ACA performed a statistical test itself and found that different levels of difficulty could be the cause for such fluctuations. It recommends to the ministry an analysis of the factors behind the fluctuations in the results. If different levels of difficulty are ascertained, measures should be taken to prevent such fluctuations in the spirit of fairness and comparability.

Mathematics as a stumbling block: female candidates performed more poorly than male candidates

The ACA’s audit also revealed that male candidates in mathematics (academic secondary schools) and applied mathematics (higher technical and vocational schools) performed considerably better than female candidates: in 2018, the difference between female and male candidates was at 7.5 percentage points at academic secondary schools; at higher technical and vocational schools it was as high as at 12 percentage points. An examination of the factors underlying the gender gap regarding the results of the centralized school-leaving examination in mathematics had not been conducted by the ministry by the end of the audit. The ACA’s recommendation is therefore as follows: to carry out a comprehensive examination of the gender gap in mathematics in order to be able to take appropriate countermeasures.

Too strict assessment criteria: 51 per cent should suffice to receive the grade “sufficient” (4)

The ACA considers the correction and assessment of the examinations as a shortcoming. In its view, for example, the points scheme underlying the assessments is problematic. At the academic secondary schools, the candidates had to reach 16 of 28 possible points to receive a “sufficient” (4) in mathematics (key competences), i.e. 57 per cent; with regard to living foreign languages the candidates had to reach 60 of 100 points. In this regard, the ACA points to a decision of the Federal Administrative Court concerning a school test, according to which 51 per cent would suffice to receive a positive grade.

In 2018 the ACA reviewed selected mathematics examinations at two academic secondary schools and two higher technical and vocational schools in Upper Austria and Carinthia respectively, which had been graded with just about “sufficient”. In this regard, it criticises the lack of justified assessment proposals. According to the ACA’s view, the grade should be substantiated. Furthermore, the assessments at the examined schools lacked consistency. The ACA recommends to the ministry that the assessors of the examinations prepare assessment proposals whose justification is in accordance with the legal provisions.

The storage of the examination books varies

During their on-site inspections of selected schools in Upper Austria and Carinthia, the ACA’s auditors found that the storage of the examination books posed a security risk as the access by unauthorised persons could not be prevented. Furthermore, the storage rooms for the books varied from a simple wooden cupboard to locked rooms and safes.