A better way to measure school performance

Choosing the best school for their child is an important decision for parents. While parents rightly consider many aspects of schools when making their choice, including sports facilities, the availability of particular subjects, anti-bullying policies and whether their child’s friends are also attending, academic achievement is often a key concern.

For this reason, the higher education progression league tables reported in this newspaper and elsewhere in the media are of great interest to many parents. However, it is widely accepted that these tables do not necessarily reflect school quality. Indeed, the Irish Times writers were careful to attach several health warnings to the published tables. Nevertheless, we are left with headlines such as “Private schools continue to dominate upper end of rankings”, giving the impression that fee-paying schools are particularly good at securing progression to higher education for their students.

Measurement problems

There are, of course, many problems with measuring progression rates given the data available. However, even if progression rates were measured perfectly, these league tables would not provide a suitable measure of school quality. We have long known that academic success in school reflects a combination of the student’s own characteristics and the effectiveness of their school. Disentangling these two effects is crucial in assessing school quality. Schools vary substantially in their student intake and second-level schools whose students were already high achieving in primary school should not be given credit for that.

Similarly, schools taking in students with lower levels of primary school achievement should not be regarded as bad schools if they cannot get 100 per cent of their students into higher education.

The Irish Times view on tensions between the US and China: Speaking up on Hong Kong

---

Boris Johnson shows he cares more about winning than governing

---

The Irish Times view on Dara Murphy’s Dáil activity: Here and there

---

The Irish Times view on Election 2020: the sooner, the better

---

The most widely accepted way to address this issue is to use a value-added approach that nets out the prior achievement levels and family characteristics of incoming students. It is only with the recent availability of the Growing Up in Ireland (GUI) data that such an analysis has become possible in Ireland. In our research, we have used this data to conduct a value-added analysis of a large sample of Irish secondary schools in respect of Junior Certificate performance. The next wave of GUI data will include Leaving Certificate results, and we will be updating our analysis at that stage. Nevertheless, given the strong correlation between Junior and Leaving Certificate results, we are confident that our analysis is highly informative about schools’ value added.

While the data is anonymised, so we cannot identify individual schools, we do know whether each school is fee-paying, secondary, vocational/community or Deis. In our research, we find that the dominance of fee-paying schools seen in the progression league tables is absent when we examine value added, despite their greater resources. While some fee-paying schools rank highly on measures of value added, as a group they perform no better on average than either voluntary secondary schools or vocational/community schools.

Similarly, secondary schools – which have traditionally been regarded as more academic than vocational/community schools – are no different in terms of value added. There are Deis schools in both the secondary and vocational/community sectors that take in students who would not be expected to do well academically given their characteristics, yet manage to raise their performance substantially. These excellent schools will never be recognised in progression league tables.

Thought experiment

Our research also shows that going to a high value-added school can make a substantial difference to academic performance. We conducted a thought experiment to see what would happen if parents chose the highest value-added school available locally and found that the resulting improvement in achievement is comparable to the impact of having a parent with a university degree.

These findings support the provision of more information to parents regarding schools’ academic quality, but it is essential that the information reflects the value added of a school, rather than the characteristics of the students it enrols. The Department of Education publishes the results of periodic whole-school evaluations, which provide useful information on individual schools. However, because these evaluations are qualitative, comparisons across schools are difficult. It should be possible for the department to assemble the data needed to calculate a measure of value added for every school, as is done in some other countries. Providing this information would greatly assist parental decision-making.

Dr Aedin Doris, Prof Donal O’Neill and Dr Olive Sweetman are researchers at the Department of Economics, Finance and Accounting at Maynooth University