Higher Education Authority publishes new system for measuring third-level institutes’ performance

The performance of each of Ireland's third-level institutions can be compared in unprecedented detail with the introduction of a new performance-profiling system published today by the Higher Education Authority.

While it effectively allows a form of “ranking” of Irish institutions, this is not the intent behind the system, according to the team that developed it.

"We wanted to support high-quality strategic planning," said lead author Muiris O'Connor of the authority's statistics unit. "We felt that the best way to achieve the appropriate balance between institutional autonomy and accountability was to focus on the transparency of performance throughout the third- level education system."

Evidential base
While some data serves very well in measuring success, particularly in the hard sciences, most ranking systems lacked a strong evidential base, Mr O'Connor said.

"One of the main limitations of global university rankings is their strong reliance on reputational surveys," he said. "Reputation is a proxy for quality, which is only very loosely correlated with high performance."

His study, co-authored by Vivienne Patterson, Abigail Chantler and Jasmin Backert, and titled Towards a Performance Evaluation Framework: Profiling Irish Higher Education, includes a review of international assessment methods. It also contains a first attempt at developing a performance-evaluation system, offering a profile for each of the higher education institutions.

Under the scheme, every institution has its own “radar” graph which gives an instant view of how they measure up against the average and against the best performances in 14 categories.

These look at things such as the student/academic staff ratio, the number of students who drop out between first and second year, the number of mature students, the number of PhDs per 10 academic staff, international enrolment and others. The data set is for the 2010-2011 academic year.

For example, University College Dublin has 7,000 postgraduates of all kinds, 4,146 students using online learning, the citation impact of its research papers and the age profile of its staff (46 per cent are between 20 and 39).

Trinity has 4,776 postgrads, 1,924 students using online learning and 44 per cent of its staff are between 20 and 39.

These figures – where relevant – are not weighted to take account of UCD’s much larger student and staff cohorts, so direct comparisons do not necessarily reveal much, Mr O’Connor said. The data does, however, provide a new level of transparency when looking for strengths and weaknesses in each of the institutions, he added.

It also helps the authority and the Government to make decisions on whether an institution is succeeding in its stated goals and aspirations.

“Institutions are free to pursue their mission but have to uphold their accountability,” Mr O’Connor said. “We have diversity [at third level] and we want to build on the diversity in the system.”