In recent weeks the quality of research performed in Irish universities and in particular the attitude of businesses to such research has been called into question. The trigger for this was a report on August 12th in the Times Higher Education (THE) magazine announcing the results of its World Academic Summit Innovation Index.

The index ranks 30 nations on the ability of their universities to “attract funding from companies in order to carry out work in innovation and research on their behalf”.

South Korea came out in pole position and Singapore in second place. Ireland's ranking at 30, however, attracted much negative coverage here, with extreme headlines such as "Irish universities are the worst in the world at attracting business funding". How seriously should we take these results and conclusions?

For many of the core activities of universities it is difficult to measure quality directly, so rankings typically use various indirect or proxy quantitative metrics. One is the “innovation score” which claims to capture a university’s “ability to engage in knowledge transfer” by dividing the research income an institution earns from industry by the total number of academic staff it employs.

This year the THE report extended the use of this innovation score to provide a national "innovation index". But on analysis there appears to be some serious flaws in the approach.

Firstly, income from industry alone is not a valid measure of the innovation contribution of a university. This approach ignores completely other key elements of technology transfer such as patents, licence deals and spin-outs.

In addition, some higher education institutions are inherently more focused on science and technology than others. So if the metric used is industry income per academic staff it is clearly inappropriate to treat comprehensive universities with large faculties of humanities, for example, in the same manner as more technology-focused institutions. A similar scale of industry investment will yield vastly different scores for two such institutions.

Specific cases in the innovation index also raise questions about the effects of including certain institutions and excluding others. India, which is ranked at number 10, has more than 550 universities yet none were included in the national calculation for India. Instead data from only three Indian institutes of technology were used. This is not a valid basis for characterising a national system.

South Korea's top rating came through processing data from just six universities, including SKKU, which is the corporate university of Samsung Corporation and Pohang University of Science and Technology, which receives massive funding from its founding industry partner Pohang Iron and Steel Co, the world's largest steelmaker. Such extreme examples of industry engagement clearly distort the analysis, especially when a crude single metric is used.

From these and other system flaws one is forced to conclude that the THE innovation index does not provide nuanced information about innovation and cannot form a valid basis for international comparisons of the degree of industry-academic research collaboration.

How has Ireland fared in other more robust comparisons?

A 2010 OECD report (main science and technology indicators) used data from 2007/8 to provide the percentage of higher education R&D financed by industry in OECD countries. At 3.9 per cent the value for Ireland is below the OECD overall value of 6.6 per cent and is well below the value for Germany (14.2 per cent), but is above the values for Japan (3.0 per cent), France (1.6 per cent) and, interestingly, Singapore (1.3 per cent).

A more recent BERD (Business Expenditure on R&D) report published by the Central Statistics Office in 2011 states that almost a fifth of enterprises in Ireland engaged in joint research with higher education institutions in Ireland in 2009. While it is clear Ireland could do better, it is by no means at the bottom of the table.

Given the economic tumult that has taken place since 2008, it is also important to examine the current situation in the Irish R&D landscape.

Science Foundation Ireland places a strong emphasis on industry engagement and, in February this year, announced a landmark investment in scientific research which is closely aligned to industry needs. Some €200 million of exchequer funding will be invested in seven research centres in Irish universities over the next six years. In addition, more than €100 million in cash and in-kind contributions will come from 156 industry partners, making it our largest ever State-industry research co-funding announcement..

Enterprise Ireland plays a key role in supporting industry-academic research collaboration through its industry-led technology centres on university campuses, its Innovation Partnership and Innovation Voucher schemes. In 2012, 525 Innovation Vouchers and 40 Innovation Partnerships were supported.

Through its Enterprise Partnership Scheme (EPS), the Irish Research Council has engaged over 200 companies to co-fund research in Irish universities in recent years. Since 2010 EPS has generated over €6 million in direct research funding from industry partners.

A key issue that is often cited as a barrier to industry-university research collaboration is the difficulty and inconsistency in agreeing contracts, especially in the context of intellectual property rights. In a partnership between the Irish Universities Association and Enterprise Ireland, a new central technology transfer office has been established and it is expected to lead to greater consistency in industry-academic relationships.

As a result of sustained exchequer investment, the research performance of Irish universities has soared dramatically in recent decades with the result that Ireland is now ranked globally in the top 20 and in the top five for some specific areas of science and technology. A high quality education and research system will be central to Ireland's social and economic recovery. It is evident that flourishing industry-university partnerships can play a key role in this. While current indicators are reasonably positive, there is room to improve our performance. The onus is on both sides to address this.

Prof Brian Mac Craith is president of Dublin City University.