Sir, – Seán Cottrell (Education Today, May 8th) argues that Ireland should withdraw from the OECD Programme for International Student Assessment (PISA). In seeking to support his position, he misrepresents key aspects of PISA, and we think it is important these matters are clarified.
First, Mr Cottrell notes that Shanghai participates in PISA, rather than all of China. But Shanghai’s participation is largely irrelevant. Unlike Ireland, which is an OECD member country, China has no say in the content of PISA, or the methodologies used in the study, nor do its students contribute to OECD benchmarks such as OECD country average scores. Although Mr Cottrell states that only the best Shanghai schools are selected to participate in PISA, in fact, OECD reports on PISA 2009 indicate that PISA schools are representative of the entire Shanghai region.
Second, Mr Cottrell states that Canada is represented in PISA by just two provinces – Alberta and Ontario – and therefore enjoys an advantage relative to other countries. In fact, Canada selects a representative and verifiable national sample that includes all of its provinces. PISA samples for all participating countries are verified by the OECD before results are published. However, countries can also select additional regional samples, which is what Canada did.
Third, Mr Cottrell views PISA performance as a reflection on teaching standards in primary schools. PISA does not seek to establish such a link, and recognises that performance at age 15 is influenced by a myriad of other factors, including structural factors in the education system (school differences in socioeconomic status, especially at post-primary level, allocation of students to vocational and academic programmes, the distribution of 15-year-olds across grade levels, the frequency of grade repetition, the nature and frequency of support for students who do not speak the language of instruction and those with learning difficulties, to name a few). It is important that we continue to interpret performance in Ireland in relation to such factors, so that reduced costs are not the only criterion upon which educational reform is based.
Fourth, Mr Cottrell asserts that students in Ireland leave the PISA assessment before the allotted testing time. In fact, students are required to remain in the test location for the entire duration, even if they have finished the test early, and/or cannot attempt the questions. They may, however, study a school text quietly until time is up.
Participation in PISA affords opportunities to investigate many important issues relating to policy and practice. For example, while the impact of Project Maths will undoubtedly be examined here in Ireland in terms of its effects on Junior and Leaving Certificate grades, and any shifts in the proportions taking higher level papers, much can also be gained by examining the effects (if any) of Project Maths on external measures of mathematics proficiency and attitudes, such as those offered by PISA. We can also draw on PISA to compare the effects of educational change in other countries that are similar to Ireland.
Of course PISA has its flaws and we have endeavoured to highlight these for some time. For example, we have argued ( erc.ie) that the approaches to analysing trend data in PISA are problematic and, notwithstanding demographic and other changes, may have disadvantaged Ireland in 2009. However, it would seem preferable to seek to address such issues from within PISA rather than walk away on the basis of one set of disappointing results. – Yours, etc,