Pisa is a triennial international survey produced by the Organisation for Economic Co-operation and Development (OECD) to benchmark the performance students worldwide.
In 2015, over half a million students, representing 28 million 15-year-old students in 72 countries, took the internationally agreed two-hour test.
They were assessed in science, mathematics, reading, collaborative problem solving and financial literacy.
Pua said the alleged manipulation by Education Ministry underscores how the authorities are more keen on pursuing statistical benchmarks and are not interested in real substantive quality and performance of students.
Ong Kian Ming: The education minister should explain so that we are not fooled into thinking that all is well and good in our education system as ‘evidenced’ by the latest Pisa scores.
11 Dec 2016
MP: Heads must roll over attempt to ‘cheat’ Pisa scores
Education Ministry officials who attempted to “cheat” the Programme for International Students Assessment (Pisa 2015) should be sacked, said Petaling Jaya Utara MP Tony Pua.
Pua said the official explanation by assessment organisers had strongly implied that Malaysia’s disqualification from the Pisa 2015 overall ranking had to do with the manipulation of assessment results.
Pisa 2015 Results Volume 1 (Excellence and Equity in Education) report released earlier this month had recorded Malaysia’s mean score for science, reading and mathematics.
However, the mean score was not ranked against other countries on Page 44. The official explanation for this, as stated on Annex A4 on page 304 of the report, is as follows:
“In Malaysia, the Pisa assessment was conducted in accordance with the operational standards and guidelines of the OECD.
“However, the weighted response rate among the initially
sampled Malaysian schools (51 percent) falls well short of the standard Pisa response rate of 85 percent.
“Therefore, the results may not be comparable to those of other countries or to results for Malaysia from previous years.”
This, to Pua, was evidence to allege that Education Ministry officials had attempted to manipulate the results by using a biased sample of schools.
“(The sample used) will not present a fair and accurate reflection of students’ performance in Malaysia,” said Pua.
8 Dec 2016
Ong Kian Ming: Did the Education Ministry artificially boost Malaysia’s school scores?
MP SPEAKS When the Programme for International Student Assessment (Pisa) 2015 results were released on Dec 6, officials from the Education Ministry were quoted in a news report as taking pride that Malaysia’s Pisa’s scores for Mathematics, Reading and Science had improved from 421, 398 and 420 respectively in 2012 to 446, 431 and 443 respectively in 2015.
No doubt, mnisters, deputy ministers and politicians from the BN will use the latest Pisa scores as ‘proof’ that Malaysia is on the ‘right track’ when it comes to the standard of education in the country.
What they would have conveniently left out is the fact that Malaysia does not feature anywhere in the 2015 Pisa rankings for Mathematics, Reading and Science.
The official reason stated, on Page 340 of the Pisa report for Malaysia’s non-inclusion is:
“In Malaysia, the Pisa assessment was conducted in accordance with the operational standards and guidelines of the OECD. However, the weighted response rate among the initially sample Malaysian schools (51 percent) falls well short of the standard Pisa response rate of 85 percent. Therefore, the results may not be comparable to those of other countries or to results for Malaysia from previous years.”
Why was it that only 51 percent of the schools initially chosen for the Pisa test participated in the test in 2015? Was it because the Education Ministry wanted to over-represent students from better performing schools and leave out students from low performing schools?
This 51 percent participation rate raises many suspicions, since Malaysia’s participation rate was 99.3 percent and 100 percent in Pisa 2009 (Page 103, 151 out of 152 schools participated) and Pisa 2012 respectively (Page 181).
It is hard to imagine any school principal not allowing his or her school to participate in the Pisa 2015 test if the Education Ministry had already chosen that school to be in the original sample.
One suspects that the Education Ministry over-sampled the high performing schools in the Pisa 2015 sample and excluded some of the lower performing schools from the sample.
For example, according to the website of the Negri Sembilan Department of Education, the 14 schools listed as the Pisa 2015 sample schools include all seven (100 percent) of Negri Sembilan’s secondary-level High Performing Schools or Sekolah Berprestasi Tinggi (SBT), and all eight (100 percent) of its Fully Residential Schools or Sekolah Berasrama Penuh (SBP). The average student from a SBT or SBP will clearly outperform an average student from a regular secondary school.
Out of a total sample of 8,861 students, 2,661 or 30 percent were from fully residential schools (See Table 1 below). This is clearly an over sampling of students from fully residential schools, since they only comprise less than three percent of the 15-year-old cohort in 2015.
It is highly likely that those overseeing Pisa 2015 saw that the Education Ministry in Malaysia was trying to rig the sample size in order to artificially boost its scores. Is this why Malaysia was ultimately excluded from Pisa 2015 rankings?