Z score - a sound evaluation system - UWU Vice Chancellor

By Rangana Chandrarathne

Release of results of GCE Advanced level examination was delayed due to the controversy in calculation of Z score from two different examinations under two different syllabi held in August.

The responsibility for the delay was appeared to be thrown from Department of examinations to the University Grants Commission, back and forth to result the students who sat for the examination to have sleepless nights.

The controversy prompted President Mahinda Rajapaksa to appoint a special committee to probe the controversy over Z score.

In the follow up interview with Sunday Observer , Prof. Ranjith Premalal de Silva dispels the gross misinterpretation of the very validity of the Z score system of evaluating examination marks.

The controversy was spearheaded by sections with ulterior political motives to make a national crisis out of the issue.

Professor Ranjith Premalal de Silva is the Vice Chancellor of Uva Wellassa University (UWU) and authored the text book on Spatial Statistics: Theory and Practice and several other publications related to spatial statistics joins the discussion to answer our questions. The interview is a rejoinder to the interview appeared in the December 25th edition of Sunday Observer.

Q: The controversy over the GCE Advanced level results reached to its epitome with accusations from various parties levelled against the Department Examinations. How do you explain the background of this crisis?

A: The evaluation of the answer scripts of the Advanced level examination was made with stringent guidelines adhering to the standard evaluation practices. Therefore, the original marks recorded for the individual students are with high precision as being the case for every examination conducted by the department. The marks were entered to the computer system in accordance with standard data entry procedures with all the error checks and hence can be assumed to be error free.

After the raw marks of each component of the evaluation (i.e. MCQ, Essay type marks) in each subject are entered into the database, the total marks for each student in each subject is calculated and depending on the cutoff, letter grades are assigned.

With the series of marks of each subject for all the candidates, mean and the standard deviation of each subject marks are computed and Z score for each raw mark is calculated by subtracting the mean from the raw mark and dividing the value by the standard deviation.

The Z scores calculated for the three subjects of each student is averaged and a composite Z score of each student is calculated. There was absolutely no debacle with results up to this point and everything was in order. Then the composite Z score of the students needed to be arranged from the highest to the lowest value to provide the corresponding all island merit rank.

The dilemma arose here on the approach to be adopted in preparing a single series of data for national ranking from two different examinations held for old and new syllabi.

It is obvious even for a laymen to understand that marks of two different examinations conducted for two different groups of candidates under two different syllabi cannot be pooled directly even though the marks are brought down to a standard normal distribution using Z scores.

The criticism raised here for the delay in realising the issue of two different examinations held separately is fair and examination authorities are responsible for the lack of forward planning. The discussions took place between Examination Department and UGC in transferring the responsibility of proposing a suitable approach in amalgamating Z scores of two examinations.

When the alarm bells rang, examination authorities attributed the delay due to University Grants Commission being unable to provide a prompt answer. A so-called expert committee was appointed and the committee requested for Z scores of each subject of two examinations separately and awaited without having any serious deliberations on the issue under scrutiny.

The application deadlines for foreign universities were approaching and the students and parents were anxious on the issue. Being a leader sensitive to the pulse of common people, the President realised the concerns of the students and urgency in resolving the issue, instructed to convene a meeting with relevant officials both from Examination department and UGC and advised to release the results without further delay.

The expert committee worked out a formula to combine the Z scores of the two examinations and the Examination department worked round the clock to combine the Z scores from raw data taking into account the mean and the variance of two separate data series as per the formula of the expert committee. The combined Z score was used to prepare the national ranking of the students and then the district ranks were computed using the allocations for each district based on the district quota.

It is in fact not a Herculean task to make these calculations and there is no need of sophisticated programming approach for these calculations. Yet an error was made in the district rank calculation and the results were released without verifying it using basic cross checks due to urgency. Further, it appears to be the lack of confidence of examination authorities on their own marks when they decided to release the results only with index numbers and without the names of the students.

This is a grave mistake and the officials cannot be pardoned for this gross negligence and irresponsible actions damaging the trust and confidence of the people on our examination system. The students, teachers and parents were quick to compare the results and to find the inconsistency of the district ranking. The Examination Department quickly responded and withdrew the results, corrected the mistakes made and then the corrected version was made available within hours. Again it was apparent that there were no proper verifications since the corrected results were released hastily.

Q: How do you respond to the accusations of releasing marks for some students for the subjects that they did not offer in the examination?

A: I do not have prima facie evidence to prove or disprove this accusation. However, from the apparent outlook and rational reasoning, it can be deduced that such an error is not possible to occur and the student would have used an incorrect index number of the examination and the provision for such a mistake was made due to non releasing of names of students along with their index numbers and results.

In order to answer this allegation, examination authorities should have quoted the details of the student through media highlighting the error in the index number of the students rather than blatantly refusing the allegation without substantiating the claim for accuracy of the results. It is not too late for the examination department to release the results with the names of the students to show their confidence on the accuracy of released results.

The doubts are cast when the examination authorities demand several days to release the list of students with best performances in each subject stream. It appears that the indexed numbers have not been mapped with the corresponding names and this requires a detailed investigation into the whole episode. The appointment of a Presidential Committee by the President is commendable and it was the need of the hour to have an independent impartial inquiry into the debacle. It is understood that an error was made but it was promptly corrected.

Q: How do you reestablish the credibility of national examination system?

A: There are two sides of this story. The examination authorities should not be allowed to get the cover from the political leaderships to avoid taking responsibility for the mistakes.

This is not a problem of the government or the ministry and the relevant official should be made responsible for the gross negligence which led to the error in results.

Action plan should be devised to ensure that the same mistakes would not be repeated in the future. The mistake made should be made public and relevant officials should apologise to the students taking the responsibility.

Secondly, the unsubstantiated claims from various organizations to discredit the government with their undercover political motives should be defeated.

The innocent students and parents are being misled by these unpatriotic elements releasing various media statements with their demands to hold the examinations again. They do not have any factual evidence or rational argument but joined together to create a national crisis from this issue. There is absolutely no need to be anxious for having another examination or reevaluation of the raw marks. It is a shame that the leaders of teacher unions are also in the forefront together with disgruntled political figures trying to mislead the public. Most of these individuals who are trying to create havoc in the country do not understand even the basic concept of Z score.

Q: Can you defend the use of Z score in the calculation of student rank and performance in the advanced Level and use of it for university admission?

A: There are better techniques than Z score to provide a much equitable and fair evaluation in the advanced level. However, in terms of the simplicity, computational efficiency and conceptual identity, Z score is the best solution to provide a fair and just evaluation of student performance in diverse subjects.It only requires two parameters to provide standardised results which are possible to be compared upon a continuous scale. A national workshop must be organized and those appear to be champions of issuing media statements should be invited to have a debate on this issue. We are prepared to answer any query related to the validity of Z score application in the Advanced Level examination.