A lot of work occurs after examination papers have been marked and passed through the computer analysis process. Much of this work is intended for maintaining the high standards of examination boards. Two main processes that are undertaken after the scoring and analysis processes are the grades clean-up process before candidates’ examination grades are published, and the writing of Reports to Schools.
Grades clean up process
The grades clean-up process is a system for making compensations in the grades of some examination candidates considering the stress and anxieties candidates go through during the examination period. Anxieties tend to lower the performance level of candidates, consequently lowering their grades in some of the examination papers they wrote.
With the negative effects of anxieties in mind, a number of processes are adopted by some examination boards to help counter the possible effects of stress and anxieties in the grades of some candidates. It is important to bear in mind that not all examination boards use these compensatory processes.
Some of the recorded and approved grades clean-up processes are provided in this article to help both examining boards that use some of these processes and boards that do not as yet use any of such processes. The processes discussed in this article include using the error margin in the scores of each subject in the grades compensation process; using huddles; using the examination score prediction system for cases where a candidate wrote Paper 1 of a subject and for some good reason missed writing Paper 2 etc.; and lastly consideration of cases where a candidate or school were unsatisfied with the grade or grades awarded some of their candidates and therefore ask for remarking of the examination scripts of such candidates.
Using standard error margin in grade compensation
Using the calculated standard error in each subject examination is one of the processes adopted in the processes for grades compensation. After the scoring process of examination scripts, the subject officers will calculate the reliability of each of the examination papers of each subject. For achievement tests, the reliability is expected to be 0.90 or higher.
With the standard deviation and the reliability of each of the examination papers, the subject officers will then calculate the standard error of each examination paper to be used in compensatory situations during the grade clean up process. The standard error is used in only selective cases and not over all candidates’ grades.
Assume for example, that a candidate offers 7 subjects in the examination. Assume also that the candidate obtains Grade A in 6 out of the 7 subjects offered, and Grade B, or even Grade C in the 7th subject.
The following assumptions should be followed as considerations for the application of the standard error.
- The standard error is assumed to be 2 percentage points in this example.
- The cut-off score for the Grade B in the subject of the candidate under consideration is 70-79%; while Grade A is 80% above..
- The candidate under consideration obtained 77% and is therefore awarded Grade B.
It is realized that such a candidate and others like this one, is a high achiever whose intellectual potential is worth considering. Only few candidates offer more than six subjects in major certificate examinations. This candidate obtained Grade B in one of their 7 subjects most likely because of the stressful amount of work they had to undertake in all the 7 subjects offered.
When the error margin of 2 percentage points is deducted from the upper limit of the Grade B cut-score range of 70-79, (i.e. 79-2) the result is 77%. This means that all candidates who obtained 77%, 78% and 79% in this examination paper and were awarded Grade B actually qualify to be moved to Grade A. The assumption in this case is that such candidates earned Grade B in this particular subject due to error from stress and other negative examination factors.
There is a condition however, before applying the standard error margin concept. The condition, which may depend on the examination board, is that only candidates who obtained Grade A in at least 4 subjects are considered for compensation in cases as the above. By re-writing the instructions for grades award in the subject under consideration, all candidates who obtained Grade A in at least 4 subjects, and Grade B in some of the subjects, are moved automatically to Grade A by the computer where their Grade B scores fell within the standard error margins.
In such grades clean-up process, candidates who otherwise would have been awarded 4 Grade A and 3 Grade B, could end up with 5 Grade A and 2 Grade B, where the candidate offered seven subjects for the examination.
Using huddles in grade compensation cases
Other compensations may generally be made for both high and low performing candidates using “huddles’.
Huddles are used in cases where using the standard error will not be helpful. Huddles may be used by consideration of a candidate’s performance within a subject, or by comparison of a candidate’s performance in another subject that has similar knowledge and skill requirements as the subject under consideration.
Assume a candidate is awarded Grade E, that is :Below performance” in the English Language examination; if the candidate obtained Credit with raw score of 12 out of allocated 20 marks in the comprehension component of the English Language examination, the candidate could be moved from “Below performance class” to Grade D. This is an example of grade compensations by considerations within a subject. Note that candidates who obtain grades A, B and C in the English Language examination are those who perform very well in the essay component of the English examination.
An example of cases where huddles are used across subjects could be as follows: A science candidate obtains Grade A in 6 out of 7 subjects offered for the examination, obtaining Grade B in physics with a score of 72% which could not be moved to Grade A because of being outside the error margin. In such a case, the candidate’s performance in mathematics is taken into consideration especially in this era when STEM candidates are treasured. The instruction to the processing computer could be written this way:
If a candidate obtains Grade B raw score equal to or above 72 in physics and obtains Grade A in mathematics, move the candidate from Grade B in physics to Grade A.
The grade clean up process involving the use of the standard error and huddles are carried out by the processing computer following the instructions written by the subject officer in consultation with the chief examiner, and going by the approved regulations of the Examinations Board.
Hardship cases
Hardship cases arise where a candidate could not write one component of the set of examination papers. For example, a candidate writes Paper 1 of a subject and falls sick and cannot continue with the rest of the examination in that subject. There are other cases where a candidate gets into an accident and cannot write a component of their examination subject. In West Africa where the end of school completion examinations takes place from May to end of June, the same period for the heavy monsoon rains, a candidate living in a village can easily miss one subject component because of their inability to get transportation to the examination centre situated farther away from the candidate’s village because of the rains. These are real situations. It is in such cases that statistical processes have been developed for predicting the score of candidates who suddenly experience such hardship situations.
Using score prediction in hardship cases
The score prediction process may be used in cases as described above and in cases where the examination processes have reached an advanced stage; a stage where a delay of more than a day or two cannot be tolerated because of the urgency for publishing the examination results.
Assume that a candidate missed writing Paper 2, the essay component of the English Language examination. The candidate therefore could not complete the examination in English and could consequently not obtain a grade in English Language. As a result of inability to obtain in grade in English, the candidate would not be admitted to any higher institution for further studies because of lack of a grade in English, a vital qualification in most African countries.
Where the examination body has studies on the correlation between English Language and its equivalent in the School Based Assessment (SBA) of the country, this relationship can be used to predict the score of a candidate who missed writing a part of the external English Language examination. It is used in a case where a candidate could not, for good reason, write Part 2 of the English Language examination for instance.
In a case of this nature, the prediction follows the general formula for fitting a straight line to individual scores with reference to a set of rectangular coordinate axes using the formula
Y’ = bX + c
Where:
Y’ is the predicted score in the subject component the candidate missed;
b is the raw score regression coefficient, or slope, designated as by.x and calculated as:
Sy is the standard deviation of the scores in the external English examination; and Sx the standard deviation of the scores in the internal SBA.
c is the intercept or constant calculated as: My - bMx; (My the mean of Y, the score to be predicted, and Mx the mean of the independent variable for predicting the Y score of the candidate).
The above prediction system cannot be used by many examination boards essentially because of lack of the following requirements:
Correlation indices: Correlation indices between the external examination papers and the internal school examination papers; such as the School Based Assessment results.
The above indices and other comparisons between the external examination papers and the internal school assessment papers should form an essential part of the work of the research division of the examination board. This will make it easier to make score predictions for candidates’ who, by some unfortunate circumstances, have to be given predicted scores.
The alternative to score prediction in the circumstances described above, without having to wait for the candidate to sit the examination in the next year, is to give the candidate a parallel form of the examination paper they missed, a week later. This may also not be possible until examination boards in Africa move to computerized examinations development system.
Whenever, the score predication system is used to provide a score for a candidate in hardship cases, the Board should provide a note on the certificate awarded to the candidate indicating that the grade obtained by the candidate in the specified subject was arrived at by score prediction. The note could be at the back of the certificate.
Unsatisfied candidates and re-sits
There are cases where a candidate from a school may not be satisfied with the grade obtained in the external examination. In such a case, the head of the school of the candidate will file a complaint to the Examination Board on behalf of the candidate. The board will then get the examination script of the subject in contention to be remarked by another examiner, especially by the Chief Examiner.
If there is significant disparity between the new score assigned in the remarked script as against the score originally assigned, the Chief Examiner’s score will stand.
If the candidate is still not satisfied with the Chief Examiner’s score, the candidate will be granted the chance for a re-sit. In this case, a new examination paper or set of examination papers in the subject will have to be randomly selected from the item/question bank for the re-sit and marked at a cost to the candidate.
Examination of private candidates
As indicated already, with a computerized item/question bank system, an examination board should be ready to offer examination to private candidates at different times in the year, apart from their core responsibility of offering examinations to school candidates. The same process of selecting a set of parallel forms of the examination papers will have to be followed.
Reports to Schools
Two reports must be written and distributed to schools every year. These reports are the “Syllabus Amendments to Schools” and the “Report to Schools” on candidates’ examination performance in each of the subjects examined in the year.
Syllabus Amendments to Schools
The words “syllabus” and “curriculum” may be used interchangeably depending on whether the examination board uses the English nomenclature “syllabus” or the American nomenclature “curriculum.” For countries that use the word “curriculum”, the term “syllabus” used for the document described below could be changed to “curriculum” to avoid confusion.
Syllabus Amendments
Generally, the school syllabus is developed by the curriculum division of the Ministry of Education and copies supplied to the Examination Board. The paper setting and grading division of the Examination Board will then use the relevant syllabus to set the examination papers. Printing and secure storage of examination question papers are responsibilities of the Examinations Administration Division of the Board. This however, means that the curriculum division of the Ministry of Education and the paper setting and grading division of the Examination Board should work hand in hand in many cases.
New important issues crop up each year that will need to be reflected in some of the syllabuses. For this purpose, the subject officers of the curriculum division of the examining board as well as the officers of the curriculum division of the Ministry of Education should be constantly looking out for such vital new knowledge that should be reflected in respective subject syllabuses.
Sometimes the addition or change required in a syllabus is not new knowledge but rather a re-arrangement of two or three specific objectives in a syllabus or curriculum to make learning of the objectives of a unit more meaningful. The re-arrangement or the need for a new objective in a syllabus could come from observation, from knowledge of the errors that students make in the external examinations and from a variety of documents.
At the end of every major national external examination in the various African countries, the subject officer of the Curriculum Division, working together with the Chief Examiner will identify areas of the syllabus that posed difficulties in learning in the school system. The learning difficulties could have been due to ambiguities in the manner in which the objectives in the syllabus have been stated. With the necessary corrections made to such ambiguities, the subject officer and the responsible officer on the examining panel will make required changes in the respective syllabuses and send the changes to schools in a document titled: “Syllabus Amendments to Schools”, with copies to the Ministry of Education.
The document may generally be about three of four pages and possibly up to six pages. Out of the many subjects on the curriculum, it is likely that about 15 subjects will have to send syllabus amendment documents to schools for the guidance of class teachers almost every year. While every subject syllabus may need some amendments each year, the revision of each syllabus should generally be carried out every five years and sent to schools.
Some subjects, referred to as “fast subjects”, that is Electronics, ICT, Physics and any of the subjects in which changes occur almost every year internationally, will need to be revised every three years and issued to schools. All other school syllabus (or curriculum) must be revised every five years as already indicated above.
At the end of five years, all amendments that have been issued to schools and all changes that have come into the subject should then be compiled and used for revise and issue new syllabuses. Where the knowledge in some subjects have not significantly changed and where the amendments to schools document adequately helps schools to deal with the required changes, the syllabus can go on for a longer time.
Report to Schools on examination performance
At the end of each major external examination, a Report to Schools must be issued. Using Chemistry as an example, the title on the cover of the report should look as follows:
Report to Schools:
June 2022 examination in Chemistry
The report should be in two sections: Section A and Section B. Section A will provide information on the number of candidates that wrote the chemistry examination in the year; the mean performance of the candidates and the standard deviation of the scores in the examination.
Section A should provide the above data for at least five years going backwards in order for teachers and school authorities to observe the trend of candidate numbers as well as the performance standard of candidates across the years. The example below is the type of table that the subject officers of the examining board should develop to show the data required in this section.
The table further shows that the performance in chemistry has been increasing, as reflected in the increasing mean score from the year 2017 - 2022. The standard deviations recorded in the last column show that performance in the years from 2016 to 2020 had low range indicating that performance of candidates tended to bunch in those years, while performance had more variation in the years 2021 and 2022. Greater variation means greater differentiation in the abilities of candidates.
Table 2 following, shows the percentage of candidates obtaining the various grades in the years reported.
Section B of the Report to Schools will present the Chief Examiner’s report on the year’s examination together with the views and recommendations of the subject officer based on their observations and diligent studies over the years.
The two reports discussed above, Syllabus Amendments to Schools and Report to Schools (on the year’s examination) are feedback reports providing schools the opportunity to realise their teaching and learning errors, and also learn from the advice provided in the reports.
Copies of the reports must be sent to the Ministry of Education, to all schools, and to the National Teachers Council and the National Inspectorate Board where such bodies exist in the country. It is these two bodies that are particularly expected to ensure teaching and learning improvement in the school system of the country. It is these two bodies that are expected to take immediate action on the recommendations contained in the reports and also organize in-service training sessions for teachers.
The reports should be tended at the plenary meeting of the Examination Board for notification. Although the board is expected to note that the reports had been written and supplied to the appropriated bodies, the Chief Examiners and relevant subject officers should be in attendance at the meeting and be ready to answer questions that may by posed by members of the Board or Council. Questions may generally require answers to issues on syllabus amendments, results on the examinations, remedial actions taken to correct errors in learning, the quality of textbooks and materials available for school learning, and other actions that the curriculum division is required to take to help improve teaching and learning in the school system of the country.
NOTE
This article is based on curriculum and examination practices of examining boards especially the practices of the Caribbean Examination Council (CXC), a notably efficient examining body where the writer was privileged to have worked for considerable number of years; apart from membership on the Council and School Examinations Committee of the West African Examination Council in the later years of the 1970s.
Knowledge of the practices of the British Examining Boards will also be very useful for educational improvement in African countries. The only exception is that the British examination system based on the General Certificate in Secondary Education (GCSE) which was set up in the mid 1980s to offer a single examination for both high ability candidates who previously sat for the GCE certificate and the lower ability candidates who previously sat the CSE. Bringing the two examinations GCE and CSE together in one examination system required changes in paper setting and grading systems in the GCSE examination. These are the important issues that African educational bodies should note.
Finally, readers are informed that the statistical formulae provided on an earlier page of this article can be found in many books in Statistics, Educational Statistics, and Psychology and Education. Authors of such books include Hays, Ferguson and others. Any edition of books in Educational Measurement and Statistics by Anthony J. Nitko will also be vital additions to the library of every Examination Board.
KBQ
December, 2022
