Examiner's meetings for E-learning courses

My students have completed the first COMP7310 Green ICT course at ANU. Their marks have been allocated. At this point the university requires that the person running the course and a "second examiner" agree the marks. The results are then presented to an examiner's meeting where others examine the marks. This was an experience I did not look forward to. In effect now I have finished assessing the students, my peers were going to assess me and see if the course and the results were credible.

The experience turned out to be not as bad as I was expecting. There were about six people present for the School of Computer Science (SOCS) masters meeting. A senior academic chaired the meeting and an assistant operated a laptop connected to a projection screen. This displayed course details stored in a bespoke system created by Bob Edwards at SOCS called "FAculty Information System" (FAIS) .

Each course was considered in turn. The results for all students were displayed on screen for a course, sorted in descending order by mark. The system displays the raw mark of each student, any scaling applied and the resulting grade. Also displayed is the average mark, standard deviation and frequency of each grade. Group members then ask to see details of particular students, usually those on the boundary between one grade and the next. The grades for all courses in Computer Science for this student can be displayed, to see if the mark for the course in question is consistent.

In the case of my own course I was worried that I had been too generous with the marks.; So I scaled them to the notional ANU average (65/100). My colleagues reassured me that it was up to me to decide if the marking reflected the correct result, not some statistical measure. The scaling was adjusted to make it less harsh. This is done by entering a PHP function into the system, at which point all of the marks are rescaled and the statistics recalculated. The group found the result acceptable and along with the second examiner, I was able to sign off on the results.

This process worked well and I have suggested that this functionality be added to the ANU's Moodle system (called "Wattle"). There may well be some Moodle add-on which already does this (some is covered in Daniel Servos' Google Summor of Code project: " Student projects/Animated grade statistics report. Also Dr. Eric McCreath has also produced a Marker program which could also be added to Moodle. In addition the system could be sued for analysis of overall student progress trends and for specific topics.

It should be noted that the information from the examiner's board is then entered into the student administration system. So some way to transfer marks from Moodle to the ANU Student Administration System would be useful. ANU uses PeopleSoft Enterprise Student Administration software, and PeopleSoft claim to be able to do some Moodle integration. Also it would be useful to be able to extract information from the PropleSoft system for the examiner's board (as Moodle will only have information on recent courses).

The administrative processes will also need to be adjusted slightly to allow for flexible learning. The current process assumes that all staff can attend a meeting in person. Where courses are designed and delivered flexibly, the staff involved my not be on the campus. The obvious solution would be a meeting by video conference with the web based marking system being able to support this well. However, a real time meeting would still be inconvenient for people is different time zones and an alternative form based approach should be feasible.