IIMs finally issue post-review clarification over CAT 2009

December 17, 2009

It was out on national media by early evening on December 16 that the CAT Committee would not consider scrapping the recently concluded online test and that affected candidates would be rescheduled for a date in Mid-January. Thousands of anxious test takers were however awaiting a formal clarification from the IIMs and that came in this afternoon with the authorities putting up a detailed stance on the CAT IIM website.

The Clarification note clearly indicates that the IIMs would indeed continue with the computerized CAT 2009 as it would not be fair to 96% of the students who have taken it successfully. The note also indicates how the review was taken. It states that about 2.42 lakh candidates registered for CAT 2009. In the first administration window (i.e., from November 28th to Dec 8th, 2009) about 2.16 lakh candidates completed the test. About 10% of the total registered applicants (24,000) did not show up for the test - a percentage that is similar to what has been experienced in the past. The remaining 2000 candidates who were rescheduled remain to be tested.

It also maintains that a very large majority of candidates, over 2 lakhs, completed the test successfully. It is estimated that about 8,000 students may have faced difficulty of various kinds in completing the test. The various problems faced by these candidates have been categorized based on the emails received by the Candidate Care Centre of Prometric and by the IIMs, the telephone calls made by candidates at the Candidate Care Centre, the site and audit reports made by Prometric and NIIT personnel in charge of the labs at the test centre, problems and situations reported by the media, and discussions on blogs and websites.

IIMs have taken note of the categories of problems faced by candidates, which include premature exit due to inadvertent use of "End" button on the review screen in the first few days of testing period; disruptions caused due to malfunctioning of computers; missing graphics and data; and consequent rescheduling of test. The official statement says “Prometric has provided us with data from various sources mentioned above as well as select computer data on the candidates and some video footages. We have also received analysis of computer data identifying candidates who experienced any form of disruption/rebooting of their machine. This is helping us analyze the nature and extent of problems that the candidates faced and identify candidates who were genuinely affected by them.”

The IIMs have made it clear that other complaints that have been posted by test takers on many of the popular forums may need careful reflection. Stating an instance of such claims the note says, “For instance, the "End" button problem - it appears that barely 500 candidates faced this situation over the 11 day window. More importantly, once a candidate presses the "End" button, a message first appears on the screen which alerts a candidate that there are unanswered questions and asks whether the candidate really wants to exit (i.e., by pressing an "End" button one does not exit automatically). The cursor sits on the reply NO and the candidate has to move the cursor to YES and press it before he/she exits. Besides, from day 2 onwards, all candidates were being alerted through repeated announcements to be careful in using these buttons.”

The clarification note also mentions the existence of video footages in each of the room and asserts that these footages are being used to review the test centers who were unable to administer the test satisfactorily. 

The Note states that the question sequence in any lab is also randomized. Consequently, any two candidates sitting on adjacent computers are most likely to be seeing very different questions at any given point of time. This reduces any chance of gaining from any form of cheating (which would also be identified through the video review).

Further the Clarification note  has admitted that unlike testing on a single day (as CAT was in the past), whenever multiple day testing is undertaken, a large number of questions have to be developed with the same level of difficulty. This requires a different technology for generation of tests and establishing their equivalence. The IIMs have stressed upon the fact that all the tests over this 11 day period were dissimilar even while questions were being asked on similar concepts. However, by design, the test included a few questions that were common in at most two tests. This is a standard practice followed by all tests that are offered over multiple days (including GRE, GMAT, TOEFL etc) to develop equivalence in terms of difficulty between two tests. It helps in making appropriate adjustments for varying difficulty levels thereby making the scores comparable across multiple days of testing. A few common situations may occur across different tests forms but key data has always been different in all and each situation was asking a different question or requiring a different analysis. These may have given an impression that questions have been repeated when that may not be true. This kind of testing process and its evaluation system, which is common in other established tests such as GRE, GMAT, and TOEFL, needs to be understood.

The IIMs in the note  informs all candidates that the results are safe as Prometric uses a robust encryption technology for uploading all the data and which will not be affected by any virus. Prometric has assured the IIMs that computer data on the test from all candidates have been successfully uploaded from all the labs.

The Directors of IIMs met on 13 December 2009 to take stock of the situation. Based on a detailed review, the directors decided to continue with computerized CAT 2009. To be fair to those who experienced difficulties during the test, the directors have decided to provide an opportunity to test in the second half of January 2010. On this testing date, candidates who remain to be tested, and all those who, through a review process, are found to have been genuinely affected by various glitches during the first testing period, will be asked to appear for the test. IIMs are in the process of setting up the criteria that will be used to identify candidates eligible for the test to be held in January. The candidates thus identified will be informed by Prometric in the coming weeks by email.