Genome #13: π-rates of the Indian Olympiad Qualifier in Mathematics
Deciphering 'integrity checkpoints' from roll numbers
When I was in school, the word ‘Olympiad’ had a special meaning (and continues to do so) - it would evoke a sense of grandeur and excellence. In my mind, it carried the promise of something extraordinary! My sense of this grandeur came from sitting in the olympiad camps at the lecture halls of HBCSE where they would train you for the Indian National Mathematical Olympiad (INMO). Representing your country at the International Mathematical Olympiad (IMO), the oldest and largest competition for young mathematicians, is an involved process - very much like any other olympiad.
You need everything: grit, passion, talent, and luck (not necessarily in that order) to be one of the six team members to represent India at the International Mathematical Olympiad (IMO). The final IMO team comes through multiple rounds of selection and rigorous training. The year 2024 for India was no different - we had a wonderful team. This team would go on to achieve India’s best-ever performance at the IMO, something that would not go unnoticed:
What is the big deal?
The difficulty of math problems in IMO makes it an interesting dataset for the AI companies. There is often prestige attached to difficult things in life, IMO is no different.
India ranked 4th (4 Gold + 2 Silver) among the 108 teams that participated in IMO in 2024, improving from last year’s performance when it ranked 9th (2 Gold + 2 Silver + 2 Bronze). This is a 'sharp dip’ (good ranking) if you compare our performance ever since we started participating in 1989!
India’s performance at the IMO in 2024 is a big deal considering India's past record in International Olympiads in Biology, Chemistry, Mathematics, and Informatics. Physics (IPhO) remains an exception because India has done well in Physics over multiple years (2018: 1st; 2001: 2nd; 2009: 2nd; 2008: 3rd; 2000: 4th; 2024: 4th). Biology, as evident from the plot below, is hard. India’s performance at the International Informatics Olympiad is also taking ‘sharp dips’, but I digress.
What is the IMO?
The International Mathematical Olympiad (IMO) is the oldest of the 12 International Science olympiads and has been held every year since 1959 (except 1980). Over 100 countries send their team with up to six members. The team members individually solve six problems (only!) in the exam coming from diverse fields of mathematics that are usually not part of high school mathematics: complex geometry, number theory, and functional equations to name a few. The problems are designed such that anyone with a basic understanding can understand the problem even if the solution requires a deeper understanding of mathematics. In the same vein, the solutions almost never require calculus. The underlying premise of the problems being easy to follow is to create simple-looking yet elegant problems inviting ingenious solutions.
To make it to the IMO team from India, you need to go through multiple stages:
Stage 1: IOQM
The Indian Olympiad Qualifier in Mathematics (IOQM) is a 3 hour examination with 30 questions. The answer to each of these questions is a single-digit or a two-digit number. This stage is conducted by the Mathematics Teacher’s Association of India (MTA). all other stages are conducted by HBCSE-TIFR.
For training and selecting students, HBCSE-TIFR has designated several regions and assigned them a “Regional Coordinator”. Two groups of schools: Jawahar Navodaya Vidyalayas (JNV) and Kendriya Vidyalayas (KV) are treated as separate regions and have a ‘Regional Coordinator’ each.
Stage 2: RMO
The second stage examination of the Regional Mathematical Olympiad (RMO) is a three-hour-long examination with 6 problems of “high level of difficulty and sophistication”.
If you qualify the RMO, you are invited to write INMO. But INMO is hard, so HBCSE organizes a training camp at regional levels to teach you concepts and problem solving before testing you at the INMO - the Indian National Mathematical Olympiad Training Camps (INMOTC). You get to meet some of the sharpest minds from your ‘region’ (and the country) here.
Stage 3: INMO
Once you attend the camp (at the regional level), you write the Indian National Mathematical Olympiad (INMO). The INMO is held on the third Sunday of January across the country.
Stage 4: IMOTC
Approximately 65 students - the top performing ones from INMO are invited to Stage 4 International Mathematical Olympiad Training Camp (IMOTC) which is held at HBCSE from April to May. Several selection tests are conducted throughout the course of this camp and eventually, a team of six students is selected to represent India at the IMO.
Here’s a representative summary of the steps and the dates for IMO team selection from India:
The IOQM funnel
Given the problems at each of the stages require a subjective and rigorous (elegant) answer, it is almost impossible to game the system. I do not think I can qualify even if I had access to the exam 24 hours prior. The only exception is Stage 1 which is relatively objective (answers are in single or double-digit).
Since this year’s performance at IMO brought in so much attraction, I took a look at how the statistics of Stage 1. On average, 6500 students make it through IOQM every year:
In my brief search online, I could not find any information on how many students Nstate appeared for the IOQM in 2024 (or any other years). Nstate is a crucial number if we want to calculate the rate of success for each state and how has it evolved over the years, particularly if we are interested in testing if any interventions for mathematics education have a tangible impact on performance at Olympiads. Denominators are often very important (see this post for an example of why).
I could scrape the data for 2024 (cutoff marks at state-region level + roll numbers and names of qualifying students) and 2023 (cutoff marks at state level + roll numbers of qualifying students) and 2022 (roll numbers of qualifying students). We can look at how number of qualifying students, Qstate, varies across states:
There is nothing unusual about this plot. It is partly reflective of the statem demographics (Bihar is young, Maharashtra and UP are populous states), but when you contrast this with past year’s data, things start to look a bit interesting, almost intimidating.
Bihar saw a two-fold jump of candidates qualifying in 2024 as compared to 2023 (2022: 372, 2023: 251; 2024: 606). Again, there is nothing so unusual about this plot either - some states have more candidates qualifying, and some states have fewer - what went up probably just came down.
The cut-off marks which are different for different grades, are statistically different for most states in 2024 as compared to 2023:
There is heterogeneity in how the cutoffs changed for individual states and the underlying regions. For Bihar, the cutoffs are low to begin with (10 in 2023) and changed by at max 9 points, while for Delhi they changed by a large margin of 22 points (from 17 in 2023 to 39 in 2024) yet with very little change in the overall selections.
The natural question to ask then is what really happened in Bihar? What explains the 2X jump in selections? While the above analysis is done at the state level, we can further break it down to state-region level based on the region codes (for example Bihar-North is Bihar-13 and Bihar-South is Bihar-14):
If we break the state on region, the correlation between the number of students qualifying IOQM in 2022 and in 2023 is 0.95. When comparing the 2024 numbers to 2023, the correlation drops to 0.82.
State-regions like Bihar-14, Odisha-17 and Assam-18 have more than two-times candidates qualifying in 2024 as compared to 2023. Again, something that is not so unusual.
When I was parsing roll number records from the 2024 pdfs, I could not help but notice a ‘clustering pattern’. For example, here is the registration number, roll number and standard of the students who qualified IOQM 2024 from Bihar. Exercise: Count the number of consecutive entries.
Consecutive roll numbers stuck together. I calculated the length of the sequence of consecutive roll numbers that appear at the state-region level as Cstate . For example, in a state-region if roll numbers 5,6,7,9,10,13,14,15,16,17,18,19 qualify, the length of the longest sequence is 7. Ideally, this should be done at the centre level but that information is not publically available. The statewide approximation is reasonable here - we can assume that a stretch of consecutive roll numbers was likely assigned to the same centre. Even if that were not the case, the chance of seeing a cluster of consecutive roll numbers qualify is low. We unfortunately do not know the observed success rate Qstate/Nstate to calculate this.
Bihar - 28 consecutive roll numbers that is Cstate/Qstate = 10% of the total qualified candidates made it to the shortlist., Jharkhand with 26 (Cstate/Qstate =10.35%) consecutive roll numbers comes second followed by Andhra Pradesh’s 25 (Cstate/Qstate =8.6%). Not so usual. Here is a plot showing the percentage of qualifying candidates who had the largest stretch of consecutive roll numbers:
By definition, the difference between two consecutive roll numbers is 1. We can stretch the definition to account for a difference up to 2 or 3. For example, if the qualifying roll numbers are 1,3,4,6,7,8,11, the longest stretch of consecutive roll numbers (with a difference =2) would be 6 and with a difference of 3 will be 7.
With a liberal definition of ‘consecutive’ (difference =3), the pattern in 2024 becomes even more prominent:
Results of IOQM were announced on 15th October 2024 and on 20th October, MTAI’s website read
MTA(I) has received several fresh complaints pertaining to IOQM including flagging of unlikely patterns in the roll numbers of selected students. To maintain the integrity of the examination process, MTA(I) is currently reviewing all available information and analysing the data we have. Expect a detailed update and final report by Sunday, 20th October. Till then RMO cutoffs and consequent selection list should be considered provisional. The update can be found above.
HBCSE’s website reads:
HBCSE is aware of allegations of malpractices that may have happened in some regions during the IOQM 2024 examination held on Sunday, September 8, 2024. The Mathematics Teachers’ Association (India) (MTA(I)) who conducted the IOQM 2024 examination have taken cognizance of these allegations and are investigating the matter. Please be assured that HBCSE is committed to the integrity of the Olympiad process, and necessary action, if required, will be taken after a thorough investigation.
MTAI did take cognizance and came up with this solution (emphasis mine):
Update regarding alleged malpractices: As announced earlier, MTA(I) received several complaints regarding the IOQM exam, including flagging of unlikely patterns in the roll numbers of selected students. MTA(I) has carried out a detailed analysis of the score data in order to investigate if any centres have indulged in malpractice. Multiple criteria that might indicate such malpractice were identified, since any one criterion may not be sufficient to rule out chance occurrences. After analysing the data and triangulating on the basis of the criteria identified, MTA(I) has placed a total of six centres in different regions in a “probationary” category. These centres are liable to be blacklisted for the future.
The students who appeared for the IOQM exam from “probationary” exam centres have been removed from the overall list of students who appeared for the exam and the cut-offs have been revised to admit additional students to meet the region-wise and category-wise quotas. This has led to revised lower cut-offs for some regions. A final list of students eligible for RMO based on the revised cut-offs is being prepared and will be announced shortly.
The students who appeared for the IOQM exam in the “probationary” centres and are above the respective regional cut-offs for the RMO exam are placed in a separate “probationary” list. These students will be provisionally allowed to write the RMO exam in their respective regions. However, they are liable to being disqualified from the Olympiads and further penalties if subsequent investigations reveal misconduct on their part.
The revised cutoffs will be announced by tomorrow.
The revised list of students selected along with probationary list of students will be posted on https://ioqm.manageexam.com/ by tomorrow.
TL;DR: MTA investigated complaints of malpractice in the IOQM exam, identifying six exam centres for “probationary” status due to suspicious patterns. As a solution, it then released two lists of (potentially) qualified students -1) a final RMO eligibility list and 2) a "probationary" list for students from flagged centres. Probationary students may face disqualification if misconduct is confirmed.
This is what the distribution of state-region of “probationary” students looks like:
Did MTA(I)’s solution fix everything? We can flag centres requiring an ‘integrity check’ (henceforth referred to as ‘integrity checkpoints’) by focusing on state-regions with 10 or more consecutive roll numbers in the qualifying list.
This initial list of ‘integrity checkpoints’ can be a small fraction of total selections (hence reemphasizing the importance of the denominator):
If we revise our heuristic to flag a state-region if it has a stretch of 10 consecutive roll numbers leading to at least 5% overall selections, we find Assam-18 (Cstate/Qstate =7.8%) and Telangana-06 (Cstate/Qstate =5.2%) also to be potential contenders of "probationary” status with Odisha-17 (Cstate/Qstate =4.2%) and Madhya Pradesh-11 (Cstate/Qstate =4.17%) to be close contenders. Hopefully, this will not be an exercise for the future!
Why would these integrity checkpoints arise in 2024 and not earlier? Probable because there is ‘fame’ attached to qualifying for the ‘Olympiad’ (IOQM), which only got ‘π-rated’ and amplified this year (see PIB notification), possibly a classic example of the Oprah(?) effect.
"Blocks align and π shines bright and true,
At the integrity checkpoints, π-rates break through."