Artificial intelligence continues to push boundaries, and a recent experiment has shown just how far it has come. When an IIT graduate experimented ChatGPT o3 on the questions of JEE Advanced exam, it was shocking to see the AI not only cleared the paper, but scored AIR (All-India-Rank) – 4, showing the capability of AI.
An IIT Kharagpur graduate, Anushka Aashvi, put OpenAI’s latest chatbot, ChatGPT o3, to the test on a mock version of India’s notoriously difficult engineering entrance exam, JEE Advanced 2025. The results were nothing short of astonishing, showcasing AI matching human intellect at a very high level.
ChatGPT & JEE Advanced
ChatGPT o3 achieved a jaw-dropping score of 327 out of 360 in the mock test. This phenomenal performance translates to an All India Rank of 4. To put this into perspective, only the top 17,000 out of over 15 lakh aspirants make the cut for the prestigious IITs each year, making the acceptance rate around 0.01%. The leap in performance from previous models is significant; GPT-4’s performance on a similar test a few years ago would have placed it somewhere in the 10,000s.
Anushka Aashvi designed the experiment to simulate realistic test conditions. She instructed ChatGPT o3 to solve the test with the mindset of a JEE aspirant. Crucially, the AI was asked not to use external tools like web search or Python, though it did occasionally try to invoke its Python tool which was detectable. To prevent memory bias, each question was presented in a fresh chat session, and no hints or corrections were given between questions. The AI’s answers were compared against the official answer key.
AI’s Strengths & Weakness
Despite the rigorous conditions, ChatGPT o3 demonstrated marvelous problem-solving skills. It effortlessly tackled complex calculus and lengthy algebra problems. It performed remarkably well at combining concepts from multiple chapters and even successfully interpreted compounds from skeletal chemical structures. The chatbot’s approach was often described as step-by-step and methodical, similar to that of a well-prepared student. It even double-checked its calculations frequently. In Phase 2 of the test, it scored a perfect 60s in both Mathematics and Chemistry.
However, the AI wasn’t flawless and had its share of flaws. ChatGPT o3 struggled significantly with graphical interpretation and questions demanding the use of visual tools like the Vernier scale. For one visual question, it took nearly ten minutes and still arrived at the wrong answer. This highlights its shortcomings in processing tactile and visual data compared to processing text-based problems.
The Bigger Picture
The JEE Advanced is designed to test clarity of thought, depth of understanding, and problem-solving ability with completely new and unseen questions. For an AI to score at this level is a significant event. It feels eerily similar to the moment in the 1990s when IBM’s Deep Blue defeated Chess GrandMaster Garry Kasparov, proving computers could surpass humans in domains previously thought exclusive to human intellect. This achievement underscores the accelerating progress of AI and signals that Artificial Superintelligence isn’t just coming, it’s already here in certain domains.
Read this: Texas Youth earned $34,000 using ChatGPT by doing this
ChatGPT o3’s performance on the JEE Advanced mock test is a powerful indicator of AI’s evolving capabilities. While still having limitations in processing visual data, its ability to master complex problem-solving and score among the top engineers highlights a significant shift. This event is both humbling and deeply fascinating, serving as a warning bell and an opportunity as we navigate a future where the definition of “smart” is rapidly changing. The game has truly changed.
This also means that students who are preparing for JEE entrance exam can make use of ChatGPT as its tutor and leverage the best out of it, exploring the nuances of concepts. You can also ChatGPT to crack interviews and jobs. Read more about it HERE.
Here are the key takeaways:
- ChatGPT o3 scored a remarkable 327/360 in a JEE Advanced mock test, achieving AIR 4.
- The experiment used fresh chat sessions for each question with no external tools or hints allowed.
- The AI showed marvelous problem-solving for complex math, physics, and chemistry problems.
- ChatGPT o3 struggled with graphical interpretation and visual tools like the Vernier scale.
- This result is a massive improvement from GPT-4 and indicates AI reaching top human levels.
Join our community by subscribing to our Weekly Newsletter to stay updated on the latest AI updates and technologies, including the tips and how-to guides. (Also, follow us on Instagram (@inner_detail) for more updates in your feed).
(For more such interesting informational, technology and innovation stuffs, keep reading The Inner Detail).
Kindly add ‘The Inner Detail’ to your Google News Feed by following us!
