The Agentic AI Oral Tutor and Examiner: Keeping Professors in the Loop and Scaling Deep Assessment in the Age of LLMs

WHAT:

The rapid adoption of large language models (LLMs) presents universities with two major didactic challenges. First, students increasingly rely on LLMs to engage with course material independently, leaving professors out of the loop and reducing their visibility into learning processes and outcomes. Second, LLMs are widely used to complete written assignments, making it increasingly difficult to determine whether students have genuinely understood the concepts these assignments are intended to assess

WHY:

• Despite these challenges, LLMs also create significant opportunities. They enable personalised and adaptive learning experiences that can support students at their individual level, representing a major pedagogical advantage. In assessment, the issue is not the use of LLMs itself, but rather the difficulty of verifying deep understanding. Oral examinations remain the gold standard for assessing conceptual understanding, yet they have traditionally been impractical to implement at scale. Recent advances in agentic AI now make scalable and rigorous oral assessment a realistic possibility.

HOW:

• This project develops and evaluates an Agentic AI Oral Tutor and Examiner with two parallel objectives: (1) Professor-in-the-loop learning — students interact with an LLM-based tutor guided by professor-defined learning objectives and guardrails, ensuring that educators remain informed and in control; and (2) AI Oral Examination — a validated and scalable oral examination system designed to probe deep understanding across both formative assessments (e.g., assignment submissions) and summative assessments.

• Develop a Professor-in-the-Loop AI Tutor.
• Develop an Agentic AI Oral Examiner.
• Enable Scalable Oral Assessment.
• Improve Assessment Integrity.
• Validate AI Assessment Fairness and Reliability.
• Deploy and Evaluate Across Real Courses.

• Students benefit from a personalised, always-available AI tutor that adapts to their individual learning level and pace, fostering deeper engagement with course material. Rather than passively consuming content, students receive real-time feedback and targeted guidance aligned with the intended learning objectives. The AI Oral Examiner further enhances learning by encouraging students to articulate, explain, and defend their understanding. In addition, it has the potential to support fairer assessment outcomes: unlike human examiners, the AI is not affected by fatigue or unconscious bias, and its decisions can be fully logged and audited.
• Lecturers. Lecturers regain visibility into how students engage with course material through a structured framework that keeps them in the loop without increasing their workload. The AI Oral Examiner enables rigorous assessment of deep understanding at a scale that was previously difficult to achieve, reducing reliance on written assignments that are becoming increasingly challenging to evaluate in the age of LLMs.
• Degree Programmes, Departments, and ETH. This project positions ETH at the forefront of responsible AI integration in higher education. The tools and frameworks developed are designed to be transferable across courses, departments, and institutions, offering a scalable model for AI-assisted teaching, learning, and assessment.