Concept Inventory & Socratic Tutor Bots

Project Summary

The Concept Inventory & Socratic Tutor Bots project explores how AI-driven tutoring can support conceptual understanding by engaging students in guided dialogue rather than answer generation. Built around established concept inventory frameworks, these bots use a knowledge graph RAG system to help students improve reasoning, confront misconceptions, and reflect on their understanding through structured Socratic questioning. The knowledge graph RAG system allows students to bridge their own conceptual competencies with new concepts.

This project directly addresses Bloom’s 2-Sigma problem by approximating the benefits of one-on-one conceptual tutoring at scale, providing individualized feedback that would otherwise be infeasible in large enrollment courses.

Educational Problem Addressed

Concept inventories are widely used to measure student understanding, but they traditionally function as assessment tools rather than learning tools. Students often receive limited feedback beyond whether an answer is correct or incorrect, missing the opportunity to reflect on why their reasoning is flawed or incomplete.

This project reframes concept inventories as interactive learning experiences, using bot-based tutors to transform assessment into formative feedback loops focused on conceptual change.

How AI Is Used

AI tutors are deployed to:

  • Analyze short-answer student responses to concept inventory questions
  • Identify likely correct conceptions and common misconceptions
  • Engage students in Socratic dialogue through targeted follow-up questions
  • Prompt reflection rather than provide direct answers
  • Direct students to external learning resources when appropriate

The AI does not determine grades or final correctness. Instead, it supports metacognition and conceptual repair through guided questioning.

Student Experience

  1. Students complete a concept inventory question with a written explanation.
  2. The bot reviews the response and initiates a Socratic dialogue.
  3. Students respond to follow-up prompts that challenge assumptions or clarify reasoning.
  4. Where appropriate, students are guided toward curated learning resources.
  5. Students leave with a clearer understanding of both correct concepts and misconceptions.

Instructor Experience

Instructors gain insight into:

  • Common conceptual misunderstandings across a cohort
  • Patterns in student reasoning rather than just answer accuracy
  • Opportunities to intervene instructionally at the class or group level

The system is designed to complement, not replace, instructor judgment and curricular decisions.

Deployment Status

Status: Prototype / Classroom Pilot
The system has been tested in controlled instructional contexts and continues to be refined based on student interaction patterns and instructor feedback.

Collaboration and Credit

This project was developed in collaboration with Nathan, whose contributions were central to the design and implementation of the Socratic dialogue framework and system architecture.

Alignment with BOBPE Mission

This project exemplifies the BOBPE mission by leveraging bot-based systems to deliver scalable, personalized conceptual feedback while preserving human-centered pedagogy. It demonstrates how AI can be used to approximate one-on-one tutoring experiences that promote deep understanding without increasing instructor workload.