UMD Virtual Study Assistant Tied to Lower Grades

The study warns that answer retrieval is not the same as tutoring: even a course-specific, RAG-powered assistant can be pedagogically misaligned. It is not proof that every AI tutor harms learning, but it makes deployment design and learning-outcome evaluation essential.

• –Only about 15% of offered students used VSA; nearly 74% of requests sought information, explanations, or solutions, while fewer than 1% requested feedback.
• –Most instructors retained direct-answer mode, suggesting the product behaved more like an answer engine than a Socratic tutor.
• –Engagement fell even though most students never used the tool, indicating that introducing AI may change broader classroom norms and study behavior.
• –Larger estimated losses for first-generation students raise an equity concern for AI tools marketed as universal academic support.
• –Developers should evaluate independent learning, retention, transfer, and course participation—not just response quality, usage, or retrieval accuracy.