IMA: An Interpretable Machine Learning Framework Based on Large-Scale Anonymous Evaluations to Decode the Black Box of Student-Supervisor Mentorships

Abstract

Student-supervisor relationships (SSR) play a central role in postgraduate training, academic development, and research well-being. In the context of the rapid expansion of Chinese graduate education, understanding mentorship quality has become increasingly important for both educational governance and student development. However, prior SSR studies have often relied on small-scale surveys, context-specific qualitative evidence, or linear analytical approaches, which limits their ability to capture heterogeneous, non-linear, and system-level patterns across institutions and disciplines. To address this gap, we propose Interpretable Mentorship Analytics (IMA), a scalable analytical framework built on large-scale anonymous student evaluations. First, we construct a multi-platform dataset of anonymous supervisor evaluations and transform unstructured review text into structured mentorship indicators through a preprocessing and LLM-assisted feature engineering pipeline. Second, we employ gradient-boosting models, including XGBoost, LightGBM, and Gradient Boosting, to model the relationship between mentorship-related features and overall evaluation outcomes. Third, we apply explainable machine learning methods, particularly SHAP and LIME, to identify global feature importance, local decision patterns, and non-linear interactions among mentorship dimensions. The results show that IMA can effectively uncover the key drivers of mentorship satisfaction, especially the central role of teacher-student relationship quality, while also revealing substantial heterogeneity across regions, institution types, and disciplines. By combining large-scale anonymous evaluations with interpretable predictive modeling, this study provides a transparent and data-driven framework for evaluations with interpretable predictive modeling, this study provides a transparent and data-driven framework for understanding SSR and offers empirical evidence for improving postgraduate supervision and educational policy.

Document