Methods for Assessing Theme Adherence in Student Thesis

Mikhail Tikhomirov, Natalia V. Loukachevitch, Boris V. Dobrov

Abstract: In this paper we study approaches to assessing the quality of student theses in pedagogics. We consider a specific subtask in thesis scoring of estimating its adherence to the thesis’s theme. The special document (theme header) comprising the theme, aim, object, tasks of the thesis is formed. The theme adherence is calculated as the similarity value between the theme header and thesis segments. For evaluation we order theses in the increased value of the calculated theme adherence and compare the ordering with expert grades using the NDCG measure. We explore different methods, including probabilistic topic modeling, word embeddings and ontologies. The best configuration for theses ranking is based on the weighted averaged sum of word embeddings (word2vec) and keywords extracted from the theme header.