Subjective Answer Evaluation Using Keyword Similarity and Regression Techniques

Abstract

This paper introduces a novel approach of automated grading of subjective answers using Natural Language Processing (NLP) techniques. The motivation for the project arises from the need to simplify the process of subjective answer evaluation, which is a repetitive and time-consuming task when done manually. Since no dataset is available for topic presented, we created our own dataset consisting of evaluated student answers for 1 and 3 mark questions on the topics of Social Science. For 1 mark questions, we employed a keyword similarity based grading system. On the other hand, for the 3 mark questions many techniques were explored, including using BERT, DistilBERT, and RoBERTa, which achieved no noteworthy results. Another alternative approach involving both keyword similarity and sentence-sentence similarity was created for the 3 mark questions, which slightly outperformed the previously mentioned techniques. The results for evaluation of 1 mark questions was promising, achieving 90% accuracy. However, there remains significant room for improvement for evaluation of longer answer questions. A key insight from our study is that the scope of improvement is directly related to increasing the quantity and quality of the dataset. This research adds to the ongoing conversation about automation of subjective answer evaluation, aiming to make grading methods more efficient and hassle free in the future. © 2024 IEEE.