Improving Cloze Distractor Generation Through Retrieval-based Example Selection

Cloze tests, or fill-in-the-blank exercises, are widely used in education and natural language processing to assess comprehension and language proficiency. In this study, we focus on improving the generation of distractors for cloze tests by fine-tuning an open source large language model (LLM) with instructions. Specifically, we employ instruction tuning on the Qwen-7B-Chat model and add examples to the prompts to enhance its ability to cloze distractor generation. Furthermore, our study introduces a retrieval-based example ranking method for instruction tuning in the cloze distractor generation task. We use an alternating training schema to train the retriever and LLM simultaneously. LLM is used to label examples and train the example retriever. The results of the retriever can be used to generate prompts with examples, which can improve the generation efficiency. Our experimental results demonstrate that our fine-tuned model surpasses the current state-of-the-art methods in generating high-quality cloze distractors, highlighting the effectiveness of our approach.