Peer-aided repairer: empowering large language models to repair advanced student assignments

Automated generation of feedback on programming assignments holds significant benefits for programming education, especially when it comes to advanced assignments. Automated Program Repair techniques, especially Large Language Model-based approaches, have gained notable recognition for their potential in fixing introductory assignments. However, the programs used for evaluation are relatively simple. It remains unclear how existing approaches perform in repairing programs from higher-level programming courses. To address these limitations, we curate a new advanced student assignment dataset named Defects4DS from a higher-level programming course. Subsequently, we identify the challenges related to fixing bugs in advanced assignments. Based on the analysis, we develop a framework called PaR that is powered by the Large Language Models. PaR works in three phases: Peer Solution Selection, Multi-Source Prompt Generation, and Program Repair. Peer Solution Selection identifies the closely related peer programs based on lexical, semantic, and syntactic criteria. Then Multi-Source Prompt Generation adeptly combines multiple sources of information to create a comprehensive and informative prompt for the last Program Repair stage. Evaluation reveals that PaR achieves state-of-the-art performance on Defects4DS compared to baseline approaches, with the impressive improvement of 16.13% in repair rate. And experimental results on several introductory programming assignment datasets further demonstrate the effectiveness of PaR, achieving state-of-the-art results on ITSP and IntroClass datasets.