ToolSQL: A Tool-Assisted Agent for SQL Verification and Refinement

Recent Text-to-SQL methods leverage large language models (LLMs) by incorporating feedback from the database management system. While these methods effectively address execution errors in SQL queries, they struggle with database mismatches - errors that do not trigger execution exceptions. Database mismatches include issues such as condition mismatches and stricter constraint mismatches, both of which are more prevalent in real-world scenarios. To address these challenges, we propose a tool-assisted agent framework for SQL verification and refinement, equipping the LLM-based agent with two specialized tools: a retriever and a detector, designed to diagnose and correct SQL queries with database mismatches. These tools enhance the capability of LLMs to handle real-world questions more effectively. We also introduce SpiderMismatch, a new dataset specifically constructed to reflect the condition mismatch problems encountered in real-world scenarios. Empirical studies demonstrate the effectiveness of our proposed model on Spider and Spider-Realistic datasets in few-shot settings and confirm that our model outperforms baseline methods on SpiderMismatch.