Breaking Size Barrier: Enhancing Reasoning for Large-Size Table Question Answering

Large language models (LLMs) significantly enhance their ability to process tabular data through chain-of-thought reasoning, particularly in table question answering tasks. However, LLMs encounter substantial challenges when dealing with large tables in real-world applications. Prompting LLMs with the entire table not only encounters context-length constraints but also significantly extends the reasoning path, heightening the risk of reasoning hallucination and information truncation. To address this, we construct a large-size table reasoning (LSTR) benchmark, featuring tables larger than those in existing benchmarks, to thoroughly investigate how table size affects the reasoning abilities of LLMs in answering table-related questions. Subsequently, we propose a size-adaptive-thought (SAT) approach that instructs the LLM utilizing refined metadata to employ Python commands for manipulating tables step by step, thereby facilitating efficient reasoning with tables of any size. Furthermore, we develop SAT-Llama, fine-tuned SAT on Llama3.1 (8B), which delivers performance comparable to large-size LLMs at a much lower cost, addressing the issue of inadequate code manipulation capabilities in small-size LLMs. Experimental results on the LSTR and WTQ datasets demonstrate that SAT achieves a new state-of-the-art in handling large-size tables, exhibiting significant performance advantages and high context token efficiency.