Vision-driven Adaptive Decoding for Document Understanding: A Dual-Path Visual-Language and OCR Framework with Prompt-aware Task Routing

Real-world document understanding faces challenges such as layout diversity, low-quality imaging, and multilingual text, which hinder text recognition and table structure reconstruction. Traditional OCR-based pipelines achieve high character-level precision but lack global semantic understanding, while general-purpose vision-language models (VLMs) often struggle on text-heavy documents. To address these limitations, we propose VADoc (Vision-driven Adaptive Document Understanding), a dual-branch multimodal fusion framework that integrates an OCR-based branch for token-level grounding with Qwen2.5-VL for layout and semantic reasoning. A lightweight fusion module aligns both outputs into coherent text and structured HTML representations. Experiments on OmniDocBench show that VADoc substantially improves both text accuracy and table reconstruction quality over standalone VLMs and OCR systems. These results indicate that combining OCR precision with VLM contextual reasoning effectively bridges perception and understanding, and that VADoc supports robust document digitization and structured data extraction for downstream NLP and analytics tasks.