Vision-Augmented LLM for Communication Beam Steering Compensation

High altitude platform (HAP) millimeter wave (mmWave) links are sensitive to wind-induced attitude shaking. Small roll, pitch, or yaw tilts will misalign narrow communication beams and markedly reduce the gain of the array antenna. In this paper, we propose a vision-augmented large language model (VA-LLM) that forecasts short-horizon attitudes and pre-steers communication beams before misalignment occurs. Specifically, we design a five-channel vision module to render multivariate flight data into pseudo-RGB images with reversible instance normalization (RevIN) and backbone-aligned normalization. We design a learned cross-variable attention (CVA) branch to condense intra-timestep channel relations into a compact token set. Concurrently, we explore a time-series-aware prompt-as-prefix (PaP) to inject frequency, amplitude, and top- K-based periodic statistics from the input flight-data window into the token set. After fusing vision-augmented, learned, and text tokens, a frozen LLM refines the representation; a lightweight temporal mixer with axis-wise heads directly regresses future roll/pitch/yaw, which are mapped to continuous steering angles for the array antenna. On real flight test data, VA-LLM improves the average signal-to-noise-ratio (SNR)-type array-gain ratio by 6.56% over the baseline, achieves a 7.31% gain over the baseline at the 12step horizon, and yields up to 14.10% gains over the ablations.