LatenAux: Toward latency-aware trajectory prediction for autonomous driving via consolidated auxiliary learning

What the paper says

Accurate trajectory forecasting is essential for enabling autonomous vehicles to navigate safely in complex traffic environments. Current models typically assume that predictions are latency-free, an idealistic simplification that fails in real-world settings, where processing, computation, and transmission inevitably introduce delays. During this latency window, target agents continue to move, which renders early predictions obsolete, thus degrading system performance. To address this challenge, we introduce latency-aware trajectory prediction, a new task that explicitly accounts for latency and repurposes it as a useful signal. We present LatenAux, a consolidated auxiliary learning paradigm that first decouples prediction into two tasks: a primary task that predicts valid-horizon trajectories from historical data, and an auxiliary task that utilizes latency-inclusive observations. By allowing the auxiliary branch access to latency-crafted inputs, LatenAux is then committed to transferring latency-aware knowledge to the primary branch via a progressive feature alignment strategy. This enables the primary model to internalize latency cues without explicit reliance on latency data. Our method departs from conventional auxiliary learning by introducing a soft feature-consistency function, which gradually incorporates auxiliary representations across both scene context and query state levels, enriching features while avoiding overconstraint. In addition, auxiliary queries act as informative priors for the primary branch to further enhance prediction accuracy. Extensive experiments on two large-scale real-world datasets demonstrate the effectiveness and superiority of LatenAux, showing that it consistently supports latency-aware modeling and delivers more accurate and reliable trajectory forecasts.