Attention for GNSS height time series prediction: a comparative evaluation of modern transformer-based deep learning architectures

This study presents a comprehensive comparative evaluation of modern Transformer-based architectures for daily GNSS height time series prediction. Eight models, namely Transformer, Reformer, Autoformer, Informer, Crossformer, PatchTST, iTransformer, and TimeXer, were assessed using 14 years of continuous observations from 20 globally distributed GNSS stations. The dataset was partitioned chronologically into training (2012–2021), validation (2022–2023), and independent test (2024–2025) periods. A sliding window configuration with 365-day input length and one-day forecasting horizon was adopted. Performance was evaluated using Root Mean Square Error (RMSE) and Mean Absolute Error (MAE). Results show that architecture-specific design choices significantly influence predictive accuracy. PatchTST and iTransformer achieved the best overall performance, both reaching an aggregated RMSE of 5.1 mm and MAE of 3.7 mm across the test period. TimeXer and Crossformer followed with RMSE values of 5.3 mm and 5.5 mm, respectively. In contrast, Informer showed comparatively lower performance with an RMSE of 7.6 mm and MAE of 5.2 mm. Error distribution analysis revealed near-zero mean bias for all models, while dispersion differences clearly separated the architectures. PatchTST and iTransformer exhibited the lowest standard deviation of 5.1 mm, whereas Informer reached 7.5 mm. Station-level and monthly analyses confirmed that patch-based and variate-wise formulations provide superior robustness under heterogeneous seasonal and noise conditions. The findings demonstrate that modern time-series-oriented Transformer architectures outperform both the original Transformer and sparsity-based adaptations for millimeter-level GNSS height forecasting.