Probabilistic Spatiotemporal Modelling with Kernelised Attention

Accurate modelling of spatiotemporal environmental data remains a challenging problem due to complex spatial dependency and temporal dynamics. While transformer architectures have demonstrated
strong performance in many sequence modelling tasks, their effectiveness in sparse and noisy geospatial applications is less well understood. We propose a novel framework for probabilistic spatiotemporal modelling
with kernelised attention, which integrates 3D convolutional feature extraction, structured kernel interpolation (SKI)-based attention, and Gaussian process (GP) modelling. Our approach reinterprets attention as a kernel operator and introduces SKI approximation to reduce computational complexity. We evaluate our method on satellite-derived NDVI data from four recent flood-affected sites, enabling assessment under both gradual vegetation dynamics and large-scale environmental disturbances, which are characterised
by strong seasonality and missing observations, where uncertainty-aware models are particularly crucial. Furthermore, we demonstrate that incorporating a periodic kernel yields the best performance, reflecting
the inherent seasonal structure of vegetation dynamics. These findings suggest that in noisy, partially observed spatiotemporal applications, explicitly embedding kernel structure and inductive bias into attention
mechanisms is crucial for robust learning.