Transformer-Based Architectures for Machinery Prognostics: A Review

Abstract

Machinery prognostics requires robust modeling of multivariate degradation signals under noise, non-stationarity, variable operating conditions, and limited run-to-failure labels. Transformer-based deep learning architectures have recently attracted strong interest because self-attention can capture long-range temporal dependencies and inter-sensor interactions more directly than purely recurrent or convolutional models. This focused review presents Transformer-based approaches for machinery prognostics, with emphasis on remaining useful life (RUL) estimation and degradation representation learning.} \rboth{The literature is organized using a consistent taxonomy covering PHM task, Transformer backbone, hybridization strategy, and input representation.} \redit{We also analyze preprocessing choices that strongly influence performance, including windowing, health-indicator construction, tokenization, embedding, and positional encoding. Across benchmark datasets, studied studies frequently show gains from Transformers and hybrid attention models, especially when long temporal context and multivariate dependencies are central. However, improvements are not universal and remain sensitive to evaluation protocol, signal representation, and model complexity. Key open challenges include data efficiency, computational cost, cross-condition generalization, interpretability, and uncertainty quantification. The review concludes by identifying methodological gaps in the current literature and outlining research directions for robust, efficient, and deployable Transformer-based prognostics.