Leakage-Safe, Reproducible Benchmarking for Vibration-Based Fault Diagnosis

Abstract

Vibration-based bearing fault diagnosis is a widely studied predictive maintenance problem, but reported results are often difficult to compare. Performance depends not only on the model itself, but also on the evaluation protocol, the train--test split, and the type of domain shift considered. In particular, leakage-prone window-level splitting and loosely defined source--target settings can lead to overly optimistic conclusions that do not reflect real transfer performance across changing operating conditions, acquisition regimes, or bearing identities. To address this issue, this paper introduces a leakage-safe and reproducible benchmark for cross-domain bearing fault diagnosis on the Case Western Reserve University and Paderborn University datasets. The benchmark defines six fixed source--target scenarios, enforces recording-level train--test separation, and evaluates both machine-learning and deep-learning baselines under a common protocol. Final reporting is based on a consistent evaluation setup, with repeated-seed follow-up used where necessary to support reliable conclusions for deep-learning models. The results show that scenario difficulty is highly heterogeneous. Some transfer settings are effectively saturated, while others remain substantially more challenging. Deep-learning models often achieve stronger performance, but their conclusions can be sensitive to initialization and require repeated-seed validation. Overall, the benchmark provides a reproducible basis for scenario-level evaluation and more reliable comparison of cross-domain bearing diagnosis methods. The code for this study is publicly available at https://github.com/1Sensor/pdm-bench.