Robust Baselines and Probability Calibration for TPM-Oriented Predictive Maintenance
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Apr 27, 2026
José Roberto Dale Luche
Blaha Gregory Correia dos Santos Goussain
Claudia Regina de Freitas
https://orcid.org/0000-0002-6325-9410
Claudia Regina de Freitas
Abstract
Predictive maintenance (PdM) under severe class imbalance challenges model evaluation and deployment, especially when probabilities inform maintenance decisions. Using the AI4I 2020 dataset, this study establishes a reproducible baseline for failure detection with emphasis on rigorous validation and probability calibration. Models such as Random Forest (RF), Multilayer Perceptron (MLP), and classical baselines were evaluated via nested cross-validation with strict leakage control. Metrics included Average Precision, recall, precision, Brier score, and Expected Calibration Error (ECE), while the impact of SMOTE on class imbalance was analyzed. RF achieved the most robust balance between discrimination and calibration reliability, whereas MLP with SMOTE improved sensitivity but incurred calibration and false-positive trade-offs. Torque and tool wear emerged as dominant predictors, aligning with physical degradation mechanisms. By explicitly linking predictive performance to probability calibration and operational cost considerations, this work provides an actionable, cost-aware reference baseline for PdM within Total Productive Maintenance frameworks.
##plugins.themes.bootstrap3.article.details##
Keywords
Industrial Artificial Intelligence, Reliability Engineering, Imbalanced Data Classification, Maintenance Decision Support, Interpretable Machine Learning
References
Ahuja, I. P. S., & Khamba, J. S. (2008). Total productive maintenance: literature review and directions. International journal of quality & reliability management, 25(7), 709-756. doi: 10.1108/02656710810890890.
Alshkeili, H. M. H. A., Almheiri, S. J., & Khan, M. A. (2025). Privacy-Preserving Interpretability: An Explainable Federated Learning Model for Predictive Maintenance in Sustainable Manufacturing and Industry 4.0. AI, 6(6), 117. doi: 10.3390/ai6060117.
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., ... & Herrera, F. (2020). Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information fusion, 58, 82-115. doi: 10.1016/j.inffus.2019.12.012.
Autran, J. V., Kuhn, V., Diguet, J. P., Dubois, M., & Buche, C. (2024). AI4I-PMDI: Predictive maintenance datasets with complex industrial settings’ irregularities. Procedia Computer Science, 246, 1201-1209. doi: 10.1016/j.procs.2024.09.546.
Bezerra, F. E., Oliveira Neto, G. C. D., Cervi, G. M., Francesconi Mazetto, R., Faria, A. M. D., Vido, M., ... & Amorim, M. (2024). Impacts of feature selection on predicting machine failures by machine learning algorithms. Applied Sciences, 14(8), 3337. doi: 10.3390/app14083337.
Benatia, M. A., Hafsi, M., & Ayed, S. B. (2025). A continual learning approach for failure prediction under non-stationary conditions: Application to condition monitoring data streams. Computers & Industrial Engineering, 204, 111049. doi: 10.1016/j.cie.2025.111049.
Çiftpınar, A. B., Kanar, P., & Cıcek, Z. I. E. (2025). Failure Prediction Using Ensemble Learning: A Comparative Study with Synthetic and Real-World Datasets. Afyon Kocatepe Üniversitesi Fen Ve Mühendislik Bilimleri Dergisi, 25(4), 785-797. doi: 10.35414/akufemubid.1571811.
Esteban, A., Cano, A., Ventura, S., & Zafra, A. (2025). Simultaneous fault prediction in evolving industrial environments with ensembles of Hoeffding adaptive trees. Applied Intelligence, 55(13), 930. https://doi.org/10.1007/s10489-025-06786-7.
Gomaa, A. H. (2025). Advancing Total Productive Maintenance in Smart Manufacturing: From Methodology to Implementation. International Journal of Smart Manufacturing. doi: 10.70322/ism.2025.10019.
Kamel, H. (2022, July). Artificial intelligence for predictive maintenance. In Journal of Physics: Conference Series (Vol. 2299, No. 1, p. 012001). IOP Publishing. doi: 10.1088/1742-6596/2299/1/012001.
Kang, D. H. E., Kim, D., Song, Y., Lee, D., Kwak, H., & Anthony, B. W. (2024). Harnessing the potential of shared data in a secure, inclusive, and resilient manner via multi-key homomorphic encryption. Scientific Reports, 14(1), 13626. doi: 10.1038/s41598-024-63393-1.
Liu, C. L., & Su, H. C. (2024). Temporal learning in predictive health management using channel-spatial attention-based deep neural networks. Advanced Engineering Informatics, 62, 102604. doi: 10.1016/j.aei.2024.102604.
Lyubchyk, L., Grinberg, G., & Yamkovyi, K. (2023, October). Machine Learning-Based Predictive Maintenance using Data Aggregation via Regularized Clustering. In 2023 13th International Conference on Dependable Systems, Services and Technologies (DESSERT) (pp. 1-6). IEEE. doi: 10.1109/DESSERT61349.2023.10416542.
Muchiri, P., & Pintelon, L. (2008). Performance measurement using overall equipment effectiveness (OEE): literature review and practical application discussion. International journal of production research, 46(13), 3517-3535. doi: 10.1080/00207540601142645.
Naeini, M. P., Cooper, G., & Hauskrecht, M. (2015, February). Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI conference on artificial intelligence (Vol. 29, No. 1). doi: 10.1609/aaai.v29i1.9602.
Nunes, P., Santos, J., & Rocha, E. (2023). Challenges in predictive maintenance–A review. CIRP Journal of Manufacturing Science and Technology, 40, 53-67. doi: 10.1016/j.cirpj.2022.11.004.
Pinto, S. D. L., Muniz Jr, J., Freitas, C. R. D., & Dale Luche, J. R. (2025). A Framework for the Innovation Management Capacity: Empirical Evidence from the Porto Digital Cluster in Brazil. Administrative Sciences, 15(5), 191. doi:10.3390/admsci15050191.
Prashanth, B. S., Manoj Kumar, M. V., Almuraqab, N., & Puneetha, B. H. (2025). Leveraging Safe and Secure AI for Predictive Maintenance of Mechanical Devices Using Incremental Learning and Drift Detection. Computers, Materials & Continua, 83(3), 4979–4998. doi: 10.32604/cmc.2025.060881.
Presciuttini, A., Cantini, A., & Portioli-Staudacher, A. (2024, November). From Explanations to Actions: Leveraging SHAP, LIME, and Counterfactual Analysis for Operational Excellence in Maintenance Decisions. In 2024 4th International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME) (pp. 1-6). IEEE. doi: 10.1109/ICECCME62383.2024.10796106.
Ronzoni, N., De Marco, A., & Ronchieri, E. (2022, July). Predictive Maintenance Experiences on Imbalanced Data with Bayesian Optimization Approach. In International Conference on Computational Science and Its Applications (pp. 120-137). Cham: Springer International Publishing. doi: 10.1007/978-3-031-10536-4_9.
Shivaramu, P. (2025). Optimizing Manufacturing Processes with Predictive Maintenance Using Machine Learning and Lean Six Sigma. SSRN Electronic Journal. doi: 10.2139/ssrn.5161097.
Tortorella, G., Saurin, T. A., Fogliatto, F. S., Tlapa, D., Moyano-Fuentes, J., Gaiardelli, P., ... & Forstner, F. F. (2022). The impact of Industry 4.0 on the relationship between TPM and maintenance performance. Journal of Manufacturing Technology Management, 33(3), 489-520. doi: 10.1108/JMTM-10-2021-0399.
Saito, T., & Rehmsmeier, M. (2015). The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PloS one, 10(3), e0118432. doi: 10.1371/journal.pone.0118432.
Santos, M. S., Soares, J. P., Abreu, P. H., Araujo, H., & Santos, J. (2018). Cross-validation for imbalanced datasets: avoiding overoptimistic and overfitting approaches [research frontier]. IEEE Computational Intelligence Magazine, 13(4), 59-76. doi: 10.1109/MCI.2018.2866730.
UCI Machine Learning Repository. (2020). AI4I 2020 predictive maintenance dataset. Irvine, CA: University of California, School of Information and Computer Science. doi: 10.24432/C5HS5C.
Yürek, O. E., & Birant, D. (2024). A new approach: ordinal predictive maintenance with ensemble binary decomposition (OPMEB). Turkish Journal of Electrical Engineering and Computer Sciences, 32(4), 534-554. doi: 10.55730/1300-0632.4086.
Zonta, T., Da Costa, C. A., da Rosa Righi, R., de Lima, M. J., Da Trindade, E. S., & Li, G. P. (2020). Predictive maintenance in the Industry 4.0: A systematic literature review. Computers & industrial engineering, 150, 106889. doi: 10.1016/j.cie.2020.106889
Alshkeili, H. M. H. A., Almheiri, S. J., & Khan, M. A. (2025). Privacy-Preserving Interpretability: An Explainable Federated Learning Model for Predictive Maintenance in Sustainable Manufacturing and Industry 4.0. AI, 6(6), 117. doi: 10.3390/ai6060117.
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., ... & Herrera, F. (2020). Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information fusion, 58, 82-115. doi: 10.1016/j.inffus.2019.12.012.
Autran, J. V., Kuhn, V., Diguet, J. P., Dubois, M., & Buche, C. (2024). AI4I-PMDI: Predictive maintenance datasets with complex industrial settings’ irregularities. Procedia Computer Science, 246, 1201-1209. doi: 10.1016/j.procs.2024.09.546.
Bezerra, F. E., Oliveira Neto, G. C. D., Cervi, G. M., Francesconi Mazetto, R., Faria, A. M. D., Vido, M., ... & Amorim, M. (2024). Impacts of feature selection on predicting machine failures by machine learning algorithms. Applied Sciences, 14(8), 3337. doi: 10.3390/app14083337.
Benatia, M. A., Hafsi, M., & Ayed, S. B. (2025). A continual learning approach for failure prediction under non-stationary conditions: Application to condition monitoring data streams. Computers & Industrial Engineering, 204, 111049. doi: 10.1016/j.cie.2025.111049.
Çiftpınar, A. B., Kanar, P., & Cıcek, Z. I. E. (2025). Failure Prediction Using Ensemble Learning: A Comparative Study with Synthetic and Real-World Datasets. Afyon Kocatepe Üniversitesi Fen Ve Mühendislik Bilimleri Dergisi, 25(4), 785-797. doi: 10.35414/akufemubid.1571811.
Esteban, A., Cano, A., Ventura, S., & Zafra, A. (2025). Simultaneous fault prediction in evolving industrial environments with ensembles of Hoeffding adaptive trees. Applied Intelligence, 55(13), 930. https://doi.org/10.1007/s10489-025-06786-7.
Gomaa, A. H. (2025). Advancing Total Productive Maintenance in Smart Manufacturing: From Methodology to Implementation. International Journal of Smart Manufacturing. doi: 10.70322/ism.2025.10019.
Kamel, H. (2022, July). Artificial intelligence for predictive maintenance. In Journal of Physics: Conference Series (Vol. 2299, No. 1, p. 012001). IOP Publishing. doi: 10.1088/1742-6596/2299/1/012001.
Kang, D. H. E., Kim, D., Song, Y., Lee, D., Kwak, H., & Anthony, B. W. (2024). Harnessing the potential of shared data in a secure, inclusive, and resilient manner via multi-key homomorphic encryption. Scientific Reports, 14(1), 13626. doi: 10.1038/s41598-024-63393-1.
Liu, C. L., & Su, H. C. (2024). Temporal learning in predictive health management using channel-spatial attention-based deep neural networks. Advanced Engineering Informatics, 62, 102604. doi: 10.1016/j.aei.2024.102604.
Lyubchyk, L., Grinberg, G., & Yamkovyi, K. (2023, October). Machine Learning-Based Predictive Maintenance using Data Aggregation via Regularized Clustering. In 2023 13th International Conference on Dependable Systems, Services and Technologies (DESSERT) (pp. 1-6). IEEE. doi: 10.1109/DESSERT61349.2023.10416542.
Muchiri, P., & Pintelon, L. (2008). Performance measurement using overall equipment effectiveness (OEE): literature review and practical application discussion. International journal of production research, 46(13), 3517-3535. doi: 10.1080/00207540601142645.
Naeini, M. P., Cooper, G., & Hauskrecht, M. (2015, February). Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI conference on artificial intelligence (Vol. 29, No. 1). doi: 10.1609/aaai.v29i1.9602.
Nunes, P., Santos, J., & Rocha, E. (2023). Challenges in predictive maintenance–A review. CIRP Journal of Manufacturing Science and Technology, 40, 53-67. doi: 10.1016/j.cirpj.2022.11.004.
Pinto, S. D. L., Muniz Jr, J., Freitas, C. R. D., & Dale Luche, J. R. (2025). A Framework for the Innovation Management Capacity: Empirical Evidence from the Porto Digital Cluster in Brazil. Administrative Sciences, 15(5), 191. doi:10.3390/admsci15050191.
Prashanth, B. S., Manoj Kumar, M. V., Almuraqab, N., & Puneetha, B. H. (2025). Leveraging Safe and Secure AI for Predictive Maintenance of Mechanical Devices Using Incremental Learning and Drift Detection. Computers, Materials & Continua, 83(3), 4979–4998. doi: 10.32604/cmc.2025.060881.
Presciuttini, A., Cantini, A., & Portioli-Staudacher, A. (2024, November). From Explanations to Actions: Leveraging SHAP, LIME, and Counterfactual Analysis for Operational Excellence in Maintenance Decisions. In 2024 4th International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME) (pp. 1-6). IEEE. doi: 10.1109/ICECCME62383.2024.10796106.
Ronzoni, N., De Marco, A., & Ronchieri, E. (2022, July). Predictive Maintenance Experiences on Imbalanced Data with Bayesian Optimization Approach. In International Conference on Computational Science and Its Applications (pp. 120-137). Cham: Springer International Publishing. doi: 10.1007/978-3-031-10536-4_9.
Shivaramu, P. (2025). Optimizing Manufacturing Processes with Predictive Maintenance Using Machine Learning and Lean Six Sigma. SSRN Electronic Journal. doi: 10.2139/ssrn.5161097.
Tortorella, G., Saurin, T. A., Fogliatto, F. S., Tlapa, D., Moyano-Fuentes, J., Gaiardelli, P., ... & Forstner, F. F. (2022). The impact of Industry 4.0 on the relationship between TPM and maintenance performance. Journal of Manufacturing Technology Management, 33(3), 489-520. doi: 10.1108/JMTM-10-2021-0399.
Saito, T., & Rehmsmeier, M. (2015). The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PloS one, 10(3), e0118432. doi: 10.1371/journal.pone.0118432.
Santos, M. S., Soares, J. P., Abreu, P. H., Araujo, H., & Santos, J. (2018). Cross-validation for imbalanced datasets: avoiding overoptimistic and overfitting approaches [research frontier]. IEEE Computational Intelligence Magazine, 13(4), 59-76. doi: 10.1109/MCI.2018.2866730.
UCI Machine Learning Repository. (2020). AI4I 2020 predictive maintenance dataset. Irvine, CA: University of California, School of Information and Computer Science. doi: 10.24432/C5HS5C.
Yürek, O. E., & Birant, D. (2024). A new approach: ordinal predictive maintenance with ensemble binary decomposition (OPMEB). Turkish Journal of Electrical Engineering and Computer Sciences, 32(4), 534-554. doi: 10.55730/1300-0632.4086.
Zonta, T., Da Costa, C. A., da Rosa Righi, R., de Lima, M. J., Da Trindade, E. S., & Li, G. P. (2020). Predictive maintenance in the Industry 4.0: A systematic literature review. Computers & industrial engineering, 150, 106889. doi: 10.1016/j.cie.2020.106889
Section
Technical Papers