Leaf It to Renewal Improved Predictive Maintenance Policies via Renewal Theory and Decision Trees
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Abstract
We propose a hybrid planning method for deriving prognostics-based predictive maintenance policies. The method accounts for the available decision options, the information on the future state of the system provided by a prognostic model, and the costs of the underlying renewal-reward process. It results in policies defined by only a few parameters, which can be determined based on theoretical considerations or by optimization from run-to-failure data. We demonstrate the potential of the method in two separate predictive maintenance decision settings: preventive replacement and preventive ordering. Numerical investigations show that the derived policies rival the performance of optimized benchmark policies, while being significantly more efficient and robust against overfitting.
##plugins.themes.bootstrap3.article.details##
Predictive maintenance, Heuristic policies, Online planning, Renewal theory, Decision trees
Asgarpour, M., & Sørensen, J. D. (2018). Bayesian-based Prognostic Model for Predictive Maintenance of Offshore Wind Farms. International Journal of Prognostics and Health Management, 9(1).
Bellman, R. (1966). Dynamic Programming. Science, 153(3731), 34–37.
Bismut, E., & Straub, D. (2021). Optimal Adaptive Inspection and Maintenance Planning for Deteriorating Structural Systems. Reliability Engineering & System Safety, 215, 107891.
Cha, J. H., Finkelstein, M., & Levitin, G. (2017). On preventive maintenance of systems with lifetimes dependent on a random shock process. Reliability Engineering & System Safety, 168, 90–97.
Chao, M.A., Kulkarni, C., Goebel, K., & Fink, O. (2022). Fusing physics-based and deep learning models for prognostics. Reliability Engineering & System Safety, 217, 107961.
Chelbi, A., & Aït-Kadi, D. (2001). Spare provisioning strategy for preventively replaced systems subjected to random failure. International Journal of Production Economics, 74(1-3), 183–189.
Chen, C., Shi, J., Lu, N., Zhu, Z. H., & Jiang, B. (2022). Data-driven predictive maintenance strategy considering the uncertainty in remaining useful life prediction. Neurocomputing, 494, 79–88.
Chien, Y.-H., Sheu, S.-H., & Zhang, Z. G. (2012). Optimal maintenance policy for a system subject to damage in a discrete time process. Reliability Engineering & System Safety, 103, 1–10.
De Jonge, B., & Scarf, P. A. (2020). A review on maintenance optimization. European Journal of Operational Research, 285(3), 805–824.
Dong, E., Zhan, X., Yan, H., Tan, S., Bai, Y., Wang, R., & Cheng, Z. (2025). A data-driven intelligent predictive maintenance decision framework for mechanical systems integrating transformer and kernel density estimation. Computers & Industrial Engineering, 201, 110868.
Doob, J. L. (1948). Renewal Theory From the Point of View of the Theory of Probability. Transactions of the American Mathematical Society, 63(3), 422–438.
Elwany, A. H., & Gebraeel, N. Z. (2008). Sensor-driven prognostic models for equipment replacement and spare parts inventory. IIE Transactions, 40(7), 629–639.
Ethier, S. N., & Kurtz, T. G. (2009). Markov Processes: Characterization and Convergence. John Wiley & Sons.
Fauriat, W., & Zio, E. (2020). Optimization of an aperiodic sequential inspection and condition-based maintenance policy driven by value of information. Reliability Engineering & System Safety, 204, 107133.
Finkelstein, M. (2015). On the optimal degree of imperfect repair. Reliability Engineering & System Safety, 138, 54–58.
Gneiting, T., Balabdaoui, F., & Raftery, A. E. (2007). Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society Series B: Statistical Methodology, 69(2), 243–268.
Grall, A., Dieulle, L., Bérenguer, C., & Roussignol, M. (2006). Asymptotic failure rate of a continuously monitored system. Reliability Engineering & System Safety, 91(2), 126–130.
Grimmett, G., &Stirzaker, D. (2020). Probability and Random Processes. Oxford University Press.
Guo, C., Wang, W., Guo, B., & Si, X. (2013). A maintenance optimization model for mission-oriented systems based on Wiener degradation. Reliability Engineering &System Safety, 111, 183–194.
He, R., Tian, Z., Wang, Y., Zuo, M., & Guo, Z. (2023). Condition-based maintenance optimization for multicomponent systems considering prognostic information and degraded working efficiency. Reliability Engineering & System Safety, 234, 109167.
Hendrycks, D., & Gimpel, K. (2016). Gaussian Error Linear Units (GELUs). arXiv preprint arXiv:1606.08415.
Hinderer, K., & Waldmann, K.-H. (1999). Approximate solution of Markov renewal programs with finite time horizon. SIAM journal on control and optimization, 37(2), 502–520.
Hodkiewicz, M., Bonser, D., Chin, A., & While, L. (2018). Optimizing fixed-interval maintenance periods for mobile asset sub-systems operating in remote locations. In Phm society european conference (Vol. 4).
Hoffman, J. D., & Frankel, S. (2018). Numerical Methods for Engineers and Scientists. CRC press.
Jack, N., Iskandar, B. P., & Murthy, D. P. (2009). A repair-replace strategy based on usage rate for items sold with a two-dimensional warranty. Reliability Engineering & System Safety, 94(2), 611–617.
Jiang, R. (2009). An accurate approximate solution of optimal sequential age replacement policy for a finite-time horizon. Reliability Engineering & System Safety, 94(8), 1245–1250.
Jimenez-Roa, L. A., Simão, T. D., Bukhsh, Z., Tinga, T., Molegraaf, H., Jansen, N., & Stoelinga, M. (2024). Maintenance Strategies for Sewer Pipes with Multi-State Degradation and Deep Reinforcement Learning. arXiv preprint arXiv:2407.12894.
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998). Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1-2), 99–134.
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research, 4, 237–285.
Kamariotis, A., Tatsis, K., Chatzi, E., Goebel, K., & Straub, D. (2024). A metric for assessing and optimizing data-driven prognostic algorithms for predictive maintenance. Reliability Engineering & System Safety, 242, 109723.
Kim, N.-H., An, D., & Choi, J.-H. (2017). Prognostics and Health Management of Engineering Systems. Switzerland: Springer International Publishing.
Kingma, D. P., & Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv preprint arXiv:1412.6980.
Kochenderfer, M. J. (2015). Decision Making Under Uncertainty: Theory and Application. MIT press.
Kochenderfer, M. J., Wheeler, T. A., & Wray, K. H. (2022). Algorithms for Decision Making. MIT press.
Koutas, D., Bismut, E., & Straub, D. (2024). An investigation of belief-free DRL and MCTS for inspection and maintenance planning. Journal of Infrastructure Preservation and Resilience, 5(1), 6.
Kuipers, L., & Niederreiter, H. (2012). Uniform Distribution of Sequences. Courier Corporation.
Lee, J., & Mitici, M. (2023). Deep reinforcement learning for predictive aircraft maintenance using probabilistic remaining-useful-life prognostics. Reliability Engineering & System Safety, 230, 108908.
Lei, X., & Sandborn, P. A. (2016). PHM-Based Wind Turbine Maintenance Optimization Using Real Options. Int J Progn Health Manag, 7(1), 1–14.
Luque, J., & Straub, D. (2019). Risk-based optimal inspection strategies for structural systems using dynamic Bayesian networks. Structural Safety, 76, 68–80.
Mercier, S., & Castro, I. (2013). On the modelling of imperfect repairs for a continuously monitored gamma wear process through age reduction. Journal of Applied Probability, 50(4), 1057–1076.
Mitici, M., de Pater, I., Barros, A., & Zeng, Z. (2023). Dynamic predictive maintenance for multiple components using data-driven probabilistic RUL prognostics: The case of turbofan engines. Reliability Engineering & System Safety, 234, 109199.
Nguyen, K. T., & Medjaher, K. (2019). A new dynamic predictive maintenance framework using deep learning for failure prognostics. Reliability Engineering & System Safety, 188, 251–262.
Nielsen, J. S., & Sørensen, J. D. (2018). Computational framework for risk-based planning of inspections, maintenance and condition monitoring using discrete Bayesian networks. Structure and Infrastructure Engineering, 14(8), 1082–1094.
Pecht, M. (2009). Prognostics and Health Management of Electronics. Encyclopedia of Structural Health Monitoring.
Pineau, J., Gordon, G., & Thrun, S. (2006). Anytime Point-Based Approximations for Large POMDPs. Journal of Artificial Intelligence Research, 27, 335–380.
Prakash, G., Narasimhan, S., & Pandey, M. (2017). Condition-Based Maintenance of Low Speed Rolling Element Bearings using Hidden Markov Model. International Journal of Prognostics and Health Management, 8(1).
Raiffa, H., & Schlaifer, R. (2000). Applied Statistical Decision Theory (Vol. 78). John Wiley & Sons.
Ran, Y., Zhou, X., Lin, P., Wen, Y., & Deng, R. (2019). A Survey of Predictive Maintenance: Systems, Purposes and Approaches. arXiv preprint arXiv:1912.07383, 1-36.
Reddi, S. J., Kale, S., & Kumar, S. (2019). On the Convergence of Adam and Beyond. arXiv preprint arXiv:1904.09237.
Saassouh, B., Dieulle, L., & Grall, A. (2007). Online maintenance policy for a deteriorating system with random change of mode. Reliability Engineering & System Safety, 92(12), 1677–1685.
Saxena, A., Goebel, K., Simon, D., & Eklund, N. (2008). Damage Propagation Modeling for Aircraft Engine Run-to-Failure Simulation. In 2008 international conference on prognostics and health management (pp. 1–9).
Shukla, L. (2023, 11 16). Bayesian Hyperparameter Optimization- A Primer. Retrieved from https://wandb.ai/wandb_fc/articles/reports/Bayesian-Hyperparameter-Optimization-A-Primer--Vmlldzo1NDQyNzcw (Accessed 05-August-2025)
Si, X.-S., Wang, W., Hu, C.-H., & Zhou, D.-H. (2011). Remaining useful life estimation–A review on the statistical data driven approaches. European Journal of Operational Research, 213(1), 1–14.
Sidibe, I., Khatab, A., Diallo, C., & Kassambara, A. (2017). Preventive maintenance optimization for a stochastically degrading system with a random initial age. Reliability Engineering & System Safety, 159, 255–263.
Smith, W. L. (1958). Renewal Theory and Its Ramifications. Journal of the Royal Statistical Society Series B: Statistical Methodology, 20(2), 243–284.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research, 15(1), 1929–1958.
Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: An Introduction (Vol. 1(1)) (No. 1). MIT press Cambridge.
Van der Weide, J., Suyono, & Van Noortwijk, J. (2008). Rene al theory with exponential and hyperbolic discounting. Probability in the Engineering and Informational Sciences, 22(1), 53–74.
Wang, L., Chen, Y., Zhao, X., & Xiang, J. (2024). Predictive Maintenance Scheduling for Aircraft Engines Based on Remaining Useful Life Prediction. IEEE Internet of Things Journal.
Wang, L., Zhu, Z., & Zhao, X. (2024). Dynamic predictive maintenance strategy for system remaining useful life prediction via deep learning ensemble method. Reliability Engineering & System Safety, 245, 110012.
Wang, W., &Wang,H. (2015). Preventive replacement for systems with condition monitoring and additional manual inspections. European Journal of Operational Research, 247(2), 459–471.
Wang, Y., Wang, G., Wu, Y., Zhang, G., & Wu, M. (2025). An uncertainty-aware deep learning ensemble approach for effective cutting tool predictive maintenance decision making. Measurement Science and Technology, 36(2), 026116.
Wang, Z., Hu, C., Wang, W., Kong, X., & Zhang, W. (2015). A prognostics-based spare part ordering and system replacement policy for a deteriorating system subjected to a random lead time. International Journal of Production Research, 53(15), 4511–4527.
Wu, F., Niknam, S. A., & Kobza, J. E. (2015). A cost-effective degradation-based maintenance strategy under imperfect repair. Reliability Engineering & System Safety, 144, 234–243.
Yan, T., Lei, Y., Li, N., Si, X., Pintelon, L., & Dewil, R. (2022). Online joint replacement-order optimization driven by a nonlinear ensemble remaining useful life prediction method. Mechanical Systems and Signal Processing, 173, 109053.
Zhuang, L., Xu, A., & Wang, X.-L. (2023). A prognostic-driven predictive maintenance framework based on Bayesian deep learning. Reliability Engineering & System Safety, 234, 109181.