Echo State Network for the Remaining Useful Life Prediction of a Turbofan Engine



Published Jul 5, 2016
Marco Rigamonti Piero Baraldi Enrico Zio Indranil Roychoudhury Kai Goebel Scott Poll


Among the various data-driven approaches used for RUL prediction, Recurrent Neural Networks (RNNs) have certain prima facie advantages over other approaches because the connections between internal nodes form directed cycles, thus creating internal states which enables the network to encapsulate dynamic temporal behavior and also to properly handle the noise affecting the collected signals. However, the application of traditional RNNs is limited by the difficulty of optimizing their numerous internal parameters and the significant computational effort associated with the training process. In this work, we explore the use of the Echo State Network (ESN), a relatively new type of Recurrent Neural Network (RNN). One of the main advantages of ESN is the training procedure, which is based on a simple linear regression. Unlike traditional RNNs, ESNs can be trained with fairly little computational effort, while still providing the generalization capability characteristic of RNNs. In this paper, we use Differential Evolution (DE) for the optimization of the ESN architecture for RUL prediction of a turbofan engine working under variable operating conditions. A procedure for pre-processing of the monitored signals and for identification of the onset of acceleration of degradation (i.e., the so-called elbow point in the degradation trend) will be shown. The datasets used to validate the approach have been taken from the NASA Ames Prognostics CoE Data Repository. These datasets were generated using a turbofan engine simulator, based on a detailed physical model that allows input variations of health-related parameters under variable operating conditions and records values from some specific sensor measurements. The results obtained on these data confirm the ESN’s capability to provide accurate RUL predictions.

How to Cite

Rigamonti, M., Baraldi, P., Zio, E., Roychoudhury, I., Goebel, K., & Poll, S. (2016). Echo State Network for the Remaining Useful Life Prediction of a Turbofan Engine. PHM Society European Conference, 3(1).
Abstract 584 | PDF Downloads 292



data driven prognostics, recurrent neural networks, CMAPPS datasets, echo state networks, differential evolution

Bezdek, J. C., Ehrlich, R., Full, W., (1984). FCM: The fuzzy c-means clustering algorithm. Computers & Geosciences 10.2: pp. 191-203. Bonissone, P. P., Varma, A., (2005). Predicting the best units within a fleet: prognostic capabilities enabled by peer learning, fuzzy similarity, and evolutionary design process. The 14th IEEE International Conference on Fuzzy Systems, 2005 (pp. 312-318).
Bonissone, P.P., Xue, F., Subbu, R., (2011). Fast meta-models for local fusion of multiple predictive models. Applied Soft Computing Journal, 11 (2), pp. 1529-1539.
Chen, S.J., Hwang, C.L., (1992). Fuzzy Multiple Attribute Decision Making: Methods and Applications. Springer- Verlag, Berlin.
Coble, J. B. (2010). Merging data sources to predict remaining useful life–an automated method to identify prognostic parameters. Doctoral dissertation.
Daigle, M.J., Roychoudhury, I., Biswas, G., Koutsoukos, X.D., Patterson-Hine, A., Poll, S., (2010). A comprehensive diagnosis methodology for complex hybrid systems: A case study on spacecraft power distribution systems. IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans, 40 (5), art. no. 5504182, pp. 917-931.
Ferreira, A.A., Ludermir, T.B. (2009). Genetic algorithm for reservoir computing optimization. Proceedings of the international joint conference on neural networks – IJCNN 2009, Atlanta (pp. 811–815).
Ferreira, A.A., Ludermir, T.B., De Aquino, R.R.B., (2013). An approach to reservoir computing design and training. Expert Systems with Applications, 40 (10), pp. 4172-4182.
Fink, O., Zio, E., Weidmann, U., (2013). Predicting time series of railway speed restrictions with time-dependent machine learning techniques. Expert Systems with Applications, 40 (15), pp. 6033-6040.
Fink, O., Weidmann, U., Zio, E., (2014). Extreme learning machines for predicting operation disruption events in railway systems. Safety, Reliability and Risk Analysis: Beyond the Horizon - Proceedings of the European Safety and Reliability Conference, ESREL 2013, pp. 1781-1787.
Frederick, D., DeCastro, J., Litt, J., (2007). User's Guide for the Commercial Modular Aero-Propulsion System Simulation (CMAPSS). NASA/ARL, Technical Manual TM 2007-215026.
Gasperin, M., Boskoski, P., Juricic, D., (2011). Model-based prognostics under non-stationary operating conditions. In Annual Conference of the Prognostics and Health Management Society (pp. 831-853).
Heimes, F.O., (2008). Recurrent neural networks for remaining useful life estimation. 2008 International Conference on Prognostics and Health Management, PHM 2008, art. n. 4711422.
Hu, Y., (2015). Development of prognostics and health management methods for engineering systems operating in evolving environments. PhD Thesis, Politecnico di Milano, 2015.
Huang, G.-B., Zhu, Q.-Y., Siew, C.-K., (2006). Extreme learning machine: Theory and applications. Neurocomputing, 70 (1-3), pp. 489-501.
Jaeger, H., (2001). The echo state approach to analyzing and training recurrent neural networks. Technical Report GMD Report 148, German National Research Center for Information Technology.
Jaeger, H., (2002). A Tutorial on training recurrent neural networks, covering BPTT, RTRL, EKF and the Echo state network approach. Technical Report GMD Report 159, German National Research Center for Information Technology.
Jaeger, H., Haas, H., (2004). Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication. Science, 304 (5667), pp. 78-80.
Li, D., Han, M., Wang, J., (2012). Chaotic time series prediction based on a novel robust echo state network. IEEE Transactions on Neural Networks and Learning Systems, 23 (5), art. no. 6177672, pp. 787-797.
Li, X., Qian, J., Wang, G. (2013). Fault prognostic based on hybrid method of state judgment and regression. Advances in Mechanical Engineering, 2013(149562), 1-10.
Liu, J., Saxena, A., Goebel, K., Saha, B., Wang, W., (2010). An adaptive recurrent neural network for remaining useful life prediction of lithium-ion batteries. Annual Conference of the Prognostics and Health Management Society, PHM 2010.
Lukoševičius, M., Jaeger, H., (2009). Reservoir computing approaches to recurrent neural network training. Computer Science Review, 3 (3), pp. 127-149.
Malhi, A., Yan, R., Gao, R.X., (2011). Prognosis of defect propagation based on recurrent neural networks. IEEE Transactions on Instrumentation and Measurement, 60 (3), art. no. 5710193, pp. 703-711.
Morando, S., Jemei, S., Gouriveau, R., Zerhouni, N., Hissel, D., (2013). Fuel Cells prognostics using echo state network. IECON Proceedings (Industrial Electronics Conference), art. no. 6699377, pp. 1632-1637.
Moustapha, A. I., Selmic, R. R., (2008). Wireless sensor network modeling using modified recurrent neural networks: Application to fault detection. IEEE Trans. Instrum. Meas., vol. 57, no. 5, pp. 981–988.
Olivares, B.E., Cerda Muñoz, M.A., Orchard, M.E., Silva, J.F., (2013). Particle-filtering-based prognosis framework for energy storage devices with a statistical characterization of state-of-health regeneration phenomena. IEEE Transactions on Instrumentation and Measurement, 62 (2), art. no. 6302189, pp. 364-376.
Opricovic, S., Tzeng, G.-H., (2004). Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS. European Journal of Operational Research, 156 (2), pp. 445-455.
Pecht, M.G., (2008). Prognostics and Health Management of Electronics. Prognostics and Health Management of Electronics, pp. 1-315.
Peng, Y., Wang, H., Wang, J., Liu, D., Peng, X., (2012a). A modified echo state network based remaining useful life estimation approach. PHM 2012 - 2012 IEEE Int. Conf. on Prognostics and Health Management: Enhancing Safety, Efficiency, Availability, and Effectiveness of Systems Through PHM Technology and Application, Conference Program, art. no. 6299524.
Peng, Y., Xu, Y., Liu, D., Peng, X. (2012b). Sensor selection with grey correlation analysis for remaining useful life evaluation. In Annual Conference of the PHM Society.
Qu, J., Zuo, M.J., (2012). An LSSVR-based algorithm for online system condition prognostics. Expert Systems with Applications, 39 (5), pp. 6089-6102.
Rabin, M.J.A., Hossain, M.S., Ahsan, M.S., Mollah, M.A.S., Rahman, M.T., (2013). Sensitivity learning oriented non-monotonic multi reservoir echo state network for short-term load forecasting. 2013 International Conference on Informatics, Electronics and Vision, ICIEV 2013, art. no. 6572692.
Ramasso E., Saxena, A., (2014). Performance Benchmarking and Analysis of Prognostic Methods for CMAPSS Datasets. International Journal of Prognostics and Health Management, , 2014, 5 (2), pp.1-15.
Ramasso, E., (2014). Investigating computational geometry for failure prognostics. International Journal of Prognostics and Health Management, 2014, 5(5), pp 1-18.
Samanta, B., Al-Balushi, K., (2003). Artificial neural network based fault diagnostics of rolling element bearings using time-domain features. Mechanical System Signal Processing, vol. 17, no. 2, pp. 317–328.
Saxena, A., Goebel, K., Simon, D., Eklund, N., (2008). Damage propagation modeling for aircraft engine run-to-failure simulation. 2008 International Conference on Prognostics and Health Management, PHM 2008, art. no. 4711414.
Saxena, A., Celaya, J., Saha, B., Saha, S., Goebel, K., (2010). Metrics for offline evaluation of prognostic performance. International Journal of Prognostics and Health Management, 1 (1).
Shi, Z., Han, M., (2007). Support vector echo-state machine for chaotic time-series prediction. IEEE Transactions on Neural Networks, 18 (2), pp. 359-372.
Storn, R., Price, K., (1997). Differential Evolution - A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces. Journal of Global Optimization, 11 (4), pp. 341-359.
Tse, P.W., Atherton, D.P., (1999). Prediction of machine deterioration using vibration based fault trends and recurrent neural networks. Journal of Vibration and Acoustics, Transactions of the ASME, 121 (3), pp. 355-362.
Vachtsevanos, G., Lewis, F. L., Roemer, M., Hess, A., Wu, B., (2006). Intelligent Fault Diagnosis and Prognosis for Engineering Systems, Wiley, New York.
Vukicevic, A.M., Jovicic, G.R., Stojadinovic, M.M., Prelevic, R.I., Filipovic, N.D., (2014). Evolutionary assembled neural networks for making medical decisions with minimal regret: Application for predicting advanced bladder cancer outcome. Expert Systems with Applications, 41 (18), pp. 8092-8100.
Wang, T., (2010). Trajectory similarity based prediction for remaining useful life estimation. PhD Thesis, University of Cincinnati, 2010.
Yan, T., Duwu, D., Yongqing, T., (2007). A new evolutionary neural network algorithm based on improved genetic algorithm and its application in power transformer fault diagnosis. 2nd International Conference on Bio-Inspired Computing: Theories and Applications, BICTA 2007, art. no. 4806406, pp. 1-5.
Yildiz, I.B., Jaeger, H., Kiebel, S.J., (2012). Re-visiting the echo state property. Neural Networks, 35, pp. 1-9.
Zio, E., Broggi, M., Pedroni, N., (2009). Nuclear reactor dynamics on-line estimation by Locally Recurrent Neural Networks. Progress in Nuclear Energy, 51 (3), pp. 573-581.
Zio, E., Di Maio, F., Stasi, M., (2010). A data-driven approach for predicting failure scenarios in nuclear systems. Annals of Nuclear Energy, 37 (4), pp. 482-491.
Zitzler, E., Thiele, L., (1999). Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach. IEEE Transactions on Evolutionary Computation, 3 (4), pp. 257-271.
Technical Papers