Evaluation of Multi-Modal Learning for Predicting Coolant Pump Failures in Heavy Duty Vehicles



Published Sep 4, 2023
Yuantao Fan Amine Atoui Slawomir Nowaczyk Thorsteinn Rognvaldsson


Coolant Pump failures in heavy-duty vehicles can cause severe collateral damage if they are not detected and resolved in time; the engine will overheat quickly, rendering the vehicle inoperable. Nowadays, a vast amount of heterogeneous sensor data from different sources is being collected in the automotive industry. Such multi-modal data include onboard signals reflecting the overall usage of the vehicle, multi-dimensional histograms that capture the relation between physical quantities, and categorical variables that encode the physical configuration of the vehicle. This work evaluates several multi-modal learning approaches leveraging this diverse data to build a prognosis and health management system for coolant pumps in commercial heavy-duty vehicles. Four auto-encoder architectures are examined to extract features from 2D histograms. These trained models are anticipated to capture key characteristics of the healthy system operation and yield large reconstruction errors when applied on faulty, or near end-of-life samples. Such learned representations are then combined with expert-engineered features. Both early and intermediate fusion are evaluated on a real-world coolant pump replacement dataset. Results indicate that the combination of diverse features was the most effective approach, thereby motivating further research on multimodal methods.  

Abstract 151 | PDF Downloads 118



Multi-modal Learning, Deep Autoencoders, Coolant Pump Failure, Predictive Maintenance, Prognostics and Health Management

Al-Dulaimi, A., Zabihi, S., Asif, A., & Mohammadi, A. (2019). A multimodal and hybrid deep neural network model for remaining useful life estimation. Computers in industry, 108, 186–196.

Chen, J., Li, D., Huang, R., Chen, Z., & Li, W. (2023). Aero-engine remaining useful life prediction method with self-adaptive multimodal data fusion and clusterensemble transfer regression. Reliability Engineering & System Safety, 234, 109151.

Guo, W., Wang, J., & Wang, S. (2019). Deep multimodal representation learning: A survey. IEEE Access, 7, 63373–63394.

Huang, Y., Tao, J., Sun, G., Wu, T., Yu, L., & Zhao, X. (2023). A novel digital twin approach based on deep multimodal information fusion for aero-engine fault diagnosis. Energy, 270, 126894.

Inceoglu, A., Aksoy, E. E., Ak, A. C., & Sariel, S. (2021). Fino-net: A deep multimodal sensor fusion framework for manipulation failure detection. In 2021 ieee/rsj international conference on intelligent robots and systems (iros) (pp. 6841–6847).

Li, C., Sanchez, R.-V., Zurita, G., Cerrada, M., Cabrera, D., & Vasquez, R. E. (2015). Multimodal deep support ́ vector classification with homologous features and its application to gearbox fault diagnosis. Neurocomputing, 168, 119–127.

Li, H., Huang, J., Huang, J., Chai, S., Zhao, L., & Xia, Y. (2021). Deep multimodal learning and fusion based intelligent fault diagnosis approach. Journal of Beijing Institute of Technology, 30(2), 172–185.

Pang, L., Zhu, S., & Ngo, C.-W. (2015). Deep multimodal learning for affective analysis and retrieval. IEEE Transactions on Multimedia, 17(11), 2008–2020.

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., . . . Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.

Ramachandram, D., & Taylor, G. W. (2017). Deep multimodal learning: A survey on recent advances and trends. IEEE signal processing magazine, 34(6), 96– 108.
Regular Session Papers