IndusDiff: A Latent Diffusion Framework for Industrial Audio Generation for Data-Scarce PHM
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Abstract
Prognostics and Health Management (PHM) systems increasingly rely on data-driven models for fault diagnosis and anomaly detection, yet their effectiveness is fundamentally constrained by the scarcity and imbalance of labeled fault condition data. This challenge is particularly acute in industrial audio monitoring, where failure events are rare, hazardous to induce, and highly variable across operating conditions. While recent advances in generative modeling offer a promising pathway for data augmentation, existing audio generation methods either incur high computational cost in waveform-domain diffusion models or fail to preserve phase-sensitive characteristics critical for capturing transient fault signatures.
To address these challenges, this paper proposes IndusDiff, a phase-aware latent diffusion framework for high-fidelity industrial audio generation tailored for PHM applications. The proposed approach combines a phase-aware convolutional autoencoder with a latent diffusion model to enable efficient and physically meaningful audio synthesis. The autoencoder is trained using a multi-resolution spectral loss that jointly enforces waveform fidelity, spectral consistency, and temporal phase coherence, thereby preserving diagnostically relevant features such as transients, harmonics, and high-frequency components in the latent representation. Diffusion is then performed in this compressed latent space, significantly reducing computational complexity while maintaining generation quality.
Unlike conventional latent diffusion methods that primarily focus on perceptual realism, the proposed framework explicitly addresses the unique characteristics of industrial audio signals, including non-stationarity, multi-scale temporal dynamics, and phase-sensitive fault signatures. By preserving both magnitude and phase information, the model generates synthetic audio that is not only perceptually realistic but also structurally consistent with real machine signals, making it suitable for downstream PHM tasks.
The framework is evaluated on two complementary datasets: a publicly available industrial motor dataset and an in-house dataset capturing multi-stage assembly operations. Generation quality is assessed using the Frechet Audio Distance (FAD) as the primary quantitative metric, along with waveform and spectrogram analyses for qualitative validation. Experimental results demonstrate that IndusDiff produces statistically consistent, high-fidelity industrial audio while achieving significant improvements in computational efficiency, generating 48 kHz audio samples lasting several seconds in just seconds on modern GPU hardware.
How to Cite
##plugins.themes.bootstrap3.article.details##
Latent Diffusion, Industrial Audio Generation, Data-Scarce PHM
Choi, H.-S., Kim, J.-H., Huh, J., Kim, A., Ha, J.-W., & Lee, K. (2019). Phase-aware speech enhancement with deep complex U-Net. In Proceedings of the international conference on learning representations (iclr).
Chung, Y., Lee, J., & Nam, J. (2024). T-FOLEY: A controllable waveform-domain diffusion model for temporal-event-guided foley sound synthesis. In Proceedings of the ieee international conference on acoustics, speech and signal processing (icassp). doi: 10.1109/ICASSP48485.2024.10447898
Ding, Y., Ma, L., Ma, J., Wang, M., & Lu, C. (2023). A survey on deep learning for machinery fault diagnosis and prognosis. IEEE Transactions on Instrumentation and Measurement, 72, 1–20.
Evans, Z., Parker, J. D., Simon, C. J., Carr, C. J., Zukowski, Z., & Abela, J. (2024). Stable audio open. arXiv preprint arXiv:2407.14358.
Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., . . . Bengio, Y. (2014). Generative adversarial nets. In Advances in neural information processing systems (neurips) (Vol. 27, pp. 2672–2680).
Griffin, D. W., & Lim, J. S. (1984). Signal estimation from modified short-time Fourier transform. IEEE Transactions on Acoustics, Speech, and Signal Processing, 32(2), 236–243. doi: 10.1109/TASSP.1984.1164317
Grollmisch, S., & Abeßer, J. (2021). Audio-based machine fault diagnosis with WaveNet and data augmentation. In Proceedings of the 9th european workshop on structural health monitoring (ewshm).
Grollmisch, S., Abeßer, J., Liebetrau, J., & Lukashevich, H. (2019). Sounding industry: Challenges and datasets for industrial sound analysis. In Proceedings of the 27th european signal processing conference (eusipco) (pp. 1–5). doi: 10.23919/EUSIPCO.2019.8902941
Gui, A., Gamper, H., Braun, S., & Emmanouilidou, D. (2024). Adapting Fr´echet audio distance for generative music evaluation. In Proceedings of the ieee international conference on acoustics, speech and signal processing (icassp) (pp. 1331–1335). doi: 10.1109/ICASSP48485.2024.10446663
Hershey, S., Chaudhuri, S., Ellis, D. P. W., Gemmeke, J. F., Jansen, A., Moore, R. C., . . . Wilson, K. (2017). CNN architectures for large-scale audio classification. In Proceedings of the ieee international conference on acoustics, speech and signal processing (icassp) (pp. 131–135). doi: 10.1109/ICASSP.2017.7952132
Ho, J., Jain, A., & Abbeel, P. (2020). Denoising diffusion probabilistic models. In Advances in neural information processing systems (neurips) (Vol. 33, pp. 6840– 6851).
Ho, J., & Salimans, T. (2022). Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598.
Kilgour, K., Zuluaga, M., Roblek, D., & Sharifi, M. (2019). Frechet audio distance: A reference-free metric for evaluating music enhancement algorithms. In Proceedings of interspeech 2019 (pp. 2350–2354). doi: 10.21437/Interspeech.2019-2219
Kingma, D. P., & Welling, M. (2014). Auto-encoding variational Bayes. In Proceedings of the international conference on learning representations (iclr).
Koizumi, Y., Kawaguchi, Y., Imoto, K., Nakamura, T., Niitsuma, Y., Tanabe, R., . . . Harada, N. (2020). Description and discussion on DCASE 2020 challenge task 2: Unsupervised anomalous sound detection for machine condition monitoring. In Proceedings of the detection and classification of acoustic scenes and events workshop (dcase) (pp. 1–6).
Kong, J., Kim, J., & Bae, J. (2020). HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis. In Advances in neural information processing systems (neurips) (Vol. 33, pp. 17022– 17033).
Kong, Q., Cao, Y., Iqbal, T., Wang, Y., Wang, W., & Plumbley, M. D. (2020). PANNs: Large-scale pretrained audio neural networks for audio pattern recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 28, 2880–2894. doi: 10.1109/TASLP.2020.3030497
Kong, Z., Ping, W., Huang, J., Zhao, K., & Catanzaro, B. (2021). DiffWave: A versatile diffusion model for audio synthesis. In Proceedings of the international conference on learning representations (iclr).
Kumar, K., Kumar, R., de Boissiere, T., Gestin, L., Teoh, W. Z., Sotelo, J., . . . Courville, A. (2019). MelGAN: Generative adversarial networks for conditional waveform synthesis. In Advances in neural information processing systems (neurips) (Vol. 32).
Lei, Y., Li, N., Guo, L., Li, N., Yan, T., & Lin, J. (2018). Machinery health prognostics: A systematic review from data acquisition to RUL prediction. Mechanical Systems and Signal Processing, 104, 799–834.
Li, X., Ding, Q., & Sun, J.-Q. (2018). Remaining useful life estimation in prognostics using deep convolution neural networks. Reliability Engineering & System Safety, 172, 1–11.
Liu, H., Chen, Z., Yuan, Y., Mei, X., Liu, X., Mandic, D., . . . Plumbley, M. D. (2023). AudioLDM: Text-to-audio generation with latent diffusion models. In Proceedings of the international conference on machine learning (icml) (pp. 21450–21474).
Liu, H., Yuan, Y., Liu, X., Mei, X., Kong, Q., Tian, Q., . . . Plumbley, M. D. (2024). AudioLDM 2: Learning holistic audio generation with self-supervised pretraining. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32, 2871–2883.
Lu, C., Zhou, Y., Bao, F., Chen, J., Li, C., & Zhu, J. (2022). DPM-Solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps. In Advances in neural information processing systems (neurips) (Vol. 35, pp. 5775–5787).
McLachlan, G. J., & Peel, D. (2000). Finite mixture models. New York: Wiley.
Mehri, S., Kumar, K., Gulrajani, I., Kumar, R., Jain, S., Sotelo, J., . . . Bengio, Y. (2017). SampleRNN: An unconditional end-to-end neural audio generation model. In Proceedings of the international conference on learning representations (iclr).
Nichol, A. Q., & Dhariwal, P. (2021). Improved denoising diffusion probabilistic models. In International conference on machine learning (icml) (pp. 8162–8171).
Perez, E., Strub, F., de Vries, H., Dumoulin, V., & Courville, A. (2018). FiLM: Visual reasoning with a general conditioning layer. In Aaai conference on artificial intelligence.
Purohit, H., Tanabe, R., Ichige, K., Endo, T., Nikaido, Y., Suefusa, K., & Kawaguchi, Y. (2019). MIMII dataset: Sound dataset for malfunctioning industrial machine investigation and inspection. In Proceedings of the detection and classification of acoustic scenes and events workshop (dcase) (pp. 1–6).
Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286. doi: 10.1109/5.18626
Reynolds, D. A., & Rose, R. C. (1995). Robust text-independent speaker identification using Gaussian mixture speaker models. IEEE Transactions on Speech and Audio Processing, 3(1), 72–83. doi: 10.1109/89.365379
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022). High-resolution image synthesis with latent diffusion models. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition (cvpr) (pp. 10684–10695).
Salamon, J., & Bello, J. P. (2017). Deep convolutional neural networks and data augmentation for environmental sound classification. IEEE Signal Processing Letters, 24(3), 279–283. doi: 10.1109/LSP.2017.2657381
Salimans, T., & Ho, J. (2022). Progressive distillation for fast sampling of diffusion models. In International conference on learning representations (iclr).
Song, J., Meng, C., & Ermon, S. (2021). Denoising diffusion implicit models. In Proceedings of the international conference on learning representations (iclr).
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021). Score-based generative modeling through stochastic differential equations. In Proceedings of the international conference on learning representations (iclr).
Tailleur, M., Lee, J., Lagrange, M., Choi, K., Heller, L. M., Imoto, K., & Okamoto, Y. (2024). Correlation of Fr´echet audio distance with human perception of environmental audio is embedding dependent. In Proceedings of the 32nd european signal processing conference (EUSIPCO). Lyon, France. doi: 10.48550/arXiv.2403.17508
Takaki, S., Nakashika, T., Wang, X., & Yamagishi, J. (2019). STFT spectral loss for training a neural speech waveform model. In Ieee international conference on acoustics, speech and signal processing (icassp) (pp. 7065–7069).
van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., . . . Kavukcuoglu, K. (2016). WaveNet: A generative model for raw audio. arXiv preprint arXiv:1609.03499.
Yamamoto, R., Song, E., & Kim, J.-M. (2020). Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multiresolution spectrogram. In Proc. of icassp (pp. 6199– 6203).
Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., & Gao, R. X. (2019). Deep learning and its applications to machine health monitoring. Mechanical Systems and Signal Processing, 115, 213–237.
Ziyin, L., Hartwig, T., & Ueda, M. (2020). Neural networks fail to learn periodic functions and how to fix it. In Advances in neural information processing systems (neurips) (Vol. 33, pp. 1583–1594).
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
The Prognostic and Health Management Society advocates open-access to scientific data and uses a Creative Commons license for publishing and distributing any papers. A Creative Commons license does not relinquish the author’s copyright; rather it allows them to share some of their rights with any member of the public under certain conditions whilst enjoying full legal protection. By submitting an article to the International Conference of the Prognostics and Health Management Society, the authors agree to be bound by the associated terms and conditions including the following:
As the author, you retain the copyright to your Work. By submitting your Work, you are granting anybody the right to copy, distribute and transmit your Work and to adapt your Work with proper attribution under the terms of the Creative Commons Attribution 3.0 United States license. You assign rights to the Prognostics and Health Management Society to publish and disseminate your Work through electronic and print media if it is accepted for publication. A license note citing the Creative Commons Attribution 3.0 United States License as shown below needs to be placed in the footnote on the first page of the article.
First Author et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 United States License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.