Assessment of Health Monitoring Trustworthiness of Avionics Systems



Published Jul 25, 2021
Vladimir Ulansky Dr. Igor Machalin
Dr. Iryna Terentyeva


The article provides a methodology for assessing the trustworthiness of health monitoring the dismounted avionics systems with automated test equipment (ATE). The indicators include the probabilities of false-positive, false-negative, true-positive, and true-negative. For the first time, we introduced into consideration the instability of the source of stimulus signal (SSS), the random and systematic component of the measuring channel error, and the reliability characteristics of the systems themselves. We consider a specific case of an exponential distribution of permanent failures and intermittent faults and derive formulas for calculating the trustworthiness indicators. Numerical calculations illustrate how the probabilities of correct and incorrect decisions depend on accuracy parameters. We show that the probabilities of false-positive and false-negative increase much faster than the probabilities of true-positive and true-negative decrease when the standard deviation of stimulus signal increases. For a Very High-Frequency Omni-Directional Range (VOR) receiver, we demonstrate that even with a zero random error generated by the source of the stimulus signal, the probabilities of false-positive and false-negative are different from zero.

Abstract 670 | PDF Downloads 534



Health monitoring; False-positive; False-negative; VOR receiver

Aeronautical Information Manual (2017). Official Guide to Basic Flight Information and ATC Procedures, U.S. Department of Transportation. Federal Aviation Administration, USA.
Aeroflex (2005). Avionics IRIS®2000 automatic test equipment system (2005). Plainview, New York, USA.
Bao, H., Ying, C. L., Shi, Q. W., & Teng, J. F. (2006). Research on portable maintenance aid equipment using ARM-based VXI modules. Proceedings of Sixth International Symposium on Instrumentation and Control Technology. October 13-15, Beijing, China. doi:
Breitgand, D., Goldstein, M., Henis, E., & Shehory, O. (2011). Efficient сontrol of false negative and false positive errors with separate adaptive thresholds. IEEE Trans. on Network and Service Management, 8(2), 128-140. doi: 10.1109/TNSM.2011.020111.00055
Chen, X., Zhang, Z., & Zhang, Z. (2019). Real-time equipment condition assessment for a class-imbalanced dataset based on heterogeneous ensemble learning. Eksploatacja i Niezawodnosc – Maintenance and Reliability, 21(1), 68–80. doi:
Droste, D. B., & Guilbeaux, G. (2009). Advanced architecture for achieving true vertical testability in next generation ATE. Proceedings of IEEE Autotestcon. 14-17 September, Anaheim, California. doi: 10.1109/AUTEST.2009.5314088
Ebrahimi, N. (2008). Simultaneous control of false positives and false negatives in multiple hypotheses testing. Journal of Multivariate Analysis, 99(3), 437-450. doi:
eCASS electronic consolidated automated support system (2020). Lockheed Martin. Bethesda, Maryland, USA. electronic-consolidated-automated-support-system-ecass.html.
Evlanov, L. G. (1979). Control of dynamic systems. Moscow: Nauka.
Foss, A., & Zaiane, O. R. (2008). Estimating true and false positive rates in higher dimensional problems and its data mining applications. Proceedings of IEEE International Conference on Data Mining Workshops. 15-19 December, Pisa, Italy. doi: 10.1109/ICDMW.2008.38
Hand, D., & Christen, P. (2018). A note on using the F-measure for evaluating record linkage algorithms. Statistics and Computing, 28(3), 539-547. doi:
Ho, C. Y., Lai, Y. C., Chen, I. W, Wang, F. Y. & Tai, W. H. (2012). Statistical analysis of false positives and false negatives from real traffic with intrusion detection/prevention systems. IEEE Communications Magazine, 50(3), 146-154. doi: 10.1109/MCOM.2012.6163595
Hossin, M., & Sulaiman, M. N. (2015). A review on evaluation metrics for data classification evaluations. Int. J. of Data Mining & Knowledge Management Process, 5(2), 1-11. doi: 10.5121/ijdkp.2015.5201
International Electrotechnical Commission (IEC) (2004). Higher performance protocol for the standard digital interface for programmable instrumentation - Part 1: General. In IEC, IEC-60488-1: 2004. Part 1 (p. 13). Genève, Switzerland: International Electrotechnical Commission.
International Electrotechnical Commission (IEC) (2004). Standard Digital Interface for Programmable Instrumentation - Part 2: Codes, formats, protocols and common commands. In IEC, IEC-60488-2: 2004. Part 2. Genève, Switzerland: International Electrotechnical Commission.
Ilarslan, M., & Ungar, L. Y. (2016). Mitigating the impact of false alarms and no fault found events in military systems. IEEE Instrumentation & Measurement Magazine, 8, 16-22. doi: 10.1109/MIM.2016.7524202
Khan, S, Phillips, P., Hockley, C. & Jennions, I. (2014). No fault found events in maintenance engineering part 2: Root causes, technical developments and future research. Reliability Eng. & System Safety, 123, 196-208. doi:
Kudritsky, V. D., Sinitsa, M. A., & Chinaev, P. I. (1977). Automation of health-monitoring of electronic equipment. Moscow: Soviet Radio.
Lisman, J. H. C., & van Zuylen, M. C. A. (1972). Note on the generation of most probable frequency distributions. Statistica Neerlandica, 26(1), 19-23. doi:
Ma, L., Zhang, X., Wang , K. & Xu, T. (2013). A general method for module automatic testing in avionics systems. Research Journal of Applied Sciences, Engineering and Technology, 5(20), 4796-4804. doi: 10.19026/rjaset.5.4322
Malesich, M. (2007). Advances in DoD’S ATS framework. Proceedings of IEEE Autotestcon. 17-20 September, Baltimore, MD, USA. doi: 10.1109/AUTEST.2007.4374202
Mane, S., Srivastava, J., Hwang, S. Y., & Vayghan, J. (2004). Estimation of false negatives in classification. Proceedings of Fourth IEEE International Conference on Data Mining. 1-4 November, Brighton, UK. doi: 10.1109/ICDM.2004.10048
Manning, C. D., Raghavan, P., & Schutze, H. (2008). Introduction to information retrieval. Cambridge: Cambridge University Press.
Rausand, M., & Hoyland, A. (2003). System reliability theory: models, statistical methods, and applications. New York: Wiley.
Raza, A. (2018). Maintenance model of digital avionics. Aerospace, 5(2), 1-16. doi:
Raza, A., Ulanskyi, V., Augustynek, K., & Warwas, K. (2017). Generalized cost functions of avionics breakdown maintenance strategy. Proceedings of IEEE Aerospace Conference. 4-11 March, Big Sky, MT. doi: 10.1109/AERO.2017.7943754
Raza, A., & Ulansky, V. (2018). Modelling of false alarms and intermittent faults and their impact on the maintenance cost of digital avionics. Procedia Manufacturing, 16, 107-114. doi:
Raza, A., & Ulansky, V. (2018). Analyses of warranty losses to avionics suppliers. Proceedings of IEEE Aerospace Conference. 3-10 March, Big Sky, MT. doi: 10.1109/AERO.2018.8396752
Ross, W. A. (2003). The impact of next generation test technology on aviation maintenance. Proceedings of IEEE Autotestcon. 23-25 September, Anaheim, CA, USA. doi: 10.1109/AUTEST.2003.1243547
Scott, C. (2007). Performance measures for Neyman–Pearson classification. IEEE Trans. on Information Theory, 53(8), 2852-2863. doi: 10.1109/TIT.2007.901152
Sokolova, M., Japkowicz, N., & Szpakowicz S. (2006). Beyond accuracy, F-score and ROC: a family of discriminant measures for performance evaluation. Proceedings of 19th Australian Joint Conference on Artificial Intelligence: Advances in Artificial Intelligence. 4-8 December, Hobart, TAS, Australia. doi:
Stora, M. J., & Droste D. (2003). ATE open system platform. IEEE-Pl552 structured architecture for test systems (SATS). Proceedings of IEEE Autotestcon. 22-25 September, Anaheim, CA, USA. doi: 10.1109/AUTEST.2003.1243559
Spherea (2017). The prepared maintenance solution for airlines and MROS. Hamburg, Germany. https://
Ulansky, V. (1992). The trustworthiness of multiple-monitoring the operability of non-repairable electronic systems. In Ignatov V. (Ed.), Saving Technologies and Avionics Maintenance of Civil Aviation Aircraft (14-25). Kyiv: KIIGA Press.
Technical Papers