From Faulty Greenhouse Sensors to Usable Training Data An End-to-End Sensor Health Management Pipeline for Fault Detection, Data Repair, and Downstream Validation
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Abstract
Predictive models for smart greenhouses commonly assume that the low-cost sensor data they are trained on are correct.
This paper treats the health of the sensing system as the object of study, using an uncurated 34-day greenhouse irrigation deployment (947,682 raw records, six streams, no fault-management layer). Every stream carries a naturally occurring fault signature: dropouts on 26.4% of the span, an eight-day out-of-range soil-moisture episode, a zero-variance channel, a CO2 channel with a suspected calibration bias, stuck-at episodes, spikes, and, after the longest outage, a variance collapse that marks a change of operating regime. The pipeline characterizes these faults, benchmarks rule-based screening, Isolation Forest and an LSTM autoencoder on 162 injection scenarios and seven repair methods, and measures how training-data treatment changes a soil-moisture forecaster against persistence. Rules and the autoencoder are complementary, bias and drift stay hard for all detectors, and linear interpolation is the best repair or not significantly different from the best in nearly every condition up to 12 h. The original LSTM forecasters of the soil-moisture level do not beat persistence at 5 min to 6 h, mainly because they cannot extrapolate to the post-outage level; within the training range, repaired data cut their pooled 5-min error 3.9–4.9-fold against raw data when samples are grouped by the true target and 3.1–3.7-fold when grouped by the last observed value; at 5 min, with the long-gap fallback, rules-guided repair reaches the persistence level. In an exploratory analysis added after these results, level-free change models selected on training data only beat persistence by 14% at 5 min when trained on repaired data, against 5% for raw data; at 1 h validation selected persistence for raw data and for two of three repair variants, and at 6 h the selected repaired-data models are worse than persistence. A random-mask control does not show a benefit of targeted repair on the full test set.
##plugins.themes.bootstrap3.article.details##
sensor health management, sensor fault detection, data repair, measurement reliability, anomaly detection, IoT sensing, smart greenhouse
AlSalehy, A. S., & Bailey, M. (2025). Improving time series data quality: Identifying outliers and handling missing values in a multi-location gas and weather dataset. Smart Cities, 8(3), 82. doi: 10.3390/smartcities8030082
Benameur, R., Dahane, A., Kechar, B., & Benyamina, A. E. H. (2024). An innovative smart and sustainable low-cost irrigation system for anomaly detection using deep learning. Sensors, 24(4), 1162. doi: 10.3390/s24041162
Blázquez-García, A., Conde, A., Mori, U., & Lozano, J. A. (2021). A review on outlier/anomaly detection in time series data. ACM Computing Surveys, 54(3), 1–33. doi: 10.1145/3444690
Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000). LOF: Identifying density-based local outliers. In Proceedings of the 2000 acm sigmod international conference on management of data (pp. 93–104). doi: 10.1145/342009.335388
Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys, 41(3), 1–58. doi: 10.1145/1541880.1541882
Contreras-Castillo, J., Guerrero-Ibañez, J. A., Santana-Mancilla, P. C., & Anido-Rifón, L. (2023). SAgric-IoT: An IoT-Based platform and deep learning for greenhouse monitoring. Applied Sciences, 13(3), 1961. doi: 10.3390/app13031961
Diebold, F. X., & Mariano, R. S. (1995). Comparing predictive accuracy. Journal of Business & Economic Statistics, 13(3), 253–263. doi: 10.1080/07350015.1995.10524599
Du, W., Côté, D., & Liu, Y. (2023). SAITS: Self-attention-based imputation for time series. Expert Systems with Applications, 219, 119619. doi: 10.1016/j.eswa.2023.119619
El-Shafeiy, E., Alsabaan, M., Ibrahem, M. I., & Elwahsh, H. (2023). Real-time anomaly detection for water quality sensor monitoring based on multivariate deep learning technique. Sensors, 23(20), 8613. doi: 10.3390/s23208613
Ghazouani, M., Azzouazi, M., & Lamhour, M. A. (2023). A drip irrigation prediction system in a greenhouse based on long short-term memory and connected objects. Mathematical Modeling and Computing, 10(2), 524–533. doi: 10.23939/mmc2023.02.524
Harvey, D., Leybourne, S., & Newbold, P. (1997). Testing the equality of prediction mean squared errors. International Journal of Forecasting, 13(2), 281–291. doi: 10.1016/S0169-2070(96)00719-4
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. doi: 10.1162/neco.1997.9.8.1735
Hua, V., Nguyen, T., Dao, M.-S., Nguyen, H. D., & Nguyen, B. T. (2024). The impact of data imputation on air quality prediction problem. PLOS ONE, 19(9), e0306303. doi: 10.1371/journal.pone.0306303
Hundman, K., Constantinou, V., Laporte, C., Colwell, I., & Soderstrom, T. (2018). Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding. In Proceedings of the 24th acm sigkdd international conference on knowledge discovery & data mining (pp. 387–395). doi: 10.1145/3219819.3219845
Hyndman, R. J., & Koehler, A. B. (2006). Another look at measures of forecast accuracy. International Journal of Forecasting, 22(4), 679–688. doi: 10.1016/j.ijforecast.2006.03.001
Kamilaris, A., & Prenafeta-Boldú, F. X. (2018). Deep learning in agriculture: A survey. Computers and Electronics in Agriculture, 147, 70–90. doi: 10.1016/j.compag.2018.02.016
Kandamali, D. F., Porter, E., Porter, W. M., McLemore, A., Kiobia, D. O., Tavandashti, A. P., & Rains, G. C. (2025). Hybrid LSTM method for multistep soil moisture prediction using historical soil moisture and weather data. AgriEngineering, 7(8), 260. doi: 10.3390/agriengineering7080260
Khayati, M., Lerner, A., Tymchenko, Z., & Cudré-Mauroux, P. (2020). Mind the gap: an experimental evaluation of imputation of missing values techniques in time series. Proceedings of the VLDB Endowment, 13(5), 768–782. doi: 10.14778/3377369.3377383
Kim, M., Yu, J., Kim, J., Oh, T.-H., & Choi, J. K. (2024). An iterative method for unsupervised robust anomaly detection under data contamination. IEEE Transactions on Neural Networks and Learning Systems, 35(10), 13327-13339. doi: 10.1109/TNNLS.2023.3267028
Koné, B. A. T., Grati, R., Bouaziz, B., & Boukadi, K. (2023). A new long short-term memory based approach for soil moisture prediction. Journal of Ambient Intelligence and Smart Environments, 15(3), 255–268. doi: 10.3233/AIS-230035
Künsch, H. R. (1989). The jackknife and the bootstrap for general stationary observations. The Annals of Statistics, 17(3), 1217–1241. doi: 10.1214/aos/1176347265
Lamhour, M. A., Msalek, M., Ardchir, S., Ghazouani, M., & Azzouazi, M. (2026). Multi-seed XAI validation for reliable LSTM irrigation control. IEEE Access, 14, 120366-120387. doi: 10.1109/ACCESS.2026.3720462
Li, Z., Abdullah, N., & Ishak, M. K. (2025). Smart greenhouse climate control with real-time fault detection and energy-aware automation. Smart Agricultural Technology, 13, 101707. doi: 10.1016/j.atech.2025.101707
Liakos, K. G., Busato, P., Moshou, D., Pearson, S., & Bochtis, D. (2018). Machine learning in agriculture: A review. Sensors, 18(8), 2674. doi: 10.3390/s18082674
Lindemann, B., Maschler, B., Sahlab, N., & Weyrich, M. (2021). A survey on anomaly detection for technical systems using LSTM networks. Computers in Industry, 131, 103498. doi: 10.1016/j.compind.2021.103498
Little, R. J. A., & Rubin, D. B. (2019). Statistical analysis with missing data (3rd ed.). Wiley. doi: 10.1002/9781119482260
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008). Isolation Forest. In Proceedings of the 2008 eighth ieee international conference on data mining (pp. 413–422). doi: 10.1109/ICDM.2008.17
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2012). Isolation-Based anomaly detection. ACM Transactions on Knowledge Discovery from Data, 6(1), 3. doi: 10.1145/2133360.2133363
Moritz, S., & Bartz-Beielstein, T. (2017). imputeTS: Time series missing value imputation in R. The R Journal, 9(1), 207–218. doi: 10.32614/RJ-2017-009
Ni, K., Ramanathan, N., Chehade, M. N. H., Balzano, L., Nair, S., Zahedi, S., ... Srivastava, M. (2009). Sensor network data fault types. ACM Transactions on Sensor Networks, 5(3), 25. doi: 10.1145/1525856.1525863
Ojha, T., Misra, S., & Raghuwanshi, N. S. (2015). Wireless sensor networks for agriculture: The state-of-the-art in practice and future challenges. Computers and Electronics in Agriculture, 118, 66–84. doi: 10.1016/j.compag.2015.08.011
Pearson, R. K., Neuvo, Y., Astola, J., & Gabbouj, M. (2016). Generalized Hampel filters. EURASIP Journal on Advances in Signal Processing, 2016, 87. doi: 10.1186/s13634-016-0383-6
Poorter, H., Fiorani, F., Stitt, M., Schurr, U., Finck, A., Gibon, Y., ... Pons, T. L. (2012). The art of growing plants for experimental purposes: A practical guide for the plant biologist. Functional Plant Biology, 39(11), 821–838. doi: 10.1071/FP12028
Sacleanu, aD.-I., Matache, M.-G., Ro¸su, ¸S.-G., Florea, B.-C., Manciu, I.-P., & Perisoara, L.-A. (2024). IoT-enhanced decision support system for real-time greenhouse microclimate monitoring and control. Technologies, 12(11), 230. doi: 10.3390/technologies12110230
Schmidl, S., Wenig, P., & Papenbrock, T. (2022). Anomaly detection in time series: A comprehensive evaluation. Proceedings of the VLDB Endowment, 15(9), 1779-1797. doi: 10.14778/3538598.3538602
Sharma, A. B., Golubchik, L., & Govindan, R. (2010). Sensor faults: Detection methods and prevalence in real-world datasets. ACM Transactions on Sensor Networks, 6(3), 23. doi: 10.1145/1754414.1754419
Shekarian, S. M., Aminian, M., Mohammad Fallah, A., & Akbary Moghaddam, V. (2024). AI-powered sensor fault detection for cost-effective smart greenhouses. Computers and Electronics in Agriculture, 224, 109198. doi: 10.1016/j.compag.2024.109198
Simo, A., Dzitac, S., Badea, G. E., & Meianu, D. (2022). Smart agriculture: IoT-based greenhouse monitoring system. International Journal of Computers Communications & Control, 17(6), 5039. doi: 10.15837/ijccc.2022.6.5039
Teshome, F. T., Bayabil, H. K., Schaffer, B., Ampatzidis, Y., & Hoogenboom, G. (2024). Improving soil moisture prediction with deep learning and machine learning models. Computers and Electronics in Agriculture, 226, 109414. doi: 10.1016/j.compag.2024.109414
Thai, T.-T. N., Do, P., Iung, B., & Gardoni, P. (2025). Impact of sensor degradation on measurement uncertainty in prognostics and maintenance decision-making: State of the art and challenges. International Journal of Prognostics and Health Management, 16(4). doi: 10.36001/ijphm.2025.v16i4.4306
Trapani, N., & Longo, L. (2023). Fault detection and diagnosis methods for sensors systems: a scientific literature review. IFAC-PapersOnLine, 56(2), 1253–1263. doi: 10.1016/j.ifacol.2023.10.1749
Troyanskaya, O., Cantor, M., Sherlock, G., Brown, P., Hastie, T., Tibshirani, R., ... Altman, R. B. (2001). Missing value estimation methods for DNA microarrays. Bioinformatics, 17(6), 520–525. doi: 10.1093/bioinformatics/17.6.520
Tzounis, A., Katsoulas, N., Bartzanas, T., & Kittas, C. (2017). Internet of Things in agriculture, recent advances and future challenges. Biosystems Engineering, 164, 31–48. doi: 10.1016/j.biosystemseng.2017.09.007
Ulmer, M., Zgraggen, J., & Goren Huber, L. (2024). A generic machine learning framework for fully-unsupervised anomaly detection with contaminated data. International Journal of Prognostics and Health Management, 15(1), 1–12. doi: 10.36001/ijphm.2024.v15i1.3589
van Buuren, S., & Groothuis-Oudshoorn, K. (2011). mice: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3), 1–67. doi: 10.18637/jss.v045.i03
Wang, Y., Shi, L., Hu, Y., Hu, X., Song, W., & Wang, L. (2024). A comprehensive study of deep learning for soil moisture prediction. Hydrology and Earth System Sciences, 28(4), 917–943. doi: 10.5194/hess-28-917-2024
Wolfert, S., Ge, L., Verdouw, C., & Bogaardt, M.-J. (2017). Big data in smart farming– a review. Agricultural Systems, 153, 69–80. doi: 10.1016/j.agsy.2017.01.023
Zhabko, O., Laktionov, I., Diachenko, G., Vinyukov, O., & Moroz, D. (2026). Agroclimatic forecasting under degraded sensor data: A robustness benchmark of machine-learning models. Applied Sciences, 16(10), 5075. doi: 10.3390/app16105075
Zou, X., Liu, W., Huo, Z., Wang, S., Chen, Z., Xin, C., ... Shu, L. (2023). Current status and prospects of research on sensor fault diagnosis of agricultural Internet of Things. Sensors, 23(5), 2528. doi: 10.3390/s23052528