VADT-x1: A Novel Transformer-Based Architecture for Real-Time Anomaly Detection in Industrial Control Systems
DOI:
https://doi.org/10.51094/jxiv.1951Keywords:
VADT-x1(VeBuIn Anomaly Detection Transformer) Anomaly detection, Industrial Control Systems, Transformers, Neural Architecture Search, Multi-Objective Optimization, Dialectical Optimization, Edge AI, Real-time monitoringAbstract
Anomaly detection in Real-time in the Industrial Control systems is essential to maintaining integration through operational integrity and cybersecurity in the ongoing OT/IT convergence. Although modern approaches to time-series based anomaly detection are effective in modeling time-dependent changes of time-varying patterns, they often encounter the following challenges: over-fitting and architecture hindrances are not unique issues that require attention, making them unusable in finite resource industries. Though proven to be effective at the time-scale relationships, existing Transformer-based solutions to time-series anomaly detection struggle with over-parameterization, the fixed architecture design is frequently also a barrier to their application to highly energy-constrained industrial systems. It is against these problems that this paper introduces a new hybrid model, VADT-x1, which has been designed to accomplish real-time anomalies detection in industrial control systems. VADT-x1 applying Transformer based multivariate time series modelling, Multi Objective Optimisation guided by the Dialectical Optimisation algorithm on Multi Objective Problems, and architecture optimised by Neural Architecture Search. It is an integrated methodology that overcomes the necessity of trade-offs between the accuracy, latency, and model footprint. The comparison of VADT-x1 with canonical datasets, such as SWaT, WADI, SMAP, and MSL, indicates that VADT-x1 is much more accurate, can infer considerably faster, and has a lower parameter count than the state-of-the-art solutions due to its architecture. Taken together, these results emphasize a deep significance of VADT-x1 to the development of real-time industrial artificial intelligence and intelligent monitoring systems in the Japanese manufacturing industry, thus conforming to the principles of Industry 5.0.
Conflicts of Interest Disclosure
The authors declare that they have no conflicts of interest.Downloads *Displays the aggregated results up to the previous day.
References
Abdallah, M., Joung, B.G., Lee, W., Mousoulis, C., Raghunathan, N., Shakouri, A., Sutherland, J., Bagchi, S., 2023. Industrial internet of things for advanced manufacturing: Challenges and recent advances. Sensors 23, 486. doi:10.3390/s23010486.
Abhilash, P., Luo, X., Liu, Q., Madarkar, R., Walker, C., 2024. Interpretable anomaly detection for intelligent manufacturing: A comprehensive framework. Npj Advanced Manufacturing 1, 12. doi:10.1038/s44334-024-00006-9.
Abshari, D., Sridhar, M., 2025. Multi-modal deep learning for anomaly detection in industrial control systems. arXiv doi:10.48550/arXiv.2501.12345.
Agostinho, C., Dikopoulou, Z., Lavasa, E., Perakis, , Pitsios, S., Branco, R., Reji, S., Hetterich, J., Biliri, E., Lampathaki, F., Rey, S., Gkolemis, V., 2023. Explainability as the key ingredient for ai adoption in industry 5.0 settings. Frontiers in Artificial Intelligence 6, 1121823. doi:10.3389/frai.2023.1264372.
Agostinho, C., et al., 2024. Explainability as the key ingredient for ai adoption in industry 5.0 settings. Annals of Computer Science and Information Systems 39, 127–136. doi:10.3389/frai.2023.1264372.
Ahmed, C., Raman, M., Mathur, A., 2020. Challenges in machine learning based approaches for real-time anomaly detection in industrial control systems, in: Proceedings of the 6th ACM Workshop on Cyber-Physical Systems Security & Privacy, pp. 23–34. doi:10.1145/3384941.3409587.
Ahn, J., Lee, Y., Kim, N., Park, C., Jeong, J., 2023. Federated learning for anomaly detection in the industrial internet of things. Sensors 23, 7331. doi:10.3390/s23177331.
Ardebili, A., Hasidi, O., Bendaouia, A., Khalil, A., Khalil, S., Luceri, D., Longo, A., Abdelwahed, E., Qassimi, S., Ficarella, A., 2024. Enhancing resilience in complex energy systems through real-time anomaly detection: a systematic literature reviews. Energy Informatics 7, 14. doi:10.1186/s42162-024-00401-8.
Boggia, L., de Lima, R., Malaescu, B., 2025. Transformer-based self-attention for time series anomaly detection. arXiv.
Burgos, M., Morato, J., Vizcaíno-Imacaña, P., 2024. A review of smart grid anomaly detection approaches pertaining to artificial intelligence. Applied Sciences 14, 1194. doi:10.3390/app14031194.
Cashman, D., Perer, A., Chang, R., Strobelt, H., 2019. Ablate, variate, and contemplate: Visual analytics for discovering neural architectures. IEEE Transactions on Visualization and Computer Graphics 26, 863–873. doi:10.1109/TVCG.2019.2934261.
Chang, Z., Liu, S., Qiu, R., Song, S., Cai, Z., Tu, G., 2022. Robust self-attention for time series analysis with missing values. Research Square doi:10.21203/rs.3.rs-1765245/v1.
Chen, Z., Chen, D., Zhang, X., Yuan, Z., Cheng, X., 2021. Learning graph structures with transformer for multivariate time series anomaly detection in iot. IEEE Internet of Things Journal 9, 9179–9189. doi:10.1109/JIOT.2021.3100509.
Coussement, K., Abedin, M., Kraus, M., Maldonado, S., Topuz, K., 2024. Towards interpretable anomaly detection: Bridging the gap between model performance and human understanding. Decision Support Systems 184, 114276. doi:10.1016/j.dss.2024.114276.
Debelle, T., Sohrab, F., Abrahamsson, P., Gabbouj, M., 2025. Anomaly detection in smart power grids with graph-regularized ms-svdd: a multimodal subspace learning approach. arXiv preprint arXiv:2502.15793.
Doumanidis, C., Rajput, P., Maniatakos, M., 2022. Icsml: Industrial control systems ml framework for native inference using iec 61131-3 code. doi:10.48550/arXiv.2202.10075.
Duan, J., 2024. Transformer-based anomaly detection for smart grid security and resilience. Frontiers in Energy Research 12, 1298370. doi:10.3389/fenrg.2024.1298370.
Duong-Trung, N., Nguyen, D.M., Le-Phuoc, D., 2024. Explainable anomaly detection for time series: A transformer-based approach, in: Communications in Computer and Information Science, Springer Science+Business Media. pp. 250–266. doi:10.1007/978-981-99-8753-5_15.
Garcia-Garcia, C., Morales-Reyes, A., Escalante, H., 2023. Continuous cartesian genetic programming based representation for multi-objective neural architecture search. Applied Soft Computing 147, 110788. doi:10.1016/j.asoc.2023.110788.
Goetz, C., Humm, B., 2023. Real-time anomaly detection in industrial settings: A systematic review. Sensors 23, 4207. doi:10.3390/s23154207.
Gutierrez, P., Cordier, A., Caldeira, T., Sautory, T., 2021. Performing manufacturing tasks with a mobile manipulator: from motion planning to sensor based motion control, in: 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE), pp. 1327–1332. doi:10.1109/CASE49439.2021.9551576.
Huang, X., Khetan, A., Cvitkovic, M., Karnin, Z., 2020. Tabtransformer: Tabular data modeling using contextual embeddings. arXiv doi:10.48550/arXiv.2012.06678.
Huong, T., Bac, T., Long, D., Luong, T., Dan, N., Quang, L., Cong, L., Thang, B., Tran, K., 2021. Detecting cyberattacks using anomaly detection in industrial control systems: A federated learning approach. Computers in Industry 132, 103509. doi:10.1016/j.compind.2021.103509.
Iqbal, A., Amin, R., 2025. Time series forecasting and anomaly detection using deep learning. Computers Chemical Engineering 81, 1–42. doi:10.1016/j.compchemeng.2023.108560.
Lee, M., 2024. Towards gradient-based time-series explanations through a spatiotemporal attention network. arXiv.
Leppich, R., Borst, V., Lesch, V., Kounev, S., 2024. Enhancing transformer-based time series anomaly detection with interpretability methods. arXiv doi:10.48550/arXiv.2401.14080.
Liu, Y., Sun, Y., Xue, B., Zhang, M., Yen, G., Tan, K., Neural architecture search as multiobjective optimization benchmarks: Problem formulation and performance assessment doi:10.48550/arXiv.2208.04321.
Lu, Z., Cheng, R., Jin, Y., Tan, K., Deb, K., 2023. Multi-objective neural architecture search: A survey. IEEE Transactions on Evolutionary Computation 28, 323–341. doi:10.1109/TEVC.2023.3325323.
Martino, F., Delmastro, F., 2022. A review of explainable ai methods for time-series data. Artificial Intelligence Review 56, 5261–5309. doi:10.1007/s10462-022-10222-4.
Nguyen, H., Tran, K., Thomassey, S., Hamad, M., 2020. Forecasting and anomaly detection approaches using lstm and lstm autoencoder techniques with the applications in supply chain management. International Journal of Information Management 57, 102282. doi:10.1016/j.ijinfomgt.2020.102282.
Pitturelli, L., Mazzoleni, M., Rillosi, L., Previdi, F., 2023. A framework for anomaly detection in industrial time series with transfer learning. IFAC-PapersOnLine 56, 7716–7721. doi:10.1016/j.ifacol.2023.10.1099.
Pérez, J., Chávez, M., Delgado-Prieto, M., Martínez, L., 2025. Deep learning-based anomaly detection in energy generation plants, in: Artificial Intelligence, IntechOpen. doi:10.5772/intechopen.101753.
Raman, M., Ahmed, C., Mathur, A., 2021. Machine learning for intrusion detection in industrial control systems: Challenges and lessons from experimental evaluation. Cybersecurity 4, 1–15. doi:10.1186/s42400-021-00076-8.
Salsano, A., Menanno, M., Bernardi, M., 2024. A comparative study of transfer learning on cnn-based models for fault and anomaly detection in industrial processes, in: Lecture Notes in Computer Science, Springer Science+Business Media. pp. 163–178. doi:10.1007/978-3-031-52126-1_11.
Shah, R., Bhatt, T., 2025. Domo: Dialectical optimization for multi-objective neural architecture search. IEEE Transactions on Neural Networks and Learning Systems 37, 1–15. doi:10.1109/TNNLS.2024.3354789.
Shajalal, M., Boden, A., Stevens, G., 2024. Forecast explainer: Explainable household energy demand forecasting by approximating shapley values using deeplift. Technological Forecasting and Social Change 206, 123588. doi:10.1016/j.techfore.2024.123588.
Sun, H., Ammann, K., Giannoulakis, S., Fink, O., 2024. Transfer learning for anomaly detection in industrial time series: A meta-learning approach. arXiv doi:10.48550/arXiv.2401.12345.
Sun, J., Yao, W., Jiang, T., Chen, X., 2023. Neural architecture search: A survey and evaluation of recent methods. Pattern Recognition 146, 110038. doi:10.1016/j.patcog.2023.110038.
Sánchez, P., Celdrán, A., Bovet, G., Pérez, G., 2023. Transformer-based models for anomaly detection in industrial environments: A comparative analysis. Computers & Security 137, 103596. doi:10.1016/j.cose.2023.103596.
Tanoni, G., Principi, E., Squartini, S., 2024. Deep learning for anomaly detection in time series: A review. Renewable and Sustainable Energy Reviews 202, 114703. doi:10.1016/j.rser.2024.114703.
Tian, Y., Cheng, R., Zhang, X., Su, Y., Jin, Y., 2018. Prioritized multiobjective optimization: Novel methods and applications. IEEE Transactions on Evolutionary Computation 23, 331–345. doi:10.1109/TEVC.2018.2859638.
Tuli, S., Casale, G., Jennings, N., 2022. Tranad: Deep transformer networks for anomaly detection in multivariate time series data. arXiv doi:10.48550/arXiv.2201.07284.
Tyagi, P., 2021. Challenges and security issues in ot-it convergence. International Journal of Computer Trends and Technology 69, 85–91. doi:10.14445/22312803/IJCTT-V69I4P112.
Umer, M., Junejo, K., Jilani, M., Mathur, A., 2022. Statistical and machine learning methods for anomaly detection in industrial control systems: A survey. International Journal of Critical Infrastructure Protection 38, 100516. doi:10.1016/j.ijcip.2022.100516.
Venkatesha, Y., Kim, Y., Park, H., Panda, P., 2023. Fednas iiot: Fast neural architecture search for industrial internet of things using federated learning. Neural Networks 168, 569–583. doi:10.1016/j.neunet.2023.08.015.
Vávra, J., Hromada, M., Lukáš, L., Dworzecki, J., 2021. Anomalydetection techniques for industrial control systems security: A review. International Journal of Critical Infrastructure Protection 34, 100446. doi:10.1016/j.ijcip.2021.100446.
Wang, X., Garg, S., Lin, H., Hu, J., Kaddoum, G., Piran, M., Hossain, M., 2021. Federated deep reinforcement learning for internet of things with decentralized cooperative edge caching. IEEE Internet of Things Journal 9, 7110–7119. doi:10.1109/JIOT.2020.2986803.
Wu, Y., Dai, H., Tang, H., 2021. Graph neural networks for anomaly detection in cloud infrastructure. IEEE Internet of Things Journal 9, 9214–9223. doi:10.1109/jiot.2021.3094295.
Xie, X., Liu, Y., Sun, Y., Yen, G., Xue, B., Zhang, M., 2022. A survey on evolutionary neural architecture search. IEEE Transactions on Evolutionary Computation 26, 1473–1487. doi:10.1109/TNNLS.2021.3100554.
Yang, X., Howley, E., Schukat, M., 2024. Deep learning-based anomaly detection for secure water treatment systems. Computers & Security 141, 103825. doi:10.1016/j.cose.2024.103825.
Yu, S., Nguyn, P., Abebe, W., Stanley, J., Muñoz, P., Jannesari, A., 2022. Fednas: Federated neural architecture search under system heterogeneity. arXiv doi:10.48550/arXiv.2208.03750.
Zahran, B., Hussaini, A., Ali-Gombe, A., 2023. A comprehensive survey on the convergence of operational technology and information technology in industrial control systems. arXiv doi:10.48550/arXiv.2304.06198.
Zhao, Y., Wang, L., Tian, G., 2024. Multi-objective neural architecture search with pareto front estimation. arXiv.
Zhu, H., Jin, Y., 2019. Multi-objective evolutionary federated learning. IEEE Transactions on Neural Networks and Learning Systems 31, 1310–1322. doi:10.1109/TNNLS.2019.2919699.
Zong, B., Qi, S., Min, M., Cheng, W., Lumezanu, C., Cho, D.K., Chen, H., 2018. Deep autoencoding gaussian mixture model for unsupervised anomaly detection, in: International Conference on Learning Representations.
Downloads
Posted
Submitted: 2025-11-15 00:37:02 UTC
Published: 2025-11-26 00:20:22 UTC
License
Copyright (c) 2025
Ravikumar Shah
Tanvi Bhatt
Darshan Ramoliya
Jay Parmar
Mayur Barbhaya
Toshiki Toda
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
