Rust detection from 3D point clouds using sensor fusion
DOI:
https://doi.org/10.51094/jxiv.1160Keywords:
LiDAR, rust, segment anything model (SAM), sensor fusion, tunnelAbstract
In this study, we developed a method leveraging sensor fusion technology that combines camera images and LiDAR point clouds obtained using Matterport for efficient inspection of rust in conduit tunnels. Conventional methods relying solely on point cloud data showed difficulty in detecting rust due to its minimal surface irregularities and shape changes. In contrast, high-precision rust detection was achieved by utilizing differences in color and texture from camera images, enabling the estimation of rust locations within tunnels. By integrating information from images and LiDAR, it became possible to calculate values such as rust location and area, which are difficult to estimate from images alone. The sensor fusion approach accurately estimated the distance per image pixel, achieving a mean absolute error of 1.3×10⁻³m and a mean absolute percentage error (MAPE) of 6.7%.
Conflicts of Interest Disclosure
No potential conflicts of interest were disclosed.Downloads *Displays the aggregated results up to the previous day.
References
久保栞，全邦釘，伊藤克雄：YOLOv5 を用いた導水路トンネルにおけるチョーキング箇所の検出. AI・データサイエンス論文集，2巻J2号， pp. 87-96， 2021.
Kubo, S., Nakayama, N., Matsuda, S., & Chun, P. J.: Corrosion Damage Detection in Headrace Tunnel Using YOLOv7 with Continuous Wall Images. Appl. Sci., Vol. 13, Issue 16, pp. 9388, 2023.
劉佳明，党紀，全邦釘：DeepLabv3+を用いた橋梁腐食損傷とその精度の向上，AI・データサイエンス論文集，3巻J2号，pp. 802-810，2022.
斎藤 嘉人，板倉 健太，山本 一哉，二宮 和則，近藤 直： 可視・近赤外画像のセマンティックセグメンテーションによるバレイショ塊茎表面の病害検出，Ai・データサイエンス論文集，3巻J2号，pp. 175–181，2022.
T. Takemoto, Z. Huang, K.A. Omwange, Y. Saito, K. Konagaya, T. Suzuki, Y. Ogawa, N. Kondo: Label-free technology for traceable identification of single green pepper through features in UV fluorescent images, Comput. Electron. Agric, Vol. 211, pp. 107960, 2023.
辻井純平，合田哲朗，中野雅章：土木構造物の点群解析に向けた局所形状の畳み込みを伴う深層学習手法の適用，AI・データサイエンス論文集，4巻3号，pp. 442-450，2023.
関和彦，山口愛加，窪田諭：3次元点群データを用いた道路橋の損傷抽出とヒートマップ表示，土木学会論文集，79巻10 号，pp. 22-00071，2023.
辻井 純平, 合田 哲朗, 中野 雅章：構造物の点群データに対する深層学習を用いた耐荷性能推定の基礎検討，AI・データサイエンス論文集, 5 巻3 号， pp. 328-336，2024.
吉谷 薫，小林 巧，大住 道生：震後点検における点群計測による鋼アーチ橋のゆがみ調査，AI・データサイエンス論文集，5 巻 3 号， pp. 678-687，2024.
板倉健太，林拓哉，上脇優人，全邦釘：橋梁の3次元点群を利用した構造情報の計算，Jxiv，doi: https://doi.org/10.51094/jxiv.901
板倉 健太，林 拓哉，上脇 優人，全 邦釘：LiDARとカメラのセンサーフュージョンによる点群からのノイズ除去，AI・データサイエンス論文集, 5 巻3 号，pp. 757-768，2024.
板倉健太，林拓哉，上脇優人，全邦釘：セマンティックセグメンテーションやセンサーフュージョンを利用した橋梁の構造情報の推定のための点群処理手法の開発，土木学会AI・データサイエンス論文集，5 巻 3 号 ，pp. 10-21，2024.
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W. Y., Dollar, P., Girshick, R.: Segment anything, ICCV, pp. 4015-4026, 2023.
Liu, H., Wu, C., and Wang, H.: Real time object detection using LiDAR and camera fusion for autonomous driving, Sci. Rep., Vol. 13, pp. 8056, 2023.
Hoppe, H., T. DeRose, T. Duchamp, J. Mcdonald, and W. Stuetzle: Surface Reconstruction from Unorganized Points, Computer Graphics, SIGGRAPH, Vol. 26, Issue. 2, pp. 71–78, 1992.
Torr, P. H., and Zisserman, A., MLESAC: A new robust estimator with application to estimating image geometry, Comput. Vis. Image Und., Vol. 78, Issue 1, pp. 138-156, 2000.
Fischler, M. A., and Bolles, R. C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography, Com. ACM, Vol. 24, Issue 6, pp. 381-395, 1981.
Tan, L., Chen, X., Yuan, D., & Tang, T. (2024). DSNet: A Computer Vision‐Based Detection and Corrosion Segmentation Network for Corroded Bolt Detection in Tunnel. Struct. Control Health Monit., (1), pp. 1898088, 2024.
Z. Guo, X. Cheng, Q. Xie, H. Zhou: Spatial Adaptive Improvement Detection Network for Corroded Bolt Detection in Tunnels, Buildings, Vol. 14, pp. 2560, 2024.
Itakura, K., Narita, Y., Noaki, S., & Hosoi, F.: Automatic pear and apple detection by videos using deep learning and a Kalman filter. Osa Continuum, vol. 4, Issue 5, pp. 1688-1695, 2021.
板倉健太，林拓哉，野秋収平，上脇優人，& 細井文樹：深層学習を用いた根菜類の個数カウンティングによる収量推定法の開発，AI・データサイエンス論文集，3巻J2号 ，pp.6-16，2022.
稲富翔伍，全邦釘：点群の画像化とディープラーニングを用いた橋梁点群のセグメンテーション，AI・データサイエンス論文集，Vol. 2，No. 2，pp. 418-427，2021.
Itakura, K., and Hosoi, F.: Three-dimensional tree monitoring in urban cities using automatic tree detection method with mobile LiDAR data, AI Data Sci., Vol. 2, Issue 2, pp. 1-10, 2021.
Itakura, K., and Hosoi, F.: Automated tree detection from 3D lidar images using image processing and machine learning, Appl. Opt.,Vol. 58, Issue 14, pp. 3807-3811, 2019.
板倉健太，細井文樹：画像処理や3次元深層学習を用いた航空機ライダー点群データからの樹木の検出，AI・データサイエンス論文集，1巻J1号，pp. 320-328，2020.
Hosoi, F., Umeyama, S., and Kuo, K.: Estimating 3D chlorophyll content distribution of trees using an image fusion method between 2D camera and 3D portable scanning lidar, Remote Sens., Vol. 11, Issue 18, pp. 2134, 2019.
Narváez, F. J. Y., del Pedregal, J. S., Prieto, P. A., Torres-Torriti, M., and Cheein, F. A. A.: LiDAR and thermal images fusion for ground-based 3D characterisation of fruit trees, Biosys. Eng., Vol. 151, pp. 479-494, 2016.
Downloads
Posted
Submitted: 2025-03-19 01:39:06 UTC
Published: 2025-03-25 08:18:21 UTC
License
Copyright (c) 2025
Kenta ITAKURA
Takuya HAYASHI
Yoshito SAITO
Pang-jo CHUN
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
