Automatic generation of bridge defect descriptions using image captioning techniques

Chai, Chengzhang ORCID: https://orcid.org/0000-0001-6911-8048, Gao, Yan ORCID: https://orcid.org/0000-0001-5890-9717, Li, Haijiang ORCID: https://orcid.org/0000-0001-6326-8133 and Xiong, Guanyu 2024. Automatic generation of bridge defect descriptions using image captioning techniques. Presented at: The 10th International Conference on Construction Engineering and Project Management, Sapporo, Hokkaido, Japan, 29 July - 01 August 2024. ICCEPM 2024 Conference Proceedings. pp. 319-326.

Preview

PDF - Accepted Post-Print Version Download (729kB) | Preview

Abstract

Bridge inspection is crucial for infrastructure maintenance. Current inspections based on computer vision primarily focus on identifying simple defects such as cracks or corrosion. These detection results can serve merely as preliminary references for bridge inspection reports. To generate detailed reports, on-site engineers must still present the structural conditions through lengthy textual descriptions. This process is time-consuming, costly, and prone to human error. To bridge this gap, we propose a deep learning-based framework to generate detailed and accurate textual descriptions, laying the foundation for automating bridge inspection reports. This framework is built around an encoderdecoder architecture, utilizing Convolutional Neural Networks (CNN) for encoding image features and Gated Recurrent Units (GRU) as the decoder, combined with a dynamically adaptive attention mechanism. The experimental results demonstrate this approach's effectiveness, proving that the introduction of the attention mechanism contributes to improved generation results. Moreover, it is worth noting that, through comparative experiments on image restoration, we found that the model requires further improvement in terms of explainability. In summary, this study demonstrates the potential and practical application of image captioning techniques for bridge defect detection, and future research can further explore the integration of domain knowledge with artificial intelligence (AI).

Item Type:	Conference or Workshop Item (Paper)
Status:	Published
Schools:	Engineering
Subjects:	T Technology > TA Engineering (General). Civil engineering (General)
Date of First Compliant Deposit:	7 September 2024
Date of Acceptance:	7 July 2024
Last Modified:	02 Nov 2024 02:30
URI:	https://orca.cardiff.ac.uk/id/eprint/171911

Actions (repository staff only)

Edit Item