Multimodal deep learning-based automatic generation of repair proposals for steel bridge shallow damage

Song, Honghong, Zhu, Xiaofeng, Li, Haijiang ORCID: https://orcid.org/0000-0001-6326-8133 and Yang, Gang 2025. Multimodal deep learning-based automatic generation of repair proposals for steel bridge shallow damage. Automation in Construction 171 , 105961. 10.1016/j.autcon.2025.105961

Full text not available from this repository.

Abstract

As bridges age, manual repair decision-making methods struggle to meet growing maintenance demands. This paper develops AI systems that can imitate experts’decision processes by mining implicit relationships between bridge damage images and corresponding repair proposals. A multimodal deep learning-based end-to-end decision-making method is proposed to extract and map features of bridge damage images and repair proposal texts, automating damage repair proposal generation. The model is trained and validated using a dataset from historical inspection reports. The model’s image feature extraction is evaluated using Class Activation Mapping (CAM), while text generation achieved BLEU-1 to BLEU-4 scores of 0.76, 0.743, 0.712, and 0.705, respectively, with 82 % accuracy in human evaluation. The results indicate the model’s effectiveness in handling complex image features and generating long text, addressing challenges in automated bridge repair decision-making.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Schools > Engineering
Subjects:	T Technology > TA Engineering (General). Civil engineering (General) T Technology > TG Bridge engineering
Publisher:	Elsevier
ISSN:	0926-5805
Date of Acceptance:	7 January 2025
Last Modified:	24 Jan 2025 12:15
URI:	https://orca.cardiff.ac.uk/id/eprint/175402

Actions (repository staff only)

Edit Item