Digital twin and multimodal neural networks for automated coastal railway bridge maintenance

Khudhair, Ali ORCID: https://orcid.org/0000-0002-5062-7448, Zhu, Xiaofeng, Li, Haijiang ORCID: https://orcid.org/0000-0001-6326-8133, Ahmadian, Reza ORCID: https://orcid.org/0000-0003-2665-4734, Adeagbo, Mujib and Liu, Jiucai 2025. Digital twin and multimodal neural networks for automated coastal railway bridge maintenance. Presented at: EG-ICE 2025, Glasgow, UK, 1-3 July 2025. Published in: Moreno-Rangel, Alejandro and Kumar, Bimal eds. EG-ICE 2025. Glasgow: University of Strathclyde Publishing, 10.17868/strath.00093263

PDF - Published Version Available under License Creative Commons Attribution. Download (1MB)

Abstract

Coastal railway bridges are exposed to accelerated deterioration due to harsh marine environments, making their inspection and maintenance both costly and complex. This paper proposes a semi-automated framework that integrates Digital Twin (DT) technology with a Multimodal Neural Network (MNN) to generate natural language repair strategies directly from visual inspection data. The system combines an EfficientNet-based convolutional encoder with a Transformer decoder, trained on a domain-specific dataset of corroded bridge components annotated by experts. Unlike conventional damage detection pipelines, the proposed model outputs actionable, human-readable maintenance recommendations that are programmatically embedded into Industry Foundation Classes (IFC)-based BIM models as structured property sets. This enables seamless integration into Building Information Modelling (BIM)-based DT environments, supporting downstream decision-making and lifecycle asset management. Experimental results show that the model achieves a semantic similarity score of 0.7285 and a BLEU-3 score of 0.4193, indicating strong alignment with expert-authored strategies. While exact match accuracy is limited to 24.18%, this reflects the inherent linguistic variability in valid maintenance descriptions. The system also incorporates expert feedback to support human-in-the-loop learning and continuous improvement. These findings demonstrate the feasibility of combining DL and openBIM standards to enable scalable, automated, and semantically enriched maintenance planning for coastal railway infrastructure.

Item Type:	Conference or Workshop Item - published (Paper)
Date Type:	Publication
Status:	Published
Schools:	Schools > Engineering
Publisher:	University of Strathclyde Publishing
ISBN:	9781914241826
Date of First Compliant Deposit:	3 June 2025
Date of Acceptance:	14 May 2025
Last Modified:	28 Jan 2026 16:19
URI:	https://orca.cardiff.ac.uk/id/eprint/178718

Actions (repository staff only)

Edit Item