LibraGrad: Balancing gradient flow for universally better vision transformer attributions

Mehri, Faridoun, Baghshah, Mahdieh Soleymani and Pilehvar, Mohammad T. 2025. LibraGrad: Balancing gradient flow for universally better vision transformer attributions. Presented at: 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 10-17 June 2025. IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, pp. 67-78. 10.1109/cvpr52734.2025.00016

Full text not available from this repository.

Abstract

Why do gradient-based explanations struggle with Transformers, and how can we improve them? We identify gradient flow imbalances in Transformers that violate FullGrad completeness, a critical property for attribution faithfulness that CNNs naturally possess. To address this issue, we introduce LibraGrad—a theoretically grounded post-hoc approach that corrects gradient imbalances through pruning and scaling of backward paths, without changing the forward pass or adding computational overhead. We evaluate LibraGrad using three metric families: Faithfulness, which quantifies prediction changes under perturbations of the most and least relevant features; Completeness Error, which measures attribution conservation relative to model outputs; and Segmentation AP, which assesses alignment with human perception. Extensive experiments across 8 architectures, 4 model sizes, and 5 datasets show that LibraGrad universally enhances gradient-based methods, outperforming existing white-box methods—including Transformer-specific approaches—across all metrics. We demonstrate superior qualitative results through two complementary evaluations: precise text-prompted region highlighting on CLIP models and accurate class discrimination between co-occurring animals on ImageNet-finetuned models—two settings on which existing methods often struggle. Libra-Grad is effective even on the attention-free MLP-Mixer architecture, indicating potential for extension to other modern architectures. Our code is freely available at https://nightmachinery.github.io/LibraGrad/.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Published Online
Status:	Published
Schools:	Schools > Computer Science & Informatics
Publisher:	IEEE
ISBN:	9798331543655
ISSN:	1063-6919
Last Modified:	10 Sep 2025 10:15
URI:	https://orca.cardiff.ac.uk/id/eprint/180740

Actions (repository staff only)

Edit Item