Mehri, Faridoun, Baghshah, Mahdieh Soleymani and Pilehvar, Mohammad T.
2025.
LibraGrad: Balancing gradient flow for universally better vision transformer attributions.
Presented at: 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),
Nashville, TN, USA,
10-17 June 2025.
IEEE Computer Society Conference on Computer Vision and Pattern Recognition.
IEEE,
pp. 67-78.
10.1109/cvpr52734.2025.00016
|
Preview |
PDF
- Accepted Post-Print Version
Download (39MB) | Preview |
Abstract
Why do gradient-based explanations struggle with Transformers, and how can we improve them? We identify gradient flow imbalances in Transformers that violate FullGrad completeness, a critical property for attribution faithfulness that CNNs naturally possess. To address this issue, we introduce LibraGrad—a theoretically grounded post-hoc approach that corrects gradient imbalances through pruning and scaling of backward paths, without changing the forward pass or adding computational overhead. We evaluate LibraGrad using three metric families: Faithfulness, which quantifies prediction changes under perturbations of the most and least relevant features; Completeness Error, which measures attribution conservation relative to model outputs; and Segmentation AP, which assesses alignment with human perception. Extensive experiments across 8 architectures, 4 model sizes, and 5 datasets show that LibraGrad universally enhances gradient-based methods, outperforming existing white-box methods—including Transformer-specific approaches—across all metrics. We demonstrate superior qualitative results through two complementary evaluations: precise text-prompted region highlighting on CLIP models and accurate class discrimination between co-occurring animals on ImageNet-finetuned models—two settings on which existing methods often struggle. Libra-Grad is effective even on the attention-free MLP-Mixer architecture, indicating potential for extension to other modern architectures. Our code is freely available at https://nightmachinery.github.io/LibraGrad/.
| Item Type: | Conference or Workshop Item (Paper) |
|---|---|
| Date Type: | Published Online |
| Status: | Published |
| Schools: | Schools > Computer Science & Informatics |
| Publisher: | IEEE |
| ISBN: | 9798331543655 |
| ISSN: | 1063-6919 |
| Date of First Compliant Deposit: | 10 December 2025 |
| Date of Acceptance: | 26 February 2025 |
| Last Modified: | 10 Dec 2025 14:30 |
| URI: | https://orca.cardiff.ac.uk/id/eprint/180740 |
Actions (repository staff only)
 |
Edit Item |
Altmetric
Altmetric