Zhao, Ganlong, Li, Guanbin, Qin, Yipeng  ORCID: https://orcid.org/0000-0002-1551-9126 and Yu, Yizhou
2023.
Improved distribution matching for dataset condensation.
Presented at: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023,
Vancouver, Canada,
18 - 22 June 2023.
Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.
IEEE,
pp. 7856-7865.
10.1109/CVPR52729.2023.00759
|
Preview |
PDF
- Presentation
Download (776kB) | Preview |
Abstract
Dataset Condensation aims to condense a large dataset into a smaller one while maintaining its ability to train a well-performing model, thus reducing the storage cost and training effort in deep learning applications. However, conventional dataset condensation methods are optimization-oriented and condense the dataset by performing gradient or parameter matching during model optimization, which is computationally intensive even on small datasets and models. In this paper, we propose a novel dataset condensation method based on distribution matching, which is more efficient and promising. Specifically, we identify two important shortcomings of naive distribution matching (i.e., imbalanced feature numbers and unvalidated embeddings for distance computation) and address them with three novel techniques (i.e., partitioning and expansion augmentation, efficient and enriched model sampling, and class-aware distribution regularization). Our simple yet effective method outperforms most previous optimization-oriented methods with much fewer computational resources, thereby scaling data condensation to larger datasets and models. Extensive experiments demonstrate the effectiveness of our method. Codes are available at https://github.com/uitrbn/IDM
| Item Type: | Conference or Workshop Item - published (Paper) |
|---|---|
| Date Type: | Publication |
| Status: | Published |
| Schools: | Schools > Computer Science & Informatics |
| Publisher: | IEEE |
| ISBN: | 9798350301298 |
| Date of Acceptance: | 27 February 2023 |
| Last Modified: | 26 Mar 2026 14:37 |
| URI: | https://orca.cardiff.ac.uk/id/eprint/158033 |
Actions (repository staff only)
 |
Edit Item |
Altmetric
Altmetric