To Transfer or Not to Transfer: Unified Transferability Metric and Analysis

What the paper says

Transferability estimation is a fundamental problem in transfer learning, which aims to predict whether transferring knowledge from a source domain will improve performance on a target task. Existing research focuses on classification and neglects domain/task differences, as well as only very limited research for regression. Most importantly, there is a lack of research to determine whether to transfer or not. To address these gaps, we propose Wasserstein distance-based joint estimation (WDJE), a unified transferability metric for both classification and regression under domain and task differences. WDJE facilitates decision-making on whether to transfer by comparing the target risk with and without transfer. To enable this comparison, we estimate the unobservable post-transfer risk using a nonsymmetric, interpretable, and easy-to-calculate upper bound that remains applicable even with limited target labels. The proposed bound relates the target transfer risk to source model performance, domain, and task differences based on the Wasserstein distance. We further extend the proposed bound to the unsupervised setting and establish a generalization bound from finite empirical samples. We evaluate WDJE and the proposed risk bound across 42 transfer scenarios, including CIFAR-100 (CF100) and Office-Home image classification and C-MAPSS remaining-useful-life regression prediction. WDJE achieves a perfect consistency index ( $CI$ ) of 1 in 25 cases and an overall mean $CI$ of 0.89, accurately suggesting when transfer should (or should not) be performed. The proposed bound achieves the average Pearson correlations of 0.99 on CF100, 0.72 on Office-Home, and 0.96 on C-MAPSS, illustrating state-of-the-art performance in approximating the true post-transfer risk.