Going Beyond Linear Mode Connectivity: The Layerwise Linear Feature Connectivity 2023-07-20T17:38:13.476Z

Comment by zhanpeng_zhou on Going Beyond Linear Mode Connectivity: The Layerwise Linear Feature Connectivity · 2024-01-11T11:52:13.836Z · LW · GW

Great comments! Actually, we have made some progress in linking task arithmetic with our NeurIPS 2023 results and we are working on a new manuscript to introduce our new results. Hope our new paper could be released soon.

Comment by zhanpeng_zhou on Going Beyond Linear Mode Connectivity: The Layerwise Linear Feature Connectivity · 2023-07-21T02:05:34.818Z · LW · GW

Actually, someone found the Mode Connectivity in ViT (a quite larger NN I thought). Not sure if LLFC will be still satisfied on ViT but it worth a try. As for the pruned network, I believe that pruned network still holds the LLFC. (I think you refer the LLFC as "weak additivity", right?)