Mode connectivity refers to the phenomenon that different optima in weight space are connected by simple continuous curves (low-dimensional splines) along which the training and test loss remains nearly constant.
1 min read
Mode connectivity refers to the phenomenon that different optima in weight space are connected by simple continuous curves (low-dimensional splines) along which the training and test loss remains nearly constant.