Do Deep Neural Network Solutions Form a Star Domain?

In this session Ankit Sonthalia presented us his work with coauthors about star domains formed by deep neural networks trained with SGD. The usual perspective is that SGD solutions are fall in one convex area when permutations are taken into account, i.e., all the solutions are linearly connected without barriers in loss. Nevertheless, that is not the case for narrow neural networks and Ankit puts forward a conjecture that the solutions rather form a star domain with one star connected network. The discussion was based on the ICLR 2025 paper “Do Deep Neural Network Solutions Form a Star Domain?”.

Presentation can be found here.