Papers
1–2 of 2Research Paper·Feb 4, 2026·B2B
It's not a Lottery, it's a Race: Understanding How Gradient Descent Adapts the Network's Capacity to the Task
Our theoretical understanding of neural networks is lagging behind their empirical success. One of the important unexplained phenomena is why and how, during the process of training with gradient desc...
2.0 viability
Research Paper·Feb 12, 2026
Rational Neural Networks have Expressivity Advantages
We study neural networks with trainable low-degree rational activation functions and show that they are more expressive and parameter-efficient than modern piecewise-linear and smooth activations such...
2.0 viability