Smaller KANs Outperform Larger MLPs with Faster Scaling and Better Interpretability

May 1, 2024, 04:40 AM

Researchers have introduced a new neural network architecture, Kolmogorov-Arnold Networks (KAN), as a promising alternative to traditional Multi-Layer Perceptrons (MLPs). KANs, inspired by the Kolmogorov-Arnold representation theorem, offer significant improvements in accuracy and interpretability over MLPs. These networks place learnable activation functions on weights rather than on neurons, which has been shown to achieve better performance in tasks such as data fitting and solving partial differential equations (PDEs). Additionally, KANs can be intuitively visualized, enhancing their interpretability. Notably, much smaller KANs can achieve comparable or better accuracy than much larger MLPs, and they possess faster neural scaling laws.