ReLU is all you need for NASWOT

Prit Kanadiya, Raghav Agarwal, Om Doiphode, Sandip Shingade.

Submission Type: Short paper

Tldr: We generalize NASWOT to support diverse activation functions for training-free performance prediction, but surprisingly find that ReLU alone retains most of the relevant information across NLP benchmarks.

Submission Number: 24