Papers
arxiv:2508.05073

ULU: A Unified Activation Function

Published on Aug 7, 2025
Authors:

Abstract

A new non-monotonic piecewise activation function, ULU, and its adaptive variant AULU are introduced, with AULU enabling separate learnable responses for positive and negative inputs and providing a metric to quantify model inductive bias, outperforming ReLU and Mish on image classification and object detection.

We propose ULU, a novel non-monotonic, piecewise activation function defined as {f(x;α_1),x<0; f(x;α_2),x>=0 }, where f(x;α)=0.5x(tanh(αx)+1),α>0. ULU treats positive and negative inputs differently. Extensive experiments demonstrate ULU significantly outperforms ReLU and Mish across image classification and object detection tasks. Its variant Adaptive ULU (AULU) is expressed as {f(x;β_1^2),x<0; f(x;β_2^2),x>=0 }, where β_1 and β_2 are learnable parameters, enabling it to adapt its response separately for positive and negative inputs. Additionally, we introduce the LIB (Like Inductive Bias) metric from AULU to quantitatively measure the inductive bias of the model.

Community

Sign up or log in to comment

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2508.05073 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2508.05073 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2508.05073 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.