Activation with Intrinsic-Extrinsic Consensus

ICLR 2026 Conference Submission1653 Authors

Keywords: artificial neural network, activation, intrinsic-extrinsic consensus, channel gatekeeping, neural response, neural representation, deep learning

Abstract: Artificial Neural Networks (ANNs) are powerful tools for complex pattern recognition and decision-making. While existing activation mechanisms often promote sparsity through thresholding, they lack an explicit assessment of channel relevance, making networks susceptible to interference from noisy channels. Such irrelevant activations can propagate through the network and adversely affect the final decision. Inspired by observations that channel relevance can be assessed from both intrinsic activity levels and extrinsic decision weights---and that a strong consensus exists between these two aspects---this paper proposes AIEC (Activation with Intrinsic-Extrinsic Consensus), a novel activation mechanism designed to identify and suppress irrelevant channels during training. AIEC consists of three components: an intrinsic Activation-Counting Unit that tracks channel activation statistics, an extrinsic Decision-Making Unit that learns channel decision weights, and a Consensus Gatekeeping Unit that suppresses irrelevant channels based on the agreement between the intrinsic and extrinsic assessments. Extensive experiments demonstrate that AIEC effectively suppresses irrelevant channels and facilitates sparser neural representations. Furthermore, AIEC is compatible with a wide range of mainstream ANN architectures and achieves superior performance compared to existing activation mechanisms across multiple tasks and domains.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 1653