Beyond Shannon: Semantic Information Theory and Methodology

Ping Zhang, Kai Niu, Zijian Liang, Changshuo Wang, Jiatong Wu, Yiming Liu, Wenjun Xu, Nan Ma, Xiaodong Xu, Ruichen Zhang

Abstract: The convergence of artificial intelligence (AI) and information theory has ushered in a new paradigm that extends Shannon’s framework toward the realm of meaning. This paper presents a comprehensive overview of semantic information theory, which aims to rigorously quantify, transmit, and optimize information that carries semantic and task-relevant significance. Beginning with the limitations of classical information theory, the paper traces the historical development of semantic frameworks and synthesizes recent advances into a unified theoretical foundation. Adopting a synonymity-based viewpoint, we systematically introduce central concepts such as synonymous mapping, semantic entropy, and semantic mutual information. These concepts form the basis for generalized semantic source, channel, and rate–distortion coding theorems that extend Shannon’s classical results and provide a principled methodology for designing semantic communication systems. Finally, representative architectures and methods are discussed to illustrate how these theoretical insights can guide AI-empowered communication and networking, outlining a path toward intelligent, interpretable, and meaning-centric communication systems that move beyond Shannon.

External IDs:dblp:journals/tnse/ZhangNLWWLXMXZ26