Junxuan Wang

PhD student, Shanghai Innovation Institute

Joined

May 2024

Names

Junxuan Wang (Preferred)

Emails

****@m.fudan.edu.cn (Confirmed)

,

****@m.fudan.edu.cn (Confirmed)

Personal Links

Career & Education History

PhD student

Shanghai Innovation Institute (sii.edu.cn)

2025 – Present

Undergrad student

Fudan University (fudan.edu.cn)

2021 – 2025

Advisors, Relations & Conflicts

PhD Advisor

2025 – Present

Coauthor

2024 – Present

Expertise

Mechanistic Interpretability

2024 – Present

Publications

Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition
Zhengfu He, Junxuan Wang, Rui Lin, Xuyang Ge, Wentao Shu, Qiong Tang, Junping Zhang, Xipeng Qiu
- ICLR 2026 Poster
- Readers: Everyone
Attention Layers Add Into Low-Dimensional Residual Subspaces
Junxuan Wang, Xuyang Ge, Wentao Shu, Zhengfu He, Xipeng Qiu
- Submitted to ICLR 2026
- Readers: Everyone
Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition
Zhengfu He, Junxuan Wang, Rui Lin, Xuyang Ge, Wentao Shu, Qiong Tang, Junping Zhang, Xipeng Qiu
- Mech Interp Workshop (NeurIPS 2025) Spotlight
- Readers: Everyone
Attention Layers Add Into Low-Dimensional Residual Subspaces
Junxuan Wang, Xuyang Ge, Wentao Shu, Zhengfu He, Xipeng Qiu
- Mech Interp Workshop (NeurIPS 2025) Poster
- Readers: Everyone
Automatically Identifying and Interpreting Sparse Circuits with Hierarchical Tracing
Xuyang Ge, Wentao Shu, Junxuan Wang, Fukang Zhu, Zhengfu He, Xipeng Qiu
- ICLR 2025 Conference Withdrawn Submission
- Readers: Everyone
Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures
Junxuan Wang, Xuyang Ge, Wentao Shu, Qiong Tang, Yunhua Zhou, Zhengfu He, Xipeng Qiu
- ICLR 2025 Poster
- Readers: Everyone
Automatically Identifying Local and Global Circuits with Linear Computation Graphs
Xuyang Ge, Fukang Zhu, Wentao Shu, Junxuan Wang, Zhengfu He, Xipeng Qiu
- ICML 2024 MI Workshop Poster
- Readers: Everyone
Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders
Zhengfu He, Wentao Shu, Xuyang Ge, Lingjie Chen, Junxuan Wang, Yunhua Zhou, Frances Liu, Qipeng Guo, Xuanjing Huang, Zuxuan Wu, Yu-Gang Jiang, Xipeng Qiu
- CoRR 2024
- Readers: Everyone
Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures
Junxuan Wang, Xuyang Ge, Wentao Shu, Qiong Tang, Yunhua Zhou, Zhengfu He, Xipeng Qiu
- CoRR 2024
- Readers: Everyone
Automatically Identifying Local and Global Circuits with Linear Computation Graphs
Xuyang Ge, Fukang Zhu, Wentao Shu, Junxuan Wang, Zhengfu He, Xipeng Qiu
- CoRR 2024
- Readers: Everyone

Co-Authors