Zhengfu He

PhD student, Shanghai Innovation Institute

PhD student, Fudan University

Joined

September 2023

Names

Zhengfu He (Preferred)

Emails

****@fudan.edu.cn (Confirmed)

Personal Links

Career & Education History

PhD student

Shanghai Innovation Institute (sii.edu.cn)

2024 – Present

PhD student

Fudan University (fudan.edu.cn)

2023 – Present

Undergrad student

Fudan University (fudan.edu.cn)

2019 – 2023

Advisors, Relations & Conflicts

PhD Advisor

Xipeng Qiu

2023 – 2028

Expertise

Mechanistic Interpretability

2023 – 2027

Publications

Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition
Zhengfu He, Junxuan Wang, Rui Lin, Xuyang Ge, Wentao Shu, Qiong Tang, Junping Zhang, Xipeng Qiu
- ICLR 2026 Poster
- Readers: Everyone
Evolution of Concepts in Language Model Pre-Training
Xuyang Ge, Wentao Shu, Jiaxing Wu, Yunhua Zhou, Zhengfu He, Xipeng Qiu
- ICLR 2026 Poster
- Readers: Everyone
Attention Layers Add Into Low-Dimensional Residual Subspaces
Junxuan Wang, Xuyang Ge, Wentao Shu, Zhengfu He, Xipeng Qiu
- Submitted to ICLR 2026
- Readers: Everyone
Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition
Zhengfu He, Junxuan Wang, Rui Lin, Xuyang Ge, Wentao Shu, Qiong Tang, Junping Zhang, Xipeng Qiu
- Mech Interp Workshop (NeurIPS 2025) Spotlight
- Readers: Everyone
Attention Layers Add Into Low-Dimensional Residual Subspaces
Junxuan Wang, Xuyang Ge, Wentao Shu, Zhengfu He, Xipeng Qiu
- Mech Interp Workshop (NeurIPS 2025) Poster
- Readers: Everyone
Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures
Junxuan Wang, Xuyang Ge, Wentao Shu, Qiong Tang, Yunhua Zhou, Zhengfu He, Xipeng Qiu
- ICLR 2025
- Readers: Everyone
Correction to: MOSS: An Open Conversational Large Language Model
Tianxiang Sun, Xiaotian Zhang, Zhengfu He, Peng Li, Qinyuan Cheng, Xiangyang Liu, Hang Yan, Yunfan Shao, Qiong Tang, Shiduo Zhang, Xingjian Zhao, Ke Chen, Yining Zheng, Zhejian Zhou, Ruixiao Li, Jun Zhan, Yunhua Zhou, Linyang Li, Xiaogui Yang, Lingling Wu et al. (4 additional authors not shown)
- Mach. Intell. Res. 2024
- Readers: Everyone
MOSS: An Open Conversational Large Language Model
Tianxiang Sun, Xiaotian Zhang, Zhengfu He, Peng Li, Qinyuan Cheng, Xiangyang Liu, Hang Yan, Yunfan Shao, Qiong Tang, Shiduo Zhang, Xingjian Zhao, Ke Chen, Yining Zheng, Zhejian Zhou, Ruixiao Li, Jun Zhan, Yunhua Zhou, Linyang Li, Xiaogui Yang, Lingling Wu et al. (4 additional authors not shown)
- Mach. Intell. Res. 2024
- Readers: Everyone
Automatically Identifying and Interpreting Sparse Circuits with Hierarchical Tracing
Xuyang Ge, Wentao Shu, Junxuan Wang, Fukang Zhu, Zhengfu He, Xipeng Qiu
- ICLR 2025 Conference Withdrawn Submission
- Readers: Everyone
Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures
Junxuan Wang, Xuyang Ge, Wentao Shu, Qiong Tang, Yunhua Zhou, Zhengfu He, Xipeng Qiu
- ICLR 2025 Poster
- Readers: Everyone

View all 27 publications

Co-Authors

View all 40 co-authors