Zhengyan Zhang

Researcher, Deepseed AI

Joined

July 2019

Names

Zhengyan Zhang

Emails

****@mails.tsinghua.edu.cn (Confirmed)

****@gmail.com (Confirmed)

Personal Links

Semantic Scholar

Career & Education History

Researcher

Deepseed AI (deepseek.com)

2024 – Present

PhD student

Department of Computer Science and TechnologyCHina, Tsinghua University (tsinghua.edu.cn)

2019 – 2024

Advisors, Relations & Conflicts

PhD Advisor

Zhiyuan Liu

2019 – Present

Expertise

Natural Language Processing

2018 – Present

Pre-trained Language Models

2018 – Present

Publications

InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
Chaojun Xiao, Pengle Zhang, Xu Han, Guangxuan Xiao, Yankai Lin, Zhengyan Zhang, Zhiyuan Liu, Maosong Sun
- LCFM 2024
- Readers: Everyone
InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
Chaojun Xiao, Pengle Zhang, Xu Han, Guangxuan Xiao, Yankai Lin, Zhengyan Zhang, Zhiyuan Liu, Maosong Sun
- NeurIPS 2024 poster
- Readers: Everyone
CA-LoRA: Adapting Existing LoRA for Compressed LLMs to Enable Efficient Multi-Tasking on Personal Devices
Weilin Zhao, Yuxiang Huang, Xu Han, Zhiyuan Liu, Zhengyan Zhang, Kuai Li, Chen Chen, TAO YANG, Maosong Sun
- COLM
- Readers: Everyone
Exploring the Benefit of Activation Sparsity in Pre-training
Zhengyan Zhang, Chaojun Xiao, Qiujieli Qin, Yankai Lin, Zhiyuan Zeng, Xu Han, Zhiyuan Liu, Ruobing Xie, Maosong Sun, Jie Zhou
- ICML 2024 Poster
- Readers: Everyone
Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules
Chaojun Xiao, Yuqi Luo, Wenbin Zhang, Pengle Zhang, Xu Han, Yankai Lin, Zhengyan Zhang, Ruobing Xie, Zhiyuan Liu, Maosong Sun, Jie Zhou
- EMNLP 2023 Findings
- Readers: Everyone
Finding Skill Neurons in Pre-trained Transformer-based Language Models
Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu, Juanzi Li
- 08 Jun 2023
- OpenReview Archive Direct Upload
- Readers: Everyone
Emergent Modularity in Pre-trained Transformers
Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Chaojun Xiao, Xu Han, Zhiyuan Liu, Maosong Sun, Jie Zhou
- 22 Sept 2022 (modified: 14 Jan 2026)
- ICLR 2023 Conference Withdrawn Submission
- Readers: Everyone
KEPLER: A unified model for knowledge embedding and pre-trained language representation
Xiaozhi Wang, Tianyu Gao, Zhaocheng Zhu, Zhengyan Zhang, Zhiyuan Liu, Juanzi Li, Jian Tang
- 13 Oct 2021 (modified: 24 Apr 2023)
- OpenReview Archive Direct Upload
- Readers: Everyone
Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks
Zhengyan Zhang, Guangxuan Xiao, Yongwei Li, Tian Lv, Fanchao Qi, Zhiyuan Liu, Yasheng Wang, Xin Jiang, Maosong Sun
- Published: 28 Jan 2022, Last Modified: 12 Oct 2025
- ICLR 2022 Submitted
- Readers: Everyone
CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models
Yuan Yao, Ao Zhang, Zhengyan Zhang, Zhiyuan Liu, Tat-Seng Chua, Maosong Sun
- Published: 28 Jan 2022, Last Modified: 12 Oct 2025
- ICLR 2022 Submitted
- Readers: Everyone

View all 29 publications

Co-Authors

View all 89 co-authors