ProteiNexus: Illuminating Protein Pathways through Structural Pre-training
Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Protein Representation Learning, Large-Scale 3D Protein Pretraining, Structural biology
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: Protein representation learning has emerged as a powerful tool for various biological tasks. Language models derived from protein sequences represent the predominant trend in many current approaches. However, recent advances reveal that protein sequences alone cannot fully encapsulate the abundant information contained within protein structures, critical for understanding protein function and aiding innovative protein design. In this study, we present ProteiNexus, an innovative approach, effectively integrating protein structure learning with numerous downstream tasks. We propose a structural encoding mechanism adept at capturing fine-grained distance details and spatial positioning. By implementing a robust pre-training strategy and fine-tuning with lightweight decoders designed for specific downstream tasks, our model exhibits outstanding performance, establishing new benchmarks across a range of tasks. The code and models could be found at github repos.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 5058
Loading