C-Pack: Packaged Resources To Advance General Chinese Embedding

Anonymous

Abstract: We introduce C-Pack, a package of resources that significantly advance the field of general Chinese embeddings. C-Pack includes three critical resources. 1) C-MTEB is a comprehensive benchmark for Chinese text embeddings covering 6 tasks and 35 datasets. 2) C-MTP is a massive text embedding dataset curated from labeled and unlabeled Chinese corpora for training embedding models. 3) C-TEM is a family of embedding models covering multiple sizes. Our models outperform all prior Chinese text embeddings on C-MTEB by up to +10% upon the time of the release. We also integrate and optimize the entire suite of training methods for C-TEM. Along with our resources on general Chinese embedding, we release our data and models for English text embeddings. The English models outperform all existing embedding models on the MTEB benchmark; meanwhile, our released English data is 2 times larger than the Chinese data. All these resources will be made publicly available.

Paper Type: long

Research Area: Resources and Evaluation

Contribution Types: Data resources

Languages Studied: Chinese, English