medBERT.de: A comprehensive German BERT model for the medical domain

Keno K. Bressem, Jens-Michalis Papaioannou, Paul Grundmann, Florian Borchert, Lisa C. Adams, Leonhard Liu, Felix Busch, Lina Xu, Jan P. Loyen, Stefan Markus Niehues, Moritz Augustin, Lennart Grosser, Marcus R. Makowski, Hugo J. W. L. Aerts, Alexander Löser

Abstract: Highlights•medBERT.de: German BERT model tailored for medical domain expertise.•Trained using 4.7 million diverse medical documents for optimal results.•Achieved new state-of-the-art on eight medical NLP benchmarks.•Efficient tokenization has minor role, focus on domain-specific training.•Public release of pre-trained model weights and new benchmarks for research.