German’s Next Language Model | Awesome LLM Papers Add your paper to Awesome LLM Papers

Awesome LLM Papers

A continuously updated collection of research papers on LLMs. Maintained by Sean Moran.

Stay Updated

Contact Sean Moran about this survey or website. Made with Jekyll and Hyde.

German's Next Language Model

Branden Chan, Stefan Schweter, Timo Möller . Proceedings of the 28th International Conference on Computational Linguistics 2020 – 75 citations

[Paper]   Search on Google Scholar   Search on Semantic Scholar
Coling Compositional Generalization Datasets Evaluation Interdisciplinary Approaches Model Architecture Multimodal Semantic Representation Training Techniques Visual Question Answering

In this work we present the experiments which lead to the creation of our BERT and ELECTRA based German language models, GBERT and GELECTRA. By varying the input training data, model size, and the presence of Whole Word Masking (WWM) we were able to attain SoTA performance across a set of document classification and named entity recognition (NER) tasks for both models of base and large size. We adopt an evaluation driven approach in training these models and our results indicate that both adding more data and utilizing WWM improve model performance. By benchmarking against existing German models, we show that these models are the best German models to date. Our trained models will be made publicly available to the research community.

Similar Work