Morphological Tagging and Lemma Selection
The Humit tagger is based on a neural network, more precisely the pre-trained Norwegian NorBERT model (huggingface.co), which was developed at the Department of Informatics at the University of Oslo. By fine-tuning this base model, the tagger performs four tasks: so-called token classifiers identify sentence boundaries, select the correct written standard (Bokmål or Nynorsk), and assign the correct morphological tags. For lemma selection, a hybrid system is used, where the Humit tagger employs the full-form lexicon *Norsk ordbank* together with the neural model.
The morphological tagging is done first. After that, the word form and morphological tag are sent to the full-form lexicon. If the tagger finds a match for the combination of word form and morphological tag, the word form is assigned the lemma from the lexicon. If there are several possible lemmas (for example, both “list” and “liste” can be lemmas of “lister” as a plural noun), the neural model decides which one to choose. The neural model also finds the lemma of compounds and other words that are not listed in the lexicon.
Evaluation
The Humit tagger for Bokmål and Nynorsk is trained and tested on the Norwegian Dependency Treebank. For the test set, the tagger achieves an overall accuracy score of 98.27% on the morphological tags.
For the lemmatisation component, the calculation is more complex. Both the Bokmål and Nynorsk standards allow for considerable variation. For the present-tense form “kjem”, for instance, “kome”, “koma”, “komme”, and “komma” are all correct lemma forms. In the test set of the Norwegian Dependency Treebank, however, only one of these lemma forms is marked as correct. Even so, the lemma model achieves an accuracy of 99.2% for Bokmål and 98.5% for Nynorsk on this test set. A closer look at the errors shows that many of them are in fact not errors, but rather instances of variation within the standard or mistakes in the gold standard (NDT). After the error analysis, the accuracy figures are 99.5% for Bokmål and 99.3% for Nynorsk. See the detailed error analysis in the article “Automatic Lemmatisation for Norwegian” (Yıldırım, Hagen, Haug 2026).
Funding
The Humit tagger was developed with support from the infrastructure project Clarino+ (clarin.w.uib.no) and ILN .
How to refer to the tagger:
Haug, D. T. T., Yildirim, A., Hagen, K., & Nøklestad, A. (2023). Rules and neural nets for morphological tagging of Norwegian - Results and challenges. In NEALT Proceedings Series, 425-435.
Yildirim, A., Hagen, K., & Haug, D. T. T. (2026). Automatic Lemmatisation for Norwegian. In Proceedings of the Workshop in Structured Linguistic Data and Evaluation, pages 93-103 May 11, 2026.
When you register publications in NVA, please mention the following projects and project number: CLARINO: Norges Forskningsråd 29 57 00