Publications

Focusing on the domain of historical text in English, this paper demonstrates that pre-training on domain-specific (i.e. historical) data from scratch yields a generally stronger background model than adapting a present-day language model. We show this on the basis of a variety of downstream tasks, ranging from established tasks such as Part-of-Speech tagging, Named Entity Recognition and Word Sense Disambiguation, to ad-hoc tasks like Sentence Periodization, which are specifically designed to test historically relevant processing.

This paper presents “MacBERTh”, a transformer-based language model pre-trained on historical English, and exhaustively assess its benefits on a large set of relevant downstream tasks. Our experiments highlight that, despite some differences across target time periods, pre-training on historical language from scratch outperforms models pre-trained on present-day language and later adapted to historical language.

The aim of this paper is to set out a ‘hands-off’ computational procedure to study specific cases of semantic change. The case we address is the development of the phrasal expression to death from a literal, resultative phrase (e.g. he was beaten to death) into an intensifier (e.g. We were just pleased to death to see her). The methodology we outline may help tackle some common challenges in the use of vector representations to study similar cases of semantic change, and pinpoint (remaining) challenges.

In this paper, we follow recent developments in non-parametric learning and show how LLMs can be efficiently fine-tuned to achieve strong few-shot performance on WSD for historical languages. We test our hypothesis using (i) a large, general evaluation set taken from large lexical databases, and (ii) a small real-world scenario involving an ad-hoc WSD task. Moreover, this paper marks the release of GysBERT, a LLM for historical Dutch (1500-1950).