Evaluating the Impact of OCR Errors on Topic Modeling

Stephen Mutuvi (1) , Antoine Doucet (2) , Moses Odeo (1) , Adam Jatowt (3)

1 MMU - Multimedia University
2 L3I - Laboratoire Informatique, Image et Interaction - EA 2118
3 Kyoto University

Historical documents pose a challenge for character recognition due to various reasons such as font disparities across different materials, lack of orthographic standards where same words are spelled differently, material quality and unavailability of lexicons of known historical spelling variants. As a result, optical character recognition (OCR) of those documents often yield unsatisfactory OCR accuracy and render digital material only partially discoverable and the data they hold difficult to process. In this paper, we explore the impact of OCR errors on the identification of topics from a corpus comprising text from historical OCRed documents. Based on experiments performed on OCR text corpora, we observe that OCR noise negatively impacts the stability and coherence of topics generated by topic modeling algorithms and we quantify the strength of this impact.

Mots clés: Topic modeling; Topic coherence; Text mining Topic stability

Domaines: Traitement du texte et du document; Apprentissage [cs.LG]; Recherche d'information [cs.IR]; Interface homme-machine [cs.HC]; Bibliothèque électronique [cs.DL]; Informatique et langage [cs.CL]; Intelligence artificielle [cs.AI]

Stephen Mutuvi, Antoine Doucet, Moses Odeo, Adam Jatowt. Evaluating the Impact of OCR Errors on Topic Modeling. Maturity and Innovation in Digital Libraries. 20th International Conference on Asia-Pacific Digital Libraries, ICADL 2018, Hamilton, New Zealand, November 19-22, 2018, Proceedings, pp.3 - 14, 2018, ⟨10.1007/978-3-030-04257-8_1⟩. ⟨hal-03025563⟩