Abstract

Traditional active machine learning (AML) methods employed in Record Linkage (RL) or Entity Resolution (ER) tasks often struggle with model stability, slow convergence, and handling imbalanced data. Our study introduces a novel hybrid Active Machine Learning approach to address RL, overcoming the challenges of limited labeled data and imbalanced classes. By combining and balancing informativeness, which selects record pairs to reduce model uncertainty, and representativeness, which ensures the chosen pairs reflect the overall dataset patterns, our hybrid approach, called Hybrid Active Machine Learning for Imbalanced Record Linkage (HAML-IRL), demonstrates significant advancements.HAML-IRL achieves an average 12% improvement in F1-scores across eleven real-world datasets, including structured, textual, and dirty data, when compared to state-of-the-art AML methods. Our approach also requires up to 60% - 85% fewer labeled samples dependening on the datasets, accelerates model convergence, and offers superior stability across iterations, making it a robust and efficient solution for real-world record linkage tasks.

Key words: Record Linkage, Entity Resolution, Active Machine Learning, Hybrid Query.