Another great article, thanks. I thought a lot about that too, and basically realized the same thing (this wasn't the coding agent), that frequency in the dataset mattered a lot for how useful a match is, and made frequency lists from my datasets, both on how common the literal names were and the phoneme-reduced names.
But I'm worried about that "bitter lesson" the TypeSafe CEO refers to, and that we're retreading the steps of natural language processing and a lot of other fields, trying to come up with clever rules, when the rule-based approach simply never gets good enough. I need matching to be good, it's absolutely central to genealogy, and I've seen the damage blindly linking by hand-crafted matching formulas can do.
But I'm worried about that "bitter lesson" the TypeSafe CEO refers to, and that we're retreading the steps of natural language processing and a lot of other fields, trying to come up with clever rules, when the rule-based approach simply never gets good enough. I need matching to be good, it's absolutely central to genealogy, and I've seen the damage blindly linking by hand-crafted matching formulas can do.