Spotting two records for the same business

You scanned a district and pulled three hundred records. The real number of businesses is two hundred and sixty. Those forty extra records are enough to make you write to the same place twice and spend your first impression on nothing.

Where the duplicates come from

A business can appear more than once in open data, and the reasons differ from each other.

Two contributors add the same place at different times: one as Ahmet Usta Kebap, the other as Ahmet Usta Kebap Salonu. Or a business moves, the old node is never deleted, and a new one goes in. And when a building and a business are tagged as separate objects, the same place comes back both as an area and as a point.

Those three want different treatment. The first two should collapse to one record; in the third there is only one business and the geometry is being counted twice.

Comparing names is not enough

The usual approach is to normalise names and compare them. Lower-case them, transliterate accented characters, strip suffixes like ltd, gmbh, salon, market, then match what is left.

It works, and it fails in both directions. Öz Urfa and Oz Urfa Kebap may well be one place; two records named Migros on the same street really are two branches. A matcher that trusts names misses the first and wrongly merges the second.

So a name should be a candidate generator, not evidence. Name similarity says “look at these together”; other fields make the decision.

The three fields that decide

The phone number. If normalised numbers match, it is the same business, with a couple of known exceptions: a chain’s call centre number and a shared switchboard in an office block. Counting how many records carry a number filters those out; a number appearing in more than five records has stopped being an identity.

Distance between coordinates. Two similarly named records within fifty metres are nearly always the same place. Two records five hundred metres apart may be two branches even with identical names. The threshold has to move with the setting: thirty metres in a dense high street, two hundred in an industrial area.

The domain. If two records’ websites resolve to the same domain, it is the same business or the same chain. A social media link does the same job and, for local businesses, turns up more often than a website.

As a rule: name similarity plus a match on any one of those three is enough to merge. Name similarity alone is not.

Which record survives a merge

Finding duplicates is half the work. Which one to keep is a decision too, and it has an easy default: not the most recently edited record, but the one with the most fields filled in.

Freshness is a tempting criterion, but in open data an edit can be one corrected tag. An older record with five populated fields is more useful than a newer one with two. The right move is to merge both: take the fuller record as the base, fill its gaps from the other, and prefer the newer value wherever the two conflict.

Keep the identifiers of the records you merged. Pull the list again a month later and the same two records come back, and you do not want to make that decision twice.

Clearing the rest by eye

Automatic matching finds most duplicates and not all of them. For the remainder there is a practical method: group the list by neighbourhood and sector, and read it side by side.

A person can tell in three seconds whether Pide Salonu and Karadeniz Pide are the same place; for a matcher to know that, you would have to hand it the street name, the door number and a photograph. On a list of thirty records that review takes ten minutes, and it stops the first message going out twice.