They're probably not paying a mortgage on a property that may have been purchased decades ago, especially if they were buying cheap land before it came into high demand. Insurance and security for vacant buildings and especially vacant lots is pennies; certainly much cheaper than for a developed property with tenants. The only serious cost of a non-productive property for a landowner are property taxes, which are currently structured such that developing a property causes them to go up, while leaving a property undeveloped keeps taxes low.
Taxing the land value is a major change to the incentive structure.
Taxing the land value is a major change to the incentive structure.