Even in that case it is often possible to still merge it… I just clean up my mess and bit and then merge it into the good one.
I think the only place I hit a Delete is when I find an orphan’d recording with no links to a release but 10 AcoustIDs attached. In that example the AcoustIDs would cause a misleading mess if I tried to merge the recording into something just to preserve a recording MBID. So in that one case I bring out the flame thrower.
And some of those awful messed up DiscIDs that are clearly badly bogus and miles off the release… but that is a separate topic.