I disagree. A lack of votes should not be an assumption of wrong data. I am often working in obscure artists doing edits that never get votes. Probably due to people not always knowing much about these artists. Rejecting edits because they are being done in an unpopular area is wrong.
Sock puppet accounts would soon stand out. You would especially notice them as you see so many of the edits yourself. If the same users are always voting for the same users then it soon gets obvious.
I do agree with the frustration of the notes system. I loose count of the number of times I have tried to engage someone in conversation due to bad data or bad edits. Usually starting with Abstain votes to go with a comment. Only kicking that to a No if they clearly break the guidelines or are known bad data.
That looks perfect to me