So as a recap for anyone else who might have this problem:
I had lots of files with incomplete, bad, or no tag at all. I used scan to get the artists and titles (doesn’t always work right though, specially for non-English songs so do in small batches and double check them). At this point the sliders people talk about doesn’t work because tracks are being matched only by their finger print which probably has matched to a collection, thanks to people who have submitted every fingerprint for collections. Then I removed almost all the tags except for the title and artist, drag them back to unmatched on the left and hit lookup. Now the sliders come into play and most likely you’ll get matched with the correct album rather than some random collection. If a song still gets tracked to a collection, drag it back to unmatched, google the album and add it to the tag, then lookup again.
PS: If you’re not sure how fingerprints work or haven’t struggled enough with them to understand them, please don’t submit any.
@Aerozol: I never submitted any fingerprint ID before, but seeing such a problem, when a song does not match well and I have to do a lot of digging before it matches, I submit the fingerprint so others won’t have this problem, or at least they choose by sliders if they want to get rid of those useless collections.