Using beets for the first pass can save a lot of time. beets is an automatic tagger with a command-line interface. It queries MusicBrainz, but it can also look at other sources (such as Discogs) if you install the plugins.
I would point beets at the entire collection, and set it to auto-tag albums that are a near-certain match (the threshold is adjustable). If you know where most of your tracks came from, you can improve the matching accuracy even more by setting your preferred release country and media (CD, digital media, etc.) in yours beets config file.
The processing is pretty fast, and you can watch its progress. You can also stop and resume the whole tagging job. When it finishes, use Picard to make adjustments and to tag the remainder of your collection. beets makes that job much smaller by automating the easy decisions.
Here’s how beets handles duplicates. There is also a Duplicates plugin that you can use to audit your collection. These may or may not meet your needs depending on what you consider to be a duplicate. See also this thread and its links.
Even though Picard and beets detect files that already have MusicBrainz tags, telling the tagger to move processed files into a new directory structure will probably help you manage the job. Keeping your inbox separate is a more efficient way to work.