The first step is to use Picard or another MB-compatible tagger to identify your music. Even if your files already have some tags, they need MB tags in order to uniquely identify them, and to associate each one with a specific album release.
Based on the screenshot you posted, you’re on the right track if you want to use Picard. Select the folders you want to match, then click “Scan” or “Lookup” (I have more success with “Scan”).
As for the database: When a music player reads your local collection to create a library, it builds a searchable database from the tags. Quod Libet is one player that is particularly powerful for search and cataloguing. (Its database format is described here.)
There is also a third-party MB client called beets, which is an alternative to Picard. It scans your files, matches them to MB records, and creates an SQLite database of the matches. You then have the option to allow beets to change your tags to sync them with its database. You can query the database using beets, or through the SQLite client of your choice.