How difficult is it to setup a local MusicBrainz server?

Tags: #<Tag:0x00007f756c1efe18>

I read about the possibility to setup your own local MusiBrainz server.
It might be an interesting thing for me to try, but…

While I am savvy with Windows, my experience with- and talent for Linux is less then mediocre.
In the past I have build a nas-server running on NAS4Free (freebsd based), and I managed to keep it running and updated for many years, but using ssh, console and related Linux commands where a big challenge and a kind of self-torture for me.
(I abandoned it after a while and am now using a Windows Server nas)

So, my question is this:

For as far as I understand after a quick-read, I could setup a Windows pc for the task, use VirtualBox , and then load some sort of a largely pre-configured MB database image?

Considering my above admittance of being terrible with Linux, and usually only succeeding after lots of help from experienced users that are willing and able to give some sort of spoon-feeding support, would it be better for me to just forget about even trying this, or is it not that difficult and would I have a chance in getting it to work?

1 Like

What are you trying to achieve by running your own server is the question I would be looking at.

  • Do you want a replica database that you can run sql queries?
  • Do you want a replica database and server that you can point picard to and query the api?

The vm is designed to allow you to download and run it but you need to be a little mindful of how the network works on the vm.
You will be using NAT so the vm will sit inside it’s own network and be able to connect outwards to download replication packets without too much work.
Configuring the networking in the vm to forward certain ports needs to be set up to allow your pc to connect into the vm.
The vm also uses a technology called docker and this is doing it’s own networking so you need to run a few commands inside the vm to expose the ports for the database if you want to connect this.

If all you want is a sql database https://github.com/lalinsky/mbdata may be an option but I do not have experience with this.

2 Likes

If you try to setup a local MB Server on Linux (like Ubuntu) it’s very complicated to get it running from the source code.

BUT:
You can use the (nearly) ready-to-use Virtual Machine in your mentioned VirtualBox. There is only one drawback: This VM is highly outdated and you need some manual steps to get it up, running and actualized.

2 Likes

That’s a very good question, and the answer is: I am not really sure.

I wasn’t able to find a layman’s explanation on why you would want to create a local MB database, so I started guessing.

My idea/intention is this:

Once in a while I would want to do a streak of updating my music library through Picard.
The implemented delay of 1 second per track can already make that a time-consuming job.
But especially with classical music and using some plugins, it will take a lot longer.
(that probably has to do with lots of ‘relationships’)
And the same goes for separate tracks and compilation albums.
They can also take a very long time to get matched.

So I am guessing that if I had the MB database locally, most of that delay would disappear.

Is this a correct assumption?

1 Like

Yes, but not by default. You have to adjust this rate limit in Picard too.

And you need a very fast machine with a lot of RAM to get your results faster locally then from MB.

5 Likes

Thanks InvisableMan78,

So if speed gains are not that obvious or easily achieved, what are other reasons for users to setup a local MB database?

In my case there are 2 main reasons:

a) I bulk tag my files with my own commandline tagging program, direct accessing the local MB database (query the VM version with SQL queries) getting only the few metadata I really need.

b) I fill an additional SQL table in the local MB database with all my local song filenames to access them more easily (search & play).

2 Likes

I would like to add main reason why I use local vm, at times I don’t have web connection at remote place that I often visit.

1 Like

By chance, just how much is ‘a lot’ with reference to running a local instance of the MB server?

I gave up on it after three days of seeing no progress once it started replicating. I wanted to take the VM and image it back to real hardware just so I could actually see if it was doing anything or not.

My desire for the local instance is along the lines of the reason’s you’ve given, though mainly b) with a focus towards learning more about how Picard interacts with MB as well.

The more RAM the better. I would not start it with less then 8 GB.
If you want to run your own search server too, I would suggest at least 16 GB RAM.
(The mentioned 4 GB RAM and 2 Cores from the documentation are the technical minimum, don’t even try it.)

You can open a second terminal window and enter this command line:
sudo docker-compose exec musicbrainz /usr/bin/tail -f slave.log
Then you can see the processing of every single replication packet (produced every full hour).

BUT:
Don’t start to replicate from 2018-08-14, this will take forever.
Download and import a so called “Full Data Dump” - only some days old. Replicate then the missing packets until now.

3 Likes

My experience in terms of resources is not too bad. I’m using lenovo thinkcentre i7 with 8GB and SSD for linux host, 4 cores.

By chance, just how much is ‘a lot’ with reference to running a local instance of the MB server?

VM is allocated at 4GB and 2 CPU’s, which gives you 4 threads.

This VM is highly outdated and you need some manual steps to get it up, running and actualized.

It did run 3 days… but it eventually finished. I had to do a schema upgrade for May 2019, I’ve found instructions in the forum. Index rebuild has run in hours.

Overall, this local instance is bit snappier than the master, replicates on schedule. Quite a lot of fun actually.

3 Likes

Thanks for the feedback and information everybody!
I can now be fairly certain that this would not be a project I would enjoy much, nor could bring to a happy end.