Local Musicbrainz Server issue

[t.taras]

Hi, everyone. I am new to Musicbrainz, and I was using the Musicbrainz API, but due to the rate limit, my command using the API would take around 80 hours to complete. I wanted to recreate the server or get the database to work locally. I have tried setting up the server four times, but every time I search for an artist, I get no results — “No results found. Try refining your search query.”

I followed all the steps from the GitHub repo’s README:

1. git clone
2. docker-compose build
3. docker-compose run --rm musicbrainz createdb.sh -fetch
4. • docker-compose run --rm musicbrainz fetch-dump.sh search
   • docker-compose run --rm search load-search-indexes.sh

However, I encountered an error related to “‘annotation’ has data already,” so I tried using --force. Besides this error, I didn’t receive any other errors.

I am going to try indexing the database, but I’m not sure if it will help.

I’m using Windows 10, which might be contributing to the issue, as I haven’t seen anyone using it on Windows. plz help:)

[yvanzo]

Hi @t.taras, your steps look fine, even using --force here was appropriate.

@t.taras

every time I search for an artist, I get no results — “No results found. Try refining your search query.”

• Is a container running for the service search? You can find it out with:

  docker-compose ps search
  

• If nothing is returned, try running it with:

  docker-compose up -d search
  

• What search query do you make? Try using this basic one instead:

  http://localhost:5000/ws/2/artist?query=artist:Tchaikovsky
  

---

@t.taras

I am going to try indexing the database, but I’m not sure if it will help.

I think that it won’t help either.

---

@t.taras

I’m using Windows 10, which might be contributing to the issue

Yes it might, but I’m not seeing anything specific to Windows in what you reported so far.

[t.taras]

@yvanzo Thank you for your response!

I decided to try setting everything up on a virtual machine with Linux Ubuntu 20.04.6 LTS x64. I followed the exact same steps as on Windows and got the same result. After running sudo docker-compose run --rm search load-search-indexes.sh, I received the message “‘annotation’ has data already,” so I tried using --force again.

I also made a request to the link you recommended to check if search is working — http://localhost:5000/ws/2/artist?query=artist:Tchaikovsky — and received the response <artist-list count="0" offset="0"/>. I’ve attached a photo with this response.

Then, I ran the command docker-compose ps search as you suggested to check if the search container was running. In the attached photo, you can see that two containers are indeed running. As for the search queries I used, I simply entered well-known names like “Drake, Michael Jackson,” etc., in the search.

The only guesses I have for why it’s not working are:

1. “docker-compose run --rm musicbrainz fetch-dump.sh search” might be downloading an incomplete dump, possibly with only a limited amount of data.
2. Since I’ve switched to a VM, we can rule out the Windows-related error.
3. The pre-built indexes might not be compatible with the dump, or there might be an error that doesn’t show up during server setup.
   If you know where I could check some logs to better understand the issue, I would appreciate any pointers.

For future tests, could you please clarify whether I need to run a command to index the tables (specifically indexing or downloading and applying pre-built indexes) to make the search function work?

Image:

[yvanzo]

@t.taras

• “docker-compose run --rm musicbrainz fetch-dump.sh search” might be downloading an incomplete dump, possibly with only a limited amount of data.

The script is checking MD5 sums, so you would not have been able to go that far.

@t.taras

• The pre-built indexes might not be compatible with the dump,

This is also checked by the scripts, and even incompatible dumps would still make search return some results for Tchaikovsky.

@t.taras

or there might be an error that doesn’t show up during server setup.
If you know where I could check some logs to better understand the issue, I would appreciate any pointers.

It seems more likely. Each Docker container has logs to be checked:

sudo docker-compose logs --follow --no-log-prefix --timestamps search

sudo docker-compose logs --follow --no-log-prefix --timestamps musicbrainz

[SanderPander]

I actually had the same issue and managed to resolve it for the most part.

When running this command

$ admin/check-search-indexes all

Revealed that indexes for Solr were not loading. This was backed up by checking the documents count in solr.

http://localhost:8983/solr/#/~cores/annotation

CORE           STATUS  INDEX  DB
editor         OK      0      /0
instrument     --      0      /1007
series         --      0      /13596
place          --      0      /47851
event          --      0      /52191
tag            --      0      /109442
area           --      0      /118536
label          --      0      /204813
cdstub         --      0      /288965
annotation     --      0      /431335
work           --      0      /1521752
artist         --      0      /1844598
release-group  --      0      /2300865
release        --      0      /2922841
url            --      0      /7896372
recording      --      0      /25081689

(This is the result the thread below. I don’t want to delete my indexes after spending evenings getting it working)

Then I noticed this old thread where someone had a similar issue.

MusicBrainz server troubleshooting help

Hello – I have a local musicbrainz server instance setup via the main docker container and I’m trying to query the server with Beets (file renaming and metadata mgmt tool). The idea is to be able to perform more than 1 query per second with a local instance https://beets.readthedocs.io/en/latest/reference/config.html#musicbrainz-options The trouble I’m having is that “search by index” does not return any results … this is the case for both web browser (localhost:5000) and Beet software. I can…

Here the author increased the available memory from 4GB to 16GB. This resulted in some indexes not being searchable yet. In my case, I increased the available memory to 24GB both the Database and SOLR_HEAP. This was achieved by configuring the following file:

# <project-root>/local/compose/memory-settings.yml

services:
  db:
    command: postgres -c "shared_buffers=24GB" -c "shared_preload_libraries=pg_amqp.so"
  search:
    environment:
      - SOLR_HEAP=24g

Now, I doubt that you need to allocate 24GB to the database or that you really need 24GB available on your system. I had only 8GB of ram available to docker at the time and it appeared to still work just fine. But, your milage may vary.

These changes, finally resulted in the results you can see below.

╰─admin/check-search-indexes all
CORE           STATUS  INDEX     DB
editor         OK      0         /0
instrument     OK      1048      /1048
series         --      25582     /25591
place          --      68269     /68305
event          --      90885     /90908
area           --      119472    /119488
tag            --      205747    /218960
cdstub         OK      276489    /276489
label          --      288988    /289174
annotation     --      844665    /756768
work           --      2225360   /2225920
artist         --      2488825   /2490078
release-group  --      3462993   /3463906
release        --      4443907   /4445101
url            --      13138544  /13145485
recording      --      33688919  /33697045

In this other thread they did state that you probably don’t need STATUS OK for the Index to be functional. Just as long as the Index is not 0.

@InvisibleMan78

Since I use this docker version and download the dumps (data & indexes) I have never seen that all cores report the status “OK”. There are always some (minor) differences .

BUT:
As long as you see a zero (0) at the index columns, you don’t have a single entry for this index.
In your example you have no index for series, for tags, for works, for release-groups, for releases and for urls.

I hope this helps!