Search index not updated

The Indexed Search Syntax page states that search indexes are updated every 3 hours. However, when performing an arbitrary search, the result page states, “Last updated: 2016-12-22 16:42 UTC” meaning that the indexes haven’t been updated in three months. As a result, when using the API, I cannot retrieve songs that have been released since this date. Why haven’t the search indexes been updated, and when will the they be updated? Also, until then, how can I access the most recent entries without having to mirror the database locally?

1 Like

You mean the recording indexes, right? (the others do seem to be updated). I hadn’t noticed this, I’ve asked about it.

We’ve found the bug and the recording index is being updated right now. Thanks for noticing!

2 Likes

It seems like the bug is still occurring. When searching recordings, the results page states “Last updated: 2017-02-23 14:31 UTC.”

1 Like

Sigh. Thanks! Adding a ticket for this.

2 Likes

For (future?) reference, this is the new ticket:
https://tickets.metabrainz.org/browse/SEARCH-441

1 Like

The latest search index dump in data.metabrainz.org/pub/musicbrainz/data/search-indexes shows
latest-is-20220326-101302

The data dump index shows
latest-is-20220402-001958

Could you please start/restart the dump process for the search indexes too? Thanks!

@reosarevok & @Freso

Currently, we see this data dump:
data.metabrainz.org/pub/musicbrainz/data/fullexport/20220406-001957
but for the search index there are only dumps for
latest-is-20220326-101302

Could you please have a look at it?

@reosarevok & @Freso

The latest dumps are from
/search-indexes/20231104-041002
and
fullexport/20231104-001705
about 10 days ago.

Could you please have a look what process needs to be restarted/activated?
Thanks.

Seems those have both updated since, it’s just slow / a bit behind :slight_smile:

I think, I see the problem:

The mirror at “mirrors.dotsrc.org
is not “just slow/a bit behind”, it stopped receving updates after 2023.11.04: :wink:

image

The one at


is indeed just 6 days behind.

BTW:

On this webpage
https://metabrainz.org/datasets/download
the server at “Oregon USA” (more specifically the target directory) is not reachable anymore.
There seems to be no directory
/pub/musicbrainz/data/fullexport/
on this FTP-Server anymore.

We’re discontinuing the mirrors, so only the data.metabrainz one will continue, but we forgot to amend the datasets page. Will do that.

3 Likes

That’s a pity.
For me, the mirrors.dotsrc.org server was up to 5 times faster than the main one…

But thanks for the information.

The search dumps seems to be up to date now:
(20231111 and 20231115)

Do you renew the fullexport on the same server too?
(They still show the dates 20231104 and 20231108)

Both - search-indexes and fullexport - should share the same date, right?

Yes, those should share the same date since they run on the same days. However, the last two MB full exports (for the 11th and 15th) failed due to lack of disk space. I started another one just now. We’re working on moving these processes to another server.

Edit: It’s still failing, so we’ll need some time to prepare a new server tomorrow. Apologies for the wait.

2 Likes

Thank you for the newest dumps, both from
20231118


1 Like