Hi Everyone,
I seem to be facing an issue with setting up the musicbrainz-docker repository locally, specifically with the database dump download and creating the database.
The issue occurs when running (I am working in sample mode)
docker-compose run --rm musicbrainz recreatedb.sh -sample -fetch
and variants of of it including sudo docker-compose...
and
docker-compose run --rm musicbrainz createdb.sh -sample -fetch
The common error is the line:
Cannot allocate memory at /musicbrainz-server/perl_modules/lib/perl5/DateTime/Locale/Data.pm line 8146.
For example running:
sudo docker-compose run --rm musicbrainz recreatedb.sh -sample -fetch
results in:
found existing dumps
2024/07/26 15:35:08 Waiting for: tcp://db:5432
2024/07/26 15:35:08 Connected to tcp://db:5432
2024/07/26 15:35:08 Command finished successfully.
--> Working on .
Configuring /musicbrainz-server ... OK
<== Installed dependencies for .. Finishing.
Catalyst::Plugin::StackTrace is up to date. (0.12)
File::Slurp is up to date. (9999.32)
Plack::Handler::Starlet is up to date. (undef)
Plack::Middleware::Debug::Base is up to date. (0.18)
Server::Starter is up to date. (0.35)
Term::Size::Any is up to date. (0.002)
Cannot allocate memory at /musicbrainz-server/perl_modules/lib/perl5/DateTime/Locale/Data.pm line 8146.
Compilation failed in require at /musicbrainz-server/admin/InitDb.pl line 11.
BEGIN failed--compilation aborted at /musicbrainz-server/admin/InitDb.pl line 11.
And running:
admin/configure add local/compose/memory-settings.yml
docker-compose up -d
sudo docker-compose run --rm musicbrainz recreatedb.sh -sample -fetch
results in Failed to import dataset.
:
Fri Jul 26 15:00:16 2024 : InitDb.pl starting
Fri Jul 26 15:00:16 2024 : Creating database 'musicbrainz_db'
CREATE SCHEMA
CREATE SCHEMA
CREATE SCHEMA
CREATE SCHEMA
CREATE SCHEMA
CREATE SCHEMA
CREATE SCHEMA
CREATE SCHEMA
CREATE SCHEMA
CREATE SCHEMA
Fri Jul 26 15:00:22 2024 : Installing extensions (Extensions.sql)
Fri Jul 26 15:00:23 2024 : Creating collations ... (CreateCollations.sql)
Fri Jul 26 15:00:23 2024 : Creating types ... (CreateTypes.sql)
Fri Jul 26 15:00:24 2024 : Creating tables ... (CreateTables.sql)
Fri Jul 26 15:00:27 2024 : Creating CAA tables ... (caa/CreateTables.sql)
Fri Jul 26 15:00:28 2024 : Creating EAA tables ... (eaa/CreateTables.sql)
Fri Jul 26 15:00:29 2024 : Creating documentation tables ... (documentation/CreateTables.sql)
Fri Jul 26 15:00:31 2024 : Creating json_dump tables ... (json_dump/CreateTables.sql)
Fri Jul 26 15:00:32 2024 : Creating report tables ... (report/CreateTables.sql)
Fri Jul 26 15:00:32 2024 : Creating sitemaps tables ... (sitemaps/CreateTables.sql)
Fri Jul 26 15:00:33 2024 : Creating statistics tables ... (statistics/CreateTables.sql)
Fri Jul 26 15:00:34 2024 : Creating wikidocs tables ... (wikidocs/CreateTables.sql)
Cannot allocate memory at /musicbrainz-server/perl_modules/lib/perl5/DateTime/Locale/Data.pm line 8146.
Compilation failed in require at /musicbrainz-server/admin/MBImport.pl line 14.
BEGIN failed--compilation aborted at /musicbrainz-server/admin/MBImport.pl line 14.
Failed to import dataset.
Fri Jul 26 15:00:40 2024 : InitDb.pl failed
with the same error line in common as the first failure:
Cannot allocate memory at /musicbrainz-server/perl_modules/lib/perl5/DateTime/Locale/Data.pm line 8146.
My setup is as follows:
I followed https://community.metabrainz.org/t/how-difficult-is-it-to-setup-a-local-musicbrainz-server/459658/21, also running the extra command:
admin/configure add musicbrainz-standalone
.
I also set up a local/compose/memory-settings.yml
file as suggested here
and have uncommented the MAINTENANCE lines in the DBDefs.pm
files.
I have also adjusted docker-compose.yml:
- Increased database memory:
shm_size: "8GB"
- Database shared buffers size increase:
command: postgres -c "shared_buffers=8192MB" -c "shared_preload_libraries=pg_amqp.so"
- musicbrainz, added a memory limit:
mem_limit: 8g
- Changed the
docker-compose.yml
version:version: '2.4'
I have also made sure my docker daemon has sufficient resources with my memory set to 14GB and swap to 4GB.
Any recommendations would be very welcome, or if I need to create an issue on the musicbrainz-docker github let me know.
Versions
echo MusicBrainz Docker: `git describe --always --broken --dirty --tags`
>>> MusicBrainz Docker: v-2024-07-22.0-dirty
echo Docker Compose: `docker-compose version --short`
>>> Docker Compose: 2.21.0-desktop.1
sudo docker version -f 'Docker Client/Server: {{.Client.Version}}/{{.Server.Version}}'
>>> Docker Client/Server: 24.0.6/24.0.6