Today, my Ubuntu in a virtual machine for the MusicBrainz docker asked me, if I want to upgrade to LTS 24.04.
I did it and it ended in a corrupt, unbootable Ubuntu.
This is not to blame anyone, just to inform you that if you need to have your Docker container up and running again very quickly. you should carefully think about an inplace-upgrade to LTS 24.04.
I see a large number of complaints with Ubuntu 24.04, some of which are not recent, but I donât know how that compares to the average Ubuntu release:
https://www.google.com/search?q=ubuntu+24.04+corrupt
I start downloading the newest 24.04.1 LTS release and rebuild the base for the MB docker now.
We will see how this works in a few hours 
WOW, what a mess. I canât even finish the basic installation routine in LTS 24.04.1.
After the first reboot attempt I only get a blank screen, no error, no information nothing.
That is kinda mad. I thought the point of LTS was this is the reliable bug free one they will support for many years. I guess they are following Microsoft Standard âmake the users test itâ training courses. 
Microsoftâs tests taught me a lot.
Good point @Deleted_Editor_2513197. This is clearly Unbuntu giving people training in disaster recovery. These are skills we all need. One only truly learns when everything goes wrong.
It is also the lesson of ânever install the first release of anythingâ. I always teach my clients to put off updates for a week to let everyone else test them firstâŚ
Iâm back at ubuntu-22.04.4-desktop-amd64.iso and Jammy Jellyfish.
Installation was quick and easy (without all the questions about âWhat do you want to install?â).
Now itâs just copy & paste from the great instructions found here.
The MB docker runs fine again with the above mentioned ubuntu 22.04.4.
I have no idea what they did with the LTS 24.04 that I canât boot it in a VM anymore.
Now there is just one small annoyance:
I believe I have found a possible reason for my reboot (and display) problems:
Thanks to this answer, I have deactivated the â3D graphics â Accelerate 3D graphicsâ option in the Hardware â Display settings in my VMware v17.5.2 on Windows.
Since then about 10 reboots worked fine and no more boot crashes or black screens appeared.
And if you ask why you canât copy & paste the commands from the above mentioned step-by-step instructions anymore into your terminal window in Ubuntu:
This seems to be another quirk in LTS 24.04.
Thanks to this answer, you have to type in the following command manually into your terminal windows as one of the very first commands:
sudo apt install open-vm-tools-* -y
and press Enter
Then you need to reboot your Ubuntu in the VM.
After that, you can start copy & pasting the needed commands as usual from your host system to your Ubuntu in an VM.
Maybe this helps others too.
Unfortunately, the next problem occurs with LTS 24.04 during the install of the MB docker commands:
$ git clone https://github.com/metabrainz/musicbrainz-docker.git && \
cd musicbrainz-docker && \
sudo docker-compose build
...
Traceback (most recent call last):
File "/usr/bin/docker-compose", line 33, in <module>
sys.exit(load_entry_point('docker-compose==1.29.2', 'console_scripts', 'docker-compose')())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/bin/docker-compose", line 25, in importlib_load_entry_point
return next(matches).load()
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/importlib/metadata/__init__.py", line 205, in load
module = import_module(match.group('module'))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 995, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/usr/lib/python3/dist-packages/compose/cli/main.py", line 9, in <module>
from distutils.spawn import find_executable
ModuleNotFoundError: No module named 'distutils'
It seems that the newest LTS 24.04 is not providing this needed module:
ModuleNotFoundError: No module named âdistutilsâ
My attempt with
sudo apt-get install python3-distutils
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Package python3-distutils is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package 'python3-distutils' has no installation candidate
didnât helpâŚ
So with a lot of help from Google, I found this command to search for a package:
apt-cache search distutils
and it returns this list:
...
python3-sip-dev - Python 3/C++ bindings generator development files
python-pkginfo-doc - Python module to query metadata from packages documentation
python3-bdist-nsi - Create NSIS windows installers for Python modules (Python 3)
python3-distlib - low-level components of python distutils2/packaging
python3-distutils-extra - enhancements to the Python3 build system
python3-looseversion - Version numbering for anarchists and software realists
python3-pkginfo - Python 3.x module to query metadata from packages
python3-pyqt-distutils - distutils extension to work with PyQt applications and UI files (Python3)
python3-stdeb - Python to Debian source package conversion plugins for distutils
python3-versioneer - determine version from repository tag
I tried it with
sudo apt-get install python3-distutils-extra
and the above ModuleError disappeared.
And have a guess - of course the next hurdle comes up during the sudo docker-compose build command
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 497, in _make_request
conn.request(
TypeError: HTTPConnection.request() got an unexpected keyword argument 'chunked'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
...
File "/usr/lib/python3/dist-packages/docker/api/client.py", line 197, in __init__
self._version = self._retrieve_server_version()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/docker/api/client.py", line 221, in _retrieve_server_version
raise DockerException(
docker.errors.DockerException: Error while fetching server API version:
HTTPConnection.request() got an unexpected keyword argument 'chunked'
Iâm out of ideas now and have to wait for docker-specialists or @yvanzo who could please have a look at the current situation with Ubuntu LTS 24.04 and the MB server docker.
That reads like a version incompatibility issue for the python urllib module.
Eithe the mb scripts want to use a âchunkedâ option that no longer exists, or the installed urllib is too old to provide it.
Does sudo apt upgrade show any upgradable packages? If so, install those first.
Do the mb scripts have any bootstrapping? If so, that would probably need to be reapplied.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following upgrades have been deferred due to phasing:
ubuntu-pro-client ubuntu-pro-client-l10n
0 to upgrade, 0 to newly install, 0 to remove and 2 not to upgrade.
Unfortunately, I have no idea how to check that.
Could you please tell me the necessery steps?
The current LTS version:
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 24.04.1 LTS
Release: 24.04
Codename: noble
If you had installed docker from the docker apt repositories (and not from the Ubuntu repositories) it is well possible the repository is disabled. On upgrade Ubuntu will disable third-party repositories.
That means you will then have an older docker version installed that might not be fully compatible. In this case enable the docker apt repository again. Check /etc/apt/sources.list or the files in /etc/apt/sources.list.d/.
Thank you for your help @outsidecontext
As I have very little knowledge about Ubuntu, I can only copy & paste these results:
cat /etc/apt/sources.list
# Ubuntu sources have moved to /etc/apt/sources.list.d/ubuntu.sources
cat /etc/apt/sources.list.d/ubuntu.sources
Types: deb
URIs: http://ch.archive.ubuntu.com/ubuntu/
Suites: noble noble-updates noble-backports
Components: main restricted universe multiverse
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
Types: deb
URIs: http://security.ubuntu.com/ubuntu/
Suites: noble-security
Components: main restricted universe multiverse
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
ls /etc/apt/sources.list.d/
ubuntu.sources ubuntu.sources.curtin.orig
How can I do that?
Iâm not sure right now if it is docker or docker-compose which is at fault.
Could you check what apt show docker.io shows? If should show from which repositry it was installed.
For docker-compose it is likely that you followed some documentation that installed it without package management. Older descriptions on how to install this usually just downloaded it to /usr/local/bin/docker-compose. Can you check what which docker-compose shows? It should indicate the path to the executable. If thatâs an older version located in /usr/local I would actually recommend removing that and instead install docker-compose from Ubuntu repo:
apt install docker-compose
Note: There is also the âdocker-compose-v2â package, which provides the docker compose plugin for the docker command. The old âdocker-composeâ is deprecated. But if you had running the containers with the old âdocker-composeâ I would first stick to this. It still works well. Migrating to the new command is possible, but could raise its own issues.
package: docker.io
Version: 24.0.7-0ubuntu4.1
Built-Using: glibc (= 2.39-0ubuntu8.2), golang-1.22 (= 1.22.2-2ubuntu0.1)
Priority: optional
Section: universe/admin
Source: docker.io-app
Origin: Ubuntu
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 109 MB
Depends: adduser, containerd (>= 1.2.6-0ubuntu1~), iptables, debconf (>= 0.5) | debconf-2.0, libc6 (>= 2.34), libdevmapper1.02.1 (>= 2:1.02.97), libsystemd0 (>= 209~)
Recommends: ca-certificates, git, pigz, ubuntu-fan, xz-utils, apparmor
Suggests: aufs-tools, btrfs-progs, cgroupfs-mount | cgroup-lite, debootstrap, docker-buildx, docker-compose-v2, docker-doc, rinse, zfs-fuse | zfsutils
Homepage: https://mobyproject.org/
Download-Size: 29,1 MB
APT-Manual-Installed: yes
APT-Sources: http://ch.archive.ubuntu.com/ubuntu noble-updates/universe amd64 Packages
Description: Linux container runtime
Docker complements kernel namespacing with a high-level API which operates at
the process level. It runs unix processes with strong guarantees of isolation
and repeatability across servers.
.
Docker is a great building block for automating distributed systems:
large-scale web deployments, database clusters, continuous deployment systems,
private PaaS, service-oriented architectures, etc.
N: There is 1 additional record. Please use the '-a' switch to see it
/usr/bin/docker-compose
What should I do next?
(Just to be clear: This is a brand-new vanilla installed LTS 24.04.1 âDesktopâ in VM)
NOTE that Iâm a Docker user but not an expert! I also donât have experience with the MB image.
The error âHTTPConnection.request() got an unexpected keyword argument âchunkedââ does come up multiple times in online discussions related to recent use of the outdated Docker compose v1. See Bug #2056735 âdocker.errors.DockerException: Error while fetchin... : Bugs : docker-compose package : Ubuntu
One fix I repeatedly see in discussions related to that error is to switch to Docker compose v2 (the Ubuntu package âdocker-compose-v2â that @outsidecontext mentioned; the package âdocker-composeâ contains the no longer updated compose version 1) and use the compose plugin instead of the old binary. This would mean replacing the command docker-compose <...> with docker compose <...> (note the hyphenâs been replaced with a space).
I donât know the finer details of switching from compose v1 to v2 or if anything might break in the process, though, so Iâm not recommending blindly switching to compose v2. Maybe someone else has experience running the MB Docker image on Ubuntu 24.04 with compose v2?
In any case, there are specific notes for migrating from Docker compose v1 to v2 that one should read before making the switch: Migrate to Compose V2 | Docker Docs
It might be at least worth trying. If itâs a VM anyway I would make a snapshot and try running with v2. As you wrote itâs essentially the same commands, only docker compose without a hyphen instead of docker-compose.
I have moved multiple projects to be and in the vast majority of cases things just worked without any loss. But I have no experience with that for the MB server setup, and in some rare cases I experienced things breaking when migrating to docker compose v2.
Oh, good point! I was so careful not to suggest breaking things that I ended up ignoring it was a VM.
To summarize the last couple of posts, @InvisibleMan78, you can try this:
docker-compose commands with docker composeHopefully thatâll help get the MB image running again.
Thanks for your great help @Nadim & @outsidecontext
With these steps I was able to setup the MB docker container in LTS 24.04.1 Desktop from scratch:
sudo apt install docker-compose-v2sudo apt install docker-compose-v2
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
docker-compose-v2
0 upgraded, 1 newly installed, 0 to remove and 8 not upgraded.
Need to get 12.7 MB of archives.
After this operation, 54.2 MB of additional disk space will be used.
Get:1 http://ch.archive.ubuntu.com/ubuntu noble/universe amd64 docker-compose-v2 amd64 2.24.6+ds1-0ubuntu2 [12.7 MB]
Fetched 12.7 MB in 0s (28.3 MB/s)
Selecting previously unselected package docker-compose-v2.
(Reading database ... 150317 files and directories currently installed.)
Preparing to unpack .../docker-compose-v2_2.24.6+ds1-0ubuntu2_amd64.deb ...
Unpacking docker-compose-v2 (2.24.6+ds1-0ubuntu2) ...
Setting up docker-compose-v2 (2.24.6+ds1-0ubuntu2) ...
VM-Test:~$ git clone https://github.com/metabrainz/musicbrainz-docker.git && \\
cd musicbrainz-docker && \\
sudo docker compose build
Cloning into 'musicbrainz-docker'...
remote: Enumerating objects: 3748, done.
remote: Counting objects: 100% (821/821), done.
remote: Compressing objects: 100% (307/307), done.
remote: Total 3748 (delta 587), reused 641 (delta 499), pack-reused 2927 (from 1)
Receiving objects: 100% (3748/3748), 45.63 MiB | 19.05 MiB/s, done.
Resolving deltas: 100% (2402/2402), done.
[+] Building 99.2s (32/32) FINISHED docker:default
...
=> => naming to docker.io/library/musicbrainz-docker-musicbrainz 0.0s
=> [indexer 4/4] WORKDIR /code 0.0s
=> [indexer] exporting to image 1.2s
=> => exporting layers 1.2s
=> => writing image sha256:e25faa636e4b45e34a944d1941fd3368328e1cf4e962aecdbb7a6e76842f8b3e 0.0s
=> => naming to docker.io/library/musicbrainz-docker-indexer 0.0s
docker-compose (with hypen between the words docker and compose) with the version docker compose (without hypen).This way, no more errors occurs during the setup and installation of the MB docker container.
Suggestion:
Maybe someone should adapt these steps on the original README accordingly for Ubuntu LTS 24.04.1 Desktop.