GSOC 2024: Add cover art image processing options to Picard

Contact information

Nickname: twodoorcoupe
IRC nick: twodoorcoupe
Email: [redacted]
Github: twodoorcoupe (Giorgio Fontanive) · GitHub
Timezone: UTC +2 (Central european summer time)

Proposed Project

Picard should allow to post process and filter cover art images from the providers.

  • The processing options should include resizing images if above a certain size and converting to other formats.
  • The filtering options should include discarding images if too small.
  • The options should be specific to providers and to whether the image is saved to tags or saved to a file.
  • Plugin hooks should be provided for additional processing and filtering.
  • The project should clearly separate the image processing code, so that another library can be used instead if more options are needed later on.

Project length: 350h.

Image processing options

These allow the user to modify the images downloaded from providers with processing functions.

These functions are run at different times depending on whether the images are saved to tags, a file, or both:

  • After downloading the image. (Inside set_data of CoverArtImage)
  • Before saving an image to tags. (Inside a new method of TagCoverArtImage)
  • Before saving an image to a file. (Inside save of CoverArtImage)

Qt already provides all the functionality required for the processing options with the QImage class.

  • scaled(QSize) to resize images.
  • save(QIODevice, format) to convert to other formats.
  • copy(int, int, int, int) to crop images.

The processing UI options could be added to each provider’s options page or put in a separate page of their own. This is what they could look like:

processing

Image filtering options

These allow the user to discard images downloaded based on a filter function.

These functions should be started inside _coverart_downloaded in the CoverArt class. When an image is discarded, it is not saved and next_in_queue() in CoverArt is called.

The QImage’s size() function can be used to create the filter to discard images that are too small.

The filtering options UI should be handled just like the processing options. This is what they could look like:

filtering

Performance

A separate thread pool for image processing should be used.
Each cover art should have queues for filtering and processing tasks, one for each of the places where these can take place. The download queue is used for processing tasks that run for both destinations, thus saving computation.

queues_diagram

The information of what image filtering and processing tasks to queue is specific to each provider.
The tasks in the filtering and download queue are executed first, then the two other queues are processed concurrently.

Plugins

There should be 4 plugin hooks, one for filtering and one for each of the 3 places where processing can happen.
Also, plugins should be able to specify for which providers the tasks are executed, and skip the others.
Tasks created by plugins will be added to the specific queue with the given priority, reusing the PluginPriority system.
This is an example of what the queues for a particular provider could look like with plugins:

example_queues

Timeline

Community bonding period: consider what modifications are necessary to the current cover art and providers code.

Weeks Goal
1 Add the image filtering function with Qt.
2 Add the resize image processing function with Qt.
3 Add the convert format image processing function with Qt.
4 Add the new options to cover art providers code and UI.
5 Add the filtering and download queues.
6 Add the save to file and tags queues.
7, 8 Connect the providers options with the queues.
9 Move image processing to a new thread pool.
10 Improve performance if necessary and polish both code and tests.
11, 12 Add the post processing plugin hooks.
13, 14, 15, 16 These weeks are left as a buffer in case I fall behind or anything goes wrong, since I’ll be free also during this time.

Other Information

Tell us about the computer(s) you have available for working on your SoC project!

I have a laptop with both Linux and Windows, and a desktop with Windows I can use for backup. Unfortunately I don’t have access to macOS.

When did you first start programming?

I started programming in middle school and continued through high school up to now, where I’m studying Computer Science in university.

What type of music do you listen to?

Favourite band:

Some of my favourite albums:

What aspects of the project you’re applying for (e.g., MusicBrainz, AcousticBrainz, etc.) interest you the most?

I’ve been using Picard for the last few months, what impressed me the most is how simple it is to use while still allowing tons of customization.

What aspects of the project you’re applying for (e.g., MusicBrainz, AcousticBrainz, etc.) interest you the most?

The part I’ve been enjoying the most is how easy it is to extend with plugins.

Have you contributed to other Open Source projects? If so, which projects and can we see some of your code?

This is my first time contributing to open source projects, except for the contributions I’ve made so far to Picard.

What sorts of programming projects have you done on your own time?

In my own time, except for school and university projects, I’ve made stuff like small games, random bots (like for telegram) and participated in a couple coding competitions with friends.

How much time do you have available, and how would you plan to use it?

In June I’ll still have a few uni exams left, so I plan to spend around 20h a week.
But in July, August, and the first half of September I’m on break and can do full time 40h.

3 Likes

Thanks a lot for the proposal. This is looking really good. I have some comments below, but not much to change really :slight_smile:

What I like here is that you already grasped how we want to include the functionality into the existing cover art processing and propose concrete ways to do so.

This seems to be very flexible and cover the needs.

Very good. Definitely the way to go first. You could mention that if it should turn out to be not sufficient in some way (e.g. having quality or performance issues) alternatives like pillow could be used.

I would not make this provider specific. But the resizing should definitely be target specific (file system or tags).

In generally I agree. For some providers it could be possible to skip the entire download, though, if there is metadata about available sizes (CAA) or the actual size is known in advance (fanart.tv).

Maybe there can be a generic implementation, which always checks and discards after download. That would work for any provider independent of implementation. But also have a way for a provider to be asked to check the size in advance.

3 Likes