GSoC 2018: Redefine Picard Plugin Structure

Tags: #<Tag:0x00007f050c78d6c0> #<Tag:0x00007f050c78d490>


Note: This idea is an extension of the idea here and discussion on IRC here.

Personal information

Name: Vishal Choudhary
IRC Nick: vishichoudhary1
Github: vishichoudhary
Portfolio. vishi choudhary
Time Zone. UTC+05:30



Presently, Picard uses Picard-Website to download plugins and fetch information of plugins. Picard-Website further use sources like Picard-plugins Github repository for plugins.

Following diagram illustrates how Picard will be able to use a developer plugin:


Now, in this procedure, a developer has to wait until his/her plugin is added to the website. Due to some constraints like a single repository and lack of reviewers the development in Picard plugin department could be slow.

The project proposed here involves addition of code that will restructure the Picard plugin handler. An overview of which I have described below:


This allows the users to add the plugins from anywhere in Picard. So a developer need not depend upon Picard-developer community to distribute his/her plugin. A developer can simply share the link of a plugin to the users.


With this project, What I propose to do can be described in following subtasks:

1. Picard: Change the Picard existing plugin handler code to adapt the new plugin structure. It include support i18n for plugins, a base class which plugins need to inherit.

2. Picard-Website: Update Picard Website to hold plugins link. It will includes removing existing code to make zip files and an addition of some code to provide details of plugins and their locations.

3. Picard-Plugin-tool: Refurbish Picard-Plugin-tool for new plugin structure. It comprises of creating the directory structure, managing manifest file, generating .pot file and packaging plugin as a zip file.

4. Plugins: Port the existing plugins to fit for new plugin structure.

Implementation details and code organization

Structure of Plugins:

  ├── manifest.json
  ├── po
  │   ├── fr.po
  │   └── PluginA.pot
  └── src

src/ directory will contain all the source code files of a plugin.

po/ directory will have all the translation files of a plugin.

Name of the plugin is similar to the name of the parent directory PluginA (in this case)

Currently, manifest for plugins are extracted from the source code of plugins. This adds a lot more complexity in the code.

Providing a manifest file will remove this extra complexity and it will be easy to get the information of plugins

The proposed manifest file of a plugin is shown below:

    "PLUGIN_NAME": "wikidata-genre",
    "PLUGIN_AUTHOR": "Daniel Sobey, Sambhav Kothari",
    "PLUGIN_VERSION": "1.0",
    "PLUGIN_DESCRIPTION": "query wikidata to get genre tags",

PluginBasic class:

Moving to a basic class for all plugins will be a great move.

It will provide some few generic functions, each plugin needs to inherit it. And implement some methods like activate() and deactivate().

All the available hooks will be provided by this class. So plugins need not call them directly.

It will also serve as a documentation of all the hooks available in Picard.

Below is the rough idea of that class:

  from picard.plugin import PluginFunctions

  album_metadata_processors = PluginFunctions()
  track_metadata_processors = PluginFunctions()
  All other processors

  class PluginBasic:

      def __init__(self):

      def activate(self):

      def deactivate(self):

      def register_track_metadata(self, args):

      def register_album_metadata(self, args):

      all other methods available

Installation of plugin and i18n:

Plugins will be downloaded from sources configured by the user.

A Plugin will be downloaded as a zip file but will be configured in plugin directory as of structure shown above.

For i18n, each plugin has to override its _() and N_() methods. Each plugin has to care about it’s own translation.
Picard will use its own domain and plugins will use their own.
To prevent naming conflicts we can use the names of plugins as their domains.


From where, user can add links for plugins?

Along with providing the facility to download plugins from anywhere, it’s our prime motive to
provide the user a simple and straight-forward interface. So users will be provided with an option in Picard
to add, edit and remove the link. Below is the screenshot illustrating the same :


Port from imp module to importlib

Picard uses python imp module to load plugins which have been deprecated from python3.4. So it will require to port from imp module to importlib module.

Presently used module and their replacement is following:

       imp.find_module will be replaced by importlib.util.find_spec official docs

      imp.load_module will be replaced by importlib.import_module official docs

Updation of Picard-Website:

Update Picard-Website such that it will hold the links of plugins and shows the information of
plugins. Information of plugins will be extracted from their manifest files.

Documentation of plugin development will also be provided in the Picard-Website. This will
include providing information about:

      Metadata Processor

      Context Menu Actions

      File Formats

      Tagger Script Functions

A sample plugin development tutorial will also be provided in the Picard-Website.

Inspiration source. OpenStack

For user convenience, documentation about every Picard tool should be made available in one place. So documentation about Picard-Plugin-tool will also be included in Picard-Website.

Picard-Plugin-tool Refurbishment:

Providing the plugin developers with the tool to manage their plugins will help them and will attract
developers to develop more plugins. So, it is very important to provide a powerful tool. Picard-
Plugin-tool is providing the same functionalities but it needs to change a little bit. It includes
providing the following options:

1. Create Directory Structure:
It will create a defined directory structure for a plugin.

  ppt --mkdir <name of plugin>

2. Package as zip:
Plugins will be distributed as zip files. So this will help users to zip their plugin.

  ppt --zip <path of plugin>

3. Generate .pot file:
To provide i18n support. Plugins need to have thier own .pot files. This will generate .pot file
from .py files.

  ppt --genpot <path of src/ of plugin>

Port existing plugins:

Presently Picard has more than 30 plugins, which needs to be port in the new plugin structure.

Sample of ported plugin can be found on padded_disc_and_tracknumbers


Community Bonding (April 24 – May 13)

Spend this time with community to find out what exactly I have to code. Also, discuss design
decisions with the mentor.

Phase 1 (May 14 – June 11)

Porting Picard existing code to install/update plugins from links. Links will be fetched from sources
configured by the user.

Apart from this, port the code from imp module to importlib module as imp module happens
to be deprecated in python3.official docs

Implement a base class, Which will define the basic interface and provide some functions for all plugins.
Each plugin should inherit it and implement some methods. This class will also hold all available hooks. And will also serve as documentation.

Phase 2 (June 12 – July 9)

I aim to do Picard-Website work in this phase. This will involve removing the existing code of creating
zip files from Picard plugin repo. And addition of some code to make it hold plugins link and displaying
plugins information. This phase would also involve the addition of some code in picard-plugin-tool
to create directory structure, managing manifest, and packaging plugin.

Phase 3 (July 10 – August 6)

This phase would involve porting existing Picard plugins to new plugin-structure, clean up of the
code written in earlier phases, bug fixes and add some new tests to make sure that everything works
as it is expected to be.

After GSoC:

Continue working on Picard as I have been since the last September. I will try to enhance the
concurrency code of Picard if it hasn’t been done as a part of GSoC 2018.

A detailed timeline of my work is as follows:

• Week 1: Begin with setting up the environment. Defining generic class.

• Week 2: Update Picard plugin handler code.

• Week 3: Finish work on plugin installation and write tests.

• Week 4: Solve bugs if there’s any else prepare for Phase I evaluation. PHASE I Evaluation here

• Week 5: Taking mentor evaluation into account, fix stuff in code written so far and continue coding.

• Week 6: Begin with docker setup for Picard-Website

• Week 7: Update Picad-Website

• Week 8: Update Picard-plugin-tool and complete documentation. PHASE 2 evaluation here

• Week 9: Start porting plugins

• Week 10: Complete any pending stuff.

• Week 11: Test if the plugins are working the same as they were before porting.

• Week 12: Complete previous pending task, if any.Solve bugs

• Week 13: Buffer week. Work on final submission.

Detailed Information about yourself

I am a junior CS undergrad at the National Institute of Technology, Hamirpur. I came across Picard
when looking for a music tagger last August. As I was on Arch Linux, I got a Picard dev version and
found out a bug in the search bar. I fixed that issue and have been helping out in development since then.
Here is a list of pull requests I’ve worked on over time. I have a semi-active blog which i would like
to update regularly over the Summer of Code period about my progress with the project.

Question: Tell us about the computer(s) you have available for working on your SoC project?

Answer: Currently I have an Acer laptop with Amd A8 processor and 8 GB RAM, running Manjaro

Question: When did you first start programming?

Answer: I have been programming since 10th grade, mostly html/css and was introduced to C
programming 5 years ago. I picked up Python in my freshman year, working on small applications.

Question: What type of music do you listen to?

Answer: I listen mostly pop and folk songs. My Favourite artists and songs are:
Adele, Ed sheeran, Gurdas Maan,
ColdPlay, Arijit Singh. Presently top listen are: Perfect, Something just like this.

Question: Have you ever used MusicBrainz to tag your files?

Answer: Yes I use Picard to tag my files.

Question: Have you contributed to other Open Source projects? If so, which projects and can we see some of your code?

Answer: I have contributed to the source code of Picard, and various other projects which can be
found on my GH.

Question: What sorts of programming projects have you done on your own time?

Answer: I have worked on Ball and Bridge game. And a video chat app. And various other college
level projects. Please check my GH and my portfolio.

Question: How much time do you have available, and how would you plan to use it?

Answer: I have holidays during most of the coding period and can work full time (45-50 hrs per
week) on the project.

Question: Do you plan to have a job or study during the summer in conjunction with Summer of Code?

Answer: None, if selected for GSoC.


This is an initial draft of my application. Any reviews/feedback would be greatly appreciated!


Basically what i proposed on IRC, so i agree. I wish there were few original ideas added though.

About URLs to plugins, it is important to note that they should point, either directly to a manifest file (and the manifest file points to the zip archive, and contains a checksum for it the app can validate), or a list of manifest files (for multiple plugins contained in one repo).
The app (Picard or Picard Website to start with) shouldn’t have to “search” for manifest files, nor for plugin archives.

Basically the process i have in mind is the following:

  1. Iterate through list of known “sources”
  2. If source is a list, iterate through the list
  3. for each manifest file, check for availability, retrieve it, parse & validate (mandatory fields), if not valid, ignore
  4. for the listed archive, if not available ignore
  5. (optional) download the archive, and validate checksum against manifest, if not valid, ignore
  6. (optional) install etc…

Picard website will only consider manifests (and eventually retrieve archives to validate checksums on manifest changes), web hooks could be set up to manage status (update/delete), and/or a scheduled job. The idea is to keep the website list of plugins up-to-date, and provide tools for plugin devs to propagate their status. Please dig this, that’s just ideas.

The draft is good enough to me, but i expect more on final proposal.


I agree with what @zas pointed out. But I am really not quite sure about the originality of this proposal. It seems an almost ditto copy of the ideas proposed by @zas on IRC.

Anyway, a couple of points to be noted -
The usage of ppt is not exactly correct. You want sub-commands not flags. Also --mkdir and --zip as proposed by you is already implemented in ppt , however that will have to be changed to adapt to the proposed dir structure.

I would like to see a more fleshed out implementation of the Plugin classes and the API you have in mind. The current one seems very abstract.

I would also like some thought put into how to handle incompatible plugins/crashes, the likes of which we experience while porting from 1.0 to 2.0.

You should definitely put some more thought into the plugin hooks, so that plugin authors can make use of a consistent and flexible API rather than monkey-patching Picard code. And build your proposal keeping that in mind.


As a Picard plugin (co‐)author, I don’t know how I would register a plugin or plugin repository with/on Picard’s website with this new system. This seems like a fairly central thing to include in the proposal too, maybe even with some mockups of how you would propose this workflow to be.


The process is yet to be defined, but for sure it has to be very simple. Form, captcha, MeB team validation, published.


Hello @zas, @Freso.

I made a simple system for this.
In Picard-website we will provide an option to add the plugin. (But author needs to login with musicbrainz account)

In add plugin, there will be two inputs
1) Add manifest file or link to manifest file
2) Add the location of zip.

check if the manifest file and zip file are correct if they are, add them else pop a message.

There will be other options:
1) upvote (like)
2) downvote (dislike)

If a plugin cross a threshold upvotes it will be automatically added to a trusted source, we can add administration verification here but it will become an overhead.

Similarly, if a plugin sees too many downvotes it will be deleted automatically.


In fact, i’d prefer the location of a manifest, and each manifest points to a zip. We don’t need to download the zip archive, and extract it.
Infos are provided by the JSON manifest (including checksums for the archive). So the only thing needed to register a plugin is the link to the manifest, it is up to plugin dev to ensure the manifest contains valid information.

I’m not sure it will work, who are voters ??


@Zas, It looks good to me too. But what I think while developing the plugin we don’t know where we are going to upload our plugin. So we need to modify our manifest file later to add location and checksum.
One more thing, if my plugin is added successfully in the trusted plugin. And if I update my plugin later on, there isn’t any check to ensure that I am following the right plugin structure protocols or not.

But with the system I describe we can verify this at each stage.

Regarding We don’t need to download the zip archive, and extract it. We are not storing plugin in the website. This is to just check that the plugin zip is exactly as describe in the manfiest file. This is temporary download just to make sure that plugin structure is as per rules.

Regarding who are voters ??
Anyone who uses picard website and have a MusicBrainz account. But we can skip this step and simply say Waiting for approval where the MeB team can simply approve it.

In this way we can ensure, plugin(manifest, link) available in picard-website are as per the new picard plugin structure.