# GSoC 2017: Picard v2.0
**URL:** https://community.metabrainz.org/t/gsoc-2017-picard-v2-0/227579
**Category:** GSoC applications
**Tags:** gsoc-2017, gsoc, picard
**Created:** [March 11, 2017, 8:51pm UTC](https://community.metabrainz.org/t/gsoc-2017-picard-v2-0/227579 "2017-03-11T20:51:56Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@samj1912](https://community.metabrainz.org/u/samj1912)
#### Post date: [March 11, 2017, 8:51pm UTC](https://community.metabrainz.org/t/gsoc-2017-picard-v2-0/227579/1 "2017-03-11T20:51:56Z")
## Personal information
* * *
**Name:** Sambhav Kothari
**Nickname:** samj1912
**IRC nick:** samj1912
**Email:** [sambhavs.email@gmail.com](mailto:sambhavs.email@gmail.com)
**GitHub:** [samj1912](https://github.com/samj1912)
**Blogger:** [Sam’s Blog](https://sambhavkothari.blogspot.com)
**LinkedIn:** [Sambhav Kothari](https://www.linkedin.com/in/sambhav-kothari-ba303aab/)
# Proposal
* * *
# Overview
Picard is going to be receiving a complete make-over code/feature/UI wise for v2.0. A part of the v2.0 plans is porting the existing code to PyQt5/Py3 to allow for future support and better encoding compatibility along with new features of Qt5(such as High DPI scaling, better widget and networking support and better cross-platform compatibility) and along with all the bug fixes that come with PyQt5 which will help clean up the code-base for Picard.
This GSoC proposal aims to do a bare port of the Picard to Py3 and Qt5 along with setting up the foundation for v2.0. The port will be based off stable v1.4.1 code.
# Deliverables
With this proposal I’d like to set the following parameters for the overall evaluation of my project:
1. A Qt5/Py3 port of the main Picard tagger with all of the current tests passing.
2. A working GUI port with all the basic functionality of Picard working which includes:
3. High DPI support
4. At Least 30% code coverage for specified modules where it is currently 0 and 50%+ for specified modules already having test coverage (specified modules are mentioned below)
5. Complete the Qt5/Py3 Port of the existing plugins
6. A working v2 API endpoint for PW that allows users to download specific versions of plugins instead of just the latest ones.
# Project plan and Implementation Details
The whole of project can be broken down into the following independent sub-parts:
1. Port Picard to Qt5
2. Port Picard to Py3
3. Source code cleanup and testing
4. Port all of the existing Picard plugins to Qt5 and Py3
5. Implementing a suitable end-point @ PW which allows plugins for both v2.0 and v1.4.x to be queried and updating the WS code in Picard to handle the same.
The above sub parts are listed in their planned order of execution during my GSoC period.
## Phase 1 (May 30 - June 30)
* * *
### Port to Qt5
#### Converting obsolete/moved Qt4 classes/method to their Qt5 equivalent
Porting the existing Picard code to Qt5 will involve going through the recommended checklist for PyQt4 to PyQt5 porting and covering each point. See [PyQt4-PyQt5 Differences](http://pyqt.sourceforge.net/Docs/PyQt5/pyqt4_differences.html)
Apart from this I have also plan on checking the code against the original Qt5 porting guide since PyQt one seems to be missing some of the changes(for eg. QUrl and QDrag) See [Qt 4.x to Qt5](https://wiki.qt.io/Transition_from_Qt_4.x_to_Qt5)
As a final thorough check, I plan on using [pychecker](http://pychecker.sourceforge.net/) and [Pylint](https://www.pylint.org) to verify any remaining obsolete members while porting to Qt5.
As for correcting the above errors, although the Qt porting guide linked above misses some finer details, the Qt documentation itself is very extensive about the [obsolete Qt4](http://doc.qt.io/qt-5/obsoleteclasses.html) members and their Qt5 equivalents.
The final conversion will involve manual checking, but the above plan will ensure a very thorough port to Qt5 and will help us weed out unported LoC which we might miss while doing the port manually.
#### High DPI support and uniform UI across options
Starting Qt5.4, Qt supports High DPI screens. However it was greatly reworked in Qt5.6. Thus I will be targeting Qt5.6 for my Qt port. See:
- [Retina display support for Mac OS](http://blog.qt.io/blog/2013/04/25/retina-display-support-for-mac-os-ios-and-x11/)
- [HighDPI in KDE Applications](http://blog.davidedmundson.co.uk/blog/kde_apps_high_dpi)
- [Qt5 Documentation: High DPI displays](http://doc.qt.io/qt-5/highdpi.html)
The process for support High DPI displays is also documented well. WRT Picard we only need to take care of 2 things, namely:
1. Supporting [high resolution pixmaps](http://doc.qt.io/qt-5/qpainter.html#drawing-high-resolution-versions-of-pixmaps-and-images)
2. Making sure that all of our widgets are using layouts and device independent pixels.
For High DPI Pixmaps
> To facilitate this, Qt as adopted the “@2x” convention for image filenames:
> `foo.png`
> `foo@2x.png`
> High-resolution content can be provided side-by-side with the originals. The “@2x” version will be loaded automatically when needed by the QML Image element and QIcon:
> Image { source = “foo.png” }
> QIcon icon(“foo.png”)
Another alternative for generating properly sized Pixmaps for high-dpi screens is to use SVG files and render them to size. This is achievable quite easily with Qt. See [Rendering SVG to QPixmap for High DPI screens](http://falsinsoft.blogspot.in/2016/04/qt-snippet-render-svg-to-qpixmap-for.html)
Although it has a [bug associated](https://bugreports.qt.io/browse/QTBUG-52444) with it, we can consider it as a possible alternative while implementing support for HiDPI screens.
To test the above on high res. displays Qt has a built-in env variable available which allows developers to check how Qt renders pixmaps.
To test the above on high res. displays Qt has a built-in env variable available which allows developers to check how Qt renders pixmaps.
> I only have a poor person’s screen, but it’s still easy(ish) to test:
>
> - Change the DPI under system settings → font → force font DPI to 192
> - Set the environment variable QT\_DEVICE\_PIXEL\_RATIO=2
> On a normal screen everything will appear massive, but hopefully also super smooth.
Please note that the env. variable has now been changed to `QT_SCALE_FACTOR`.
Proper layouts and device independent pixels are mostly ensured by Qt Designer which we use to create the barebones UI elements. However some manually coded Widgets in `picard/ui/*.py` need to be refactored and coded appropriately.
Eg- `ArtworkTable` Class in `infodialog.py`
### Port to Py3
#### Converting essential syntax differences
A code analysis for Picard source code shows that conversion to Py3 in terms of syntax changes should be a relatively easy task. Problems generally faced while 2to3 conversions WRT Picard will mostly be heavy lifted by PyQt5. See the table below for a detailed statistic of all the syntax changes needed.
Py2toPy3 conversion should be an easy task WRT Picard since most of the code ensures Py3 compatibility. The above mentioned errors are easy to resolve and there are multiple tools available to help with the conversion. See [python-modernize, pylint etc.](http://portingguide.readthedocs.io/en/latest/tools.html)
I plan to follow the [Py3 Porting Guide](http://portingguide.readthedocs.io/en/latest/process.html).
As for the dependencies themselves, Picard relies on [mutagen](https://caniusepython3.com/project/mutagen), [discid](https://caniusepython3.com/project/discid) and [PyQt5](https://caniusepython3.com/project/PyQt5). All of them support Py3, so there aren’t any requirements blocking a Py3 port.
A proof of concept port can be found at [here.](https://github.com/samj1912/picard/tree/py3test)
A WIP port can be found [here.](https://github.com/metabrainz/picard/pull/664)
A part of the porting process will be to decide which python version to target. Some discussion on it has already occurred but we didn’t reach a conclusion. The decision lies between Py3.4 and Py3.6.
I plan to resolve this during the community bonding period. However, since there aren’t any breaking syntactical difference between the 2 versions and with PyQt already providing a layer of abstraction, it shouldn’t impact much in the timeline or the implementation.
## Phase 2 (July 1 - July 28)
* * *
### Code cleanup
Apart from this, Picard has some hacks to get around some PyQt4 and Py2 issues.
Rel. - QNetwork bugs and improvements, NSURL bugs, shutil monkey patching.
Another major problem is the way the current tagger thread are structured. According to Qt recommendations, the main thread should only run the main GUI thread and any file I/O should be off loaded to a runner. This is not currently the case as file loading happens in the main thread which causes significant unresponsive behaviour.
A proper rewrite of the code for loading files is needed. A proof of concept commit can be found at [https://git.io/vyKc7](https://git.io/vyKc7)
Synthetic Analysis of the code shows that Picard code could use some refactoring. See:
[https://gist.github.com/samj1912/62845d3bd9f3a21de298aab33a5e2890](https://gist.github.com/samj1912/62845d3bd9f3a21de298aab33a5e2890)
The major refactoring will be limited to the modules listed in **Unit** **Testing** part of the project as described below, since it both coincides with the code analysis generated above and the major current area which needs testing.
Apart from this the current Picard directory structure also causes circular imports which might cause unpredictable behavior and needs to be corrected. The main problem in the directory structure lies in `picard.formats` and `picard.coverart.providers`. The above can be resolved by registering the providers and formats in their respective modules.
Thus the code clean up will mainly involve fixing the above issues and improving the overall code quality of Picard.
### Testing
An important way to control regression is to have test driven development(TDD).
#### Unit Testing
The code coverage for picard sits at 21%. This is below acceptable levels to catch regressions in the code. It is extremely important to have unit-testing, especially since Picard handles lots of data simultaneously and it is absolutely required that we prevent any data corruption, loss or let invalid data pass through.
You can see the detailed coverage report for picard at [https://image.ibb.co/jBAQYv/1.png](https://image.ibb.co/jBAQYv/1.png)
Some important modules that need to be tested:
- Network/API related modules:
- File and I/O related modules:
Unit testing for the network modules above will mainly require patching the get and post methods of picard.webservice using mock and returning saved local test data(basically saved MB response data) through them.
I have experience with the same and a proof of concept code implemented by me can be found at [https://git.io/vyKCv](https://git.io/vyKCv)
To facilitate unit testing for `picard.file` and `picard.tagger`, we need to refactor them to try and isolate file I/O methods from the GUI ones. This will also ensure a proper separation of the GUI and I/O code for Picard allowing for a more responsive UI. Thus the above refactoring and testing should serve to improve both development and user experience.
The above tests are required to ensure proper data while querying APIs, processing their response, loading, tagging, saving, renaming and moving files.
#### GUI Testing
So far Picard hasn’t taken advantage of the QtTest module for GUI testing. QtTest a comprehensive testing suite by Qt isn’t completely replicated by PyQt. PyQt just provides some basic GUI actions in form of [QTest](http://pyqt.sourceforge.net/Docs/PyQt4/qtest.html)) to facilitate testing which involves -
- Mouse Clicks
- Keyboard input
- Pointer movement
Taking the above into consideration, I propose we implement the following basic tests:
- Testing file loading, tag editing and saving from GUI.
- Testing initial config of options while loading Picard.
- Testing config value changes from GUI.
- Testing restore default actions.
Example proof of concept code can be found at -
[https://gist.github.com/samj1912/df744464b55dde997bda7db49d7efed1](https://gist.github.com/samj1912/df744464b55dde997bda7db49d7efed1)
## Phase 3 ( July 29 - August 29)
* * *
### Continuation of Unit-Testing
The unit-testing part of the project will extend into the Phase 3 timeline for proposal.
### Plugin API v2
Picard 2.0 is going to be PyQt5 based, which means all the existing plugins have to be ported to PyQt5/Py3. Since Picard v1 plugins will be incompatible with v2 and vice versa, we need to implement a new v2 endpoint for the Picard website. I plan to implement the basic functionality which allows PW to serve plugins for both Picard v2 and v1.X. All the plugins also need to be ported to Qt5/Py3.
A basic port of some of the plugins can be found at: [PICARD-PLUGINS: PR-69](https://github.com/metabrainz/picard-plugins/pull/69)
A basic proof of concept PR for v2 end point can be found at [PW: PR-59](https://github.com/metabrainz/picard-website/pull/59)
I plan to expand on this using the following:
- Extend the Picard WS code to query the appropriate plugin endpoint.
- Draft a ‘Version Identification and Dependency Specification’, borrowing guidelines from [PEP-440](https://www.python.org/dev/peps/pep-0440/) to ensure compatible plugins are downloaded by users and prepare a plugin contribution guide enforcing the same
# Proposed Project Timeline
* * *
### May 30 - June 14
```
Port to Qt5
```
### June 15 - June 21
```
Port to Py3
```
### June 21 - June 30
```
Buffer Period
```
### July 1 - July 14
```
Code cleanup
```
### July 15 - August 10
```
Unit Testing
GUI Testing
```
### August 11 - August 21
```
Plugin API v2
```
### August 21 - August 29
```
Buffer Period
```
# Possible extensions
* * *
In case I am finished with my GSoC project early, I plan to use the rest of my time with the following tickets and ideas:
1. Re-write the threading code for Picard. See [PICARD-975](https://tickets.metabrainz.org/browse/PICARD-975)
2. Re-design the current icons to adopt a more modern Flat/material look paving the way for a better UI/UX. See [PICARD-879](https://tickets.metabrainz.org/browse/PICARD-879) and [Picard Survey and First Impressions](https://community.metabrainz.org/t/picard-survey-and-first-impressions/189889/22)
3. Stripping down complicated additional option and features and moving them to Picard-Plugins and rewriting the plugin code to set up future development for a more modular Picard.
# About Me
* * *
> **Tell us about the computer(s) you have available for working on your SoC project!**
I have a [HP Pavillion P077tx](http://support.hp.com/us-en/product/HP-Pavilion-15-p000-Notebook-PC-series/6936226/model/7280165/product-info) on which I have thrown a Samsung SSD. On top of it I have an Ubuntu 16.10. I am a hardcore Sublime user and have modded it enough to act as a Python IDE.
> **When did you first start programming?**
I first started programming about 10 years ago with HTML/CSS and was introduced to C programming 8 years ago. In class 11th I met the current love of my coding life - Python and have been a strong advocate for it ever since. I love the beauty and readability of Python and the fact that I can automate just about anything with a small hacky script.
> **What type of music do you listen to? (Please list a series of MBIDs as examples.)**
Currently looping Ed-Sheeran’s latest album Divide (b5e39876-ca9d-402a-b789-b5e24445dc81), I am also a fan of AC/DC (cd666a28-355b-3a2b-a321-35aab16c4f89) and occasionally Anime OST (d501f231-3043-4840-9fe2-8041ede14f6c)
> **What aspects of the project you’re applying for (e.g., MusicBrainz, AcousticBrainz, etc.) interest you the most?**
I have been a data lover since long ( lurking in r/dataisbeautiful since 5 years ). I had a very unorganised library except a few titles, and I wanted to properly tag and identify them along with adding cover arts. I searched online and came across Picard and MB. My music library now has more metadata than my music player knows how to deal with. 😆
Data and Python lover meets Picard. Need I say more? 😉
> **Have you ever used MusicBrainz to tag your files?**
Yes I have. Mainly using Picard. See above point for my fangasms.
> **Have you contributed to other Open Source projects? If so, which projects and can we see some of your code?**
- I have contributed extensively to the source code for Picard fixing over[70 tickets](https://tickets.metabrainz.org/issues/?filter=-1&jql=project%20%3D%20PICARD%20AND%20status%20%3D%20Closed%20AND%20assignee%20in%20(samj1912)%20ORDER%20BY%20status%20DESC%2C%20updated%20DESC) in Jira for Picard and about [100 PRs](https://github.com/metabrainz/picard/pulls/samj1912) to Picard GH
- I am also the currently maintaining the [Picard-Website](https://tickets.metabrainz.org/projects/PW/summary).
- I have also contributed to JIRA and GH plugins for [BrainzBot]
([Pull requests · metabrainz/brainzbot-plugins · GitHub](https://github.com/metabrainz/botbot-plugins/pulls?q=is%3Apr+is%3Aclosed+author%3Asamj1912))
- I have also contributed [bug-fixes](https://bugzilla.gnome.org/page.cgi?id=describeuser.html) for Gnome-music
- Also made a [few PRs](https://github.com/peragro/peragro-at/pulls/samj1912) at Peragro-AT
- Numerous other projects can be found on my [GH profile](https://github.com/samj1912)
> **What sorts of programming projects have you done on your own time?**
Please check my [GH](https://github.com/samj1912) and my [LinkedIn](https://www.linkedin.com/in/sambhav-kothari-ba303aab/) for detailed descriptions of my personal projects. Notable ones are [fbrecog](https://github.com/samj1912/fbrecog), [sigmaShare](https://github.com/samj1912/sigmaShare), [Genesis](https://github.com/samj1912/Genesis), [acad\_hub](https://github.com/samj1912/acad_hub) and [sigma\_search](https://github.com/samj1912/sigma_search), all public repos on my GH 🙂
Apart from the above I have experience as a Python Full Stack developer. See my [LinkedIn](https://www.linkedin.com/in/sambhav-kothari-ba303aab/) for more details on that
> **How much time do you have available, and how would you plan to use it?**
I have most of my summer free(see below) and I plan on working 30-40 hrs a week on my GSoC project with GSoC being my full time project.
> **Do you plan to have a job or study during the summer in conjunction with Summer of Code?**
I have a prospective intern between May 1st to June 15 (15 days into the coding period) during which I will be free on weekends and am willing to commit to 10 hours of coding every week during this time. I plan on starting to code my proposal right from the beginning of community bonding period.
Since I am already pretty familiar with the Picard code and have been an active part of the MeB community since December, I think it will be a better utilization of my time during the community bonding period to already start coding in order to catch up for the time I will not be working full time on GSoC (the initial 15 days).
I have already made some progress on my proposal which can be found at [PICARD: PR-503](https://github.com/metabrainz/picard/pull/503)
---
### Author:  [@samj1912](https://community.metabrainz.org/u/samj1912)
#### Post date: [March 12, 2017, 11:11am UTC](https://community.metabrainz.org/t/gsoc-2017-picard-v2-0/227579/2 "2017-03-12T11:11:11Z")
The first draft of my proposal is complete. Reviews, critique and suggestions appreciated.
Comments on the py3 target version also appreciated.
---
### Author:  [@Leo\_Verto](https://community.metabrainz.org/u/Leo_Verto)
#### Post date: [March 12, 2017, 3:16pm UTC](https://community.metabrainz.org/t/gsoc-2017-picard-v2-0/227579/3 "2017-03-12T15:16:01Z")
First of all, great proposal!
However I’m not sure what exactly you mean with “A working GUI port”? Isn’t that part of porting the “main Picard tagger”?
As for the targeted Python version, I’d say 3.6 would be the better choice.
Although most of the major Linux repos currently don’t offer Python 3.6 as the “python3” package, there is already a “python3.6” package in both the latest versions of Debian and Ubuntu and it seems likely they’ll have moved to that by the time your GSoC project is done. Python 3.4 might however be better for allowing backporting, if you’re planning to do that.
Also I really don’t think you’d need any sort of community binding anymore. 😛
---
### Author:  [@samj1912](https://community.metabrainz.org/u/samj1912)
#### Post date: [March 12, 2017, 3:20pm UTC](https://community.metabrainz.org/t/gsoc-2017-picard-v2-0/227579/4 "2017-03-12T15:20:54Z")
> However I’m not sure what exactly you mean with “A working GUI port”? Isn’t that part of porting the “main Picard tagger”?
I just realized I didn’t make the distinction clear :P.
The first point emphasizes a port to Qt5/Py3 which involves passing all the currently included tests.
The GUI port however includes things not covered by the current tests. As of right now, there are times when a commit or a PR passes all the tests but Picard crashes on start. It might also crash on querying or interacting with GUI elements. Thus I wanted to make the distinction clean with a code port(all tests passing) and a working GUI port which includes the elements I have listed above.
---
### Author:  [@Freso](https://community.metabrainz.org/u/Freso)
#### Post date: [March 13, 2017, 8:17am UTC](https://community.metabrainz.org/t/gsoc-2017-picard-v2-0/227579/5 "2017-03-13T08:17:39Z")
> [@samj1912](#):
>
> For High DPI Pixmaps
Just a thought but… Does Qt5 support using SVGs for icon images? If so, that might be even better than have X number of duplicate bitmap files for varying resolutions.
---
### Author:  [@samj1912](https://community.metabrainz.org/u/samj1912)
#### Post date: [March 13, 2017, 8:37am UTC](https://community.metabrainz.org/t/gsoc-2017-picard-v2-0/227579/6 "2017-03-13T08:37:18Z")
> [@Freso](#):
>
> Just a thought but… Does Qt5 support using SVGs for icon images? If so, that might be even better than have X number of duplicate bitmap files for varying resolutions.
Just looked this up and yes, it does. See [SVG to Pixmap for High DPI screens](http://falsinsoft.blogspot.in/2016/04/qt-snippet-render-svg-to-qpixmap-for.html)
Will update the proposal accordingly! Thanks for the suggestion 🙂
---
### Author:  [@Zas](https://community.metabrainz.org/u/Zas)
#### Post date: [March 13, 2017, 12:43pm UTC](https://community.metabrainz.org/t/gsoc-2017-picard-v2-0/227579/7 "2017-03-13T12:43:16Z")
Overall a very good and detailed proposal.
@samj1912 is now well-known among the core team, as being very active and inventive, and his interest in Picard is very welcome.
About using SVG for icon images… definitively the way to go.
That said Picard currently displays bitmap files for cover art and that will need to be reworked for High DPI screens.
---
### Author:  [@samj1912](https://community.metabrainz.org/u/samj1912)
#### Post date: [March 13, 2017, 1:06pm UTC](https://community.metabrainz.org/t/gsoc-2017-picard-v2-0/227579/8 "2017-03-13T13:06:54Z")
> [@Zas](#):
>
> Overall a very good and detailed proposal.@samj1912 is now well-known among the core team, as being very active and inventive, and his interest in Picard is very welcome.
Thank you! 😃 More than happy to be a part of the Picard team!
> [@Zas](#):
>
> About using SVG for icon images… definitively the way to go.That said Picard currently displays bitmap files for cover art and that will need to be reworked for High DPI screens.
Yes, updated my proposal to reflect the same, although it has a related qt-bug which I have mentioned too.
---
### Author:  [@Sophist](https://community.metabrainz.org/u/Sophist)
#### Post date: [March 30, 2017, 6:43pm UTC](https://community.metabrainz.org/t/gsoc-2017-picard-v2-0/227579/9 "2017-03-30T18:43:49Z")
Not that my vote counts for anything when it comes to GSoC decisions, however…
I also think this is a well written proposal and that sam1912’s GSoC project would be invaluable.
---
### Author:  [@Freso](https://community.metabrainz.org/u/Freso)
#### Post date: [March 11, 2020, 5:06pm UTC](https://community.metabrainz.org/t/gsoc-2017-picard-v2-0/227579/10 "2020-03-11T17:06:46Z")