GSoC 2020: Designing Responsive UI and Completing Fragmented Features of MusicBrainz(Android)

Tags: #<Tag:0x00007f7574a95650> #<Tag:0x00007f7574a94f20> #<Tag:0x00007f7574a94c00>

Personal Information

Name: Somal Chakraborty
Email : somalchakrabortyy@gmail.com
Location: Delhi, India
Timezone: UTC+5:30 (IST - India)
Educational Institution : National Institute of Technology,Warangal.
IRC nick: SomalRudra
GitHub: https://github.com/SomalRudra

Project Background and Overview

MusicBrainz for Android was first created in 2010-11. In the year 2019,the app went through a significant update part of GsoC’19 .The complete codebase was shifted to adhere to the new Android Ecosystem.The relevant updates had been made.The core of the app is now clean and in a much better condition. However,basic feaures like the Tagging facility is yet to be implemented.Also,the app’s UI must be completely redesigned as it needs to have a responsive layout in order to stay relevant in the current Android Environment and to keep up to the current Android Users’ need.

Proposal

My Project is aimed at:

1. Fixing existing bugs and feature issues.

2. Implementing the Tagging Feature completely.

3 Creating and integrating a responsive UI for users for which redesigning of some existing features and adding new features are proposed.This will be done in 2 steps. In the first one, we will add backend code for them and in the second one, the necessary GUI will be added.

Step:1-Tag Feature

Along with fixing the existing bugs,I would like to start working on the Core of the MusicBrainz app-The Tag feature, from the very first week.

Going by the last year’s wrap on this project: The app needs to achieve complete usability with the desktop version of Picard. Many audio formats are not suppored yet. There is no support for Cover Art. In the supported formats, not all tags are currently be allowed. At many points, the JAudioTagger library being used underneath to interact with tags limits the usability. Either work has to be done at developing at a new library for JAVA which fulfills the needs or otherwise a suitable C/C++ can be used with the help of Andrid NDK and JNI.

Hence,The Tagging facility implementation is complete in fragments/partially i.e. the users can not Edit/Update/Save the metadata fields, namely, Title, Album, Artist, Track Number,Total Tracks etc.The app just Shows a Toast message when the user sends a request to perform any operation on the Metadata.

The Tag feature will be implemented in Two steps:

Extracting The Audio Fingerprint

After the request for reading the audio file has been made by the user, the app needs to perform a lookup for the metadata assets using the fingerprint.The fingerprint will be a key tool to the server in order to fetch the correct metadata from the MB database pertaining to the file.

The blueprint of the implementation will match desktop Picard’s implementation.

In order to achieve that,I will use the following tools

1. Chromaprint This library will be used for generating audio fingerprints, mainly to be used with the AcoustID service.
—Input: It needs raw audio stream (16-bit signed int) on input. The audio can have any sampling rate and any number of channels. We’ll use a native library for decoding compressed audio files and feed the result into Chromaprint.
—Output: We’ll use the 64 bit encoded String output to send the fingerprint to the required service.
2. Inorder to use Chromaprint in Android, we’ll be using the FpCalc-android wrapper.
It will produce JSON output, which would then be parsed into Java in order to render it to the User View.

Implementation details

The AcoustID fingerprint will be calculated using the FpCalc Android wrapper through Chromaprint algorithm. The audio fingerprint will be sent to the AcoustID webservice. A lookup request will be performed on MBIDs of the recordings returned by AcoustID. The result will be displayed to the user in a tabular format and the user can save the metadata.

Edit | Update | Save Metadata

Which library to be used?

The existing MB Android app has JAudioTagger implemented in order to perform existing metadata operations.The desktop picard uses Mutagen.

As discussed with the community members, Mutagen is buggy to implement and to use when it comes to Android Environment.

Also,there are some limitations with the currently implemented JAudioTagger for which Taglib is a better alternative.

Why Taglib over Jaudiotagger?

File Formats TAGLIB JAudioTagger
Mp3
Mp4
Ogg Vorbis comments
Vorbis Comments in Flac
MPC X
Speex X
WavPack X
TrueAudio X
ASF X
AIFF X
WAV

Features of Taglib that makes it suitable for this application (Reference: https://taglib.org/)

TagLib is fast - tests have shown it to be about 6 times faster than id3lib and 3 times faster than libvorbisfile at reading tags (CPU time)

TagLib is clean - TagLib is written in clean, object oriented C++ using a programming style common in KDE and Qt library programming

TagLib is easy - TagLib offers an abstraction layer that makes it easy to ignore the differences between the different file formats and their implementations

TagLib is powerful - for those that want to look under the hood TagLib provides access to the implementations of the individual file formats and provides a toolkit for doing advance manipulation of audio meta-data

TagLib is well documented - Every class, namespace, function and enumeration in TagLib is documented

TagLib supports Unicode - both ID3v2 and Ogg Vorbis standards are designed to support Unicode, so is TagLib (many tagging libraries and frontends are not)

TagLib is extensible - TagLib does not implement every feature of ID3v2, but does instead make it possible for application authors to extend TagLib to support the specific features that they need in their applications

TagLib is not toolkit specific - while TagLib does use KDE-style programming conventions and C++ by default, it does not link to either Qt or Glib (in fact it has no external dependencies) and does provide Glib-style C bindings (currently for the generic API only)

Concluding on the suitability of Taglib :
The wide range of audio file formats supported by Taglib makes it a better candidate over JAudioTagger.
The dynamic growth of the development of this library and it’s significant pace to adapt to the current requirements makes it a better alternative keeping in mind the future prospective of the MB android app.

How The Library will help us accomplish our task ?
Implentation of C++ library in Java based Android Studio will be done By using Java Native Interface (JNI) having the library as a Native Method.

Essentially, The file’s existing metadata needs to be extracted and displayed in order to compare with the metadata received from MB server database.

Metadata Extraction Using Taglib
TagLib uses the CMake build system.

To use Taglib as a native library in this project,we need to make sure that the following 2 tasks are accomplished successfully:

1. Taglib is linked with the Target Library:
Target Library is the library which will contain all the instructions related to the implementation of the fundamentals such as reading the file format passed from java, implementing Taglib’s functions,sending the extracted metadatas back to the java class etc. In essence,Taglib(imported library) needs to be linked to the Target (native library) in order to accomplish these tasks.

Here I’m attaching the GitHub repo where I’ve developed the Cmake build script and configured the Cmakelists.txt file and formatted the native library’s C++ codebase to ensure a successful binding between the native library and the imported library.

2.Successful Data transfer between Java and C++:
After the metadata’s are being extracted using Taglib in the native methods, it needs to be sent to the Java Class in order to put the data into the View Table.

The datatypes we will be extensively using are :
1. To pass metadatas like Artist details,Track details,links etc: Strings
2. To pass metadatas like Year,Rating etc: Integer,Float
3. To pass metadatas like List of tracks : Arrays
Here I have attached my github repo where I’ve demonstrated the transfer of String,Int,float,Array,Java object to C++, C++ object to Java.

Implementation details of Taglib in brief
As proposed earlier: First the metadata of the track will be read. The metadata fields, namely, Title, Album, Artist, Track Number , Total Tracks etc will be included in the search query to MusicBrainz Webservice. Each item in the search result, will be processed to calculate a Similarity score using the Weighted Levenshtein Distance . The best match will be displayed to the user and there will be an option to view all results and select an alternative as well. Then, upon clicking the save button the metadata will be written to the audio file.

Step:2-Fixing and Redesigning Existing Design Issues

.

Dashboard Design(Frontend and Backend):

Currently:The current Dashboard uses Simple LinearLayout based UI.

Proposed Update:For better appeal and a user friendly interactive Design, I propose the following Mock UI which is based on the CardView widget and a Collapsible Action Bar . Cardview is part of the updated v7 Support Libraries .

MOCK UI: https://youtu.be/0nAEx2NxRXk
(MAKE SURE TO WATCH ALL THE MOCK UI VIDEOS IN FULL SCREEN MODE)

After the UI has been implemented in the frontend:

Previous codebase needs to be modified completely in order to integrate with the frontend design proposed. Also,Since “Search” Feature will be moved to a different activity page,it requires a new Java class and associated layout in .xml file to be implemented.

Redesigning the File Selection Strategy(Backend):

Currently: Tag->Select a FIle->Select file from the files listed of the directory already set .

Proposed Update: Tag->Select a File-> Option to choose directory ->list all the files in the directory->Select audio file from the files listed in the directory.

Which will enable the user to change the directory in Runtime.

Login(Backend)

After the user has successfully logged in, The redirected page doesn’t show the update on successful login.

Therefore a feature to show the status of the login or a new redirected activity navigation will be implemented.

Updating the App Icon

The current app icon will be replaced by the Picard icon
New icon:
musicbrainzlogo (2)
Old icon:
launcher (2)

Step3-Adding New Features

SplashScreen

A Splash Screen will be designed.

A new handler will be implemented to allow to process the Runnable object i.e. the Splash Screen UI associated with the thread’s MessageQueue . After the allowed Splash screen time limit has been reached the applicaton process will be communicting back with UI thread from the background thread .

(MAKE SURE TO WATCH ALL THE MOCK UI VIDEOS IN FULL SCREEN MODE)

MOCK UI: Mock UI is attached with the Walkthrough MOCK UI link

Android Walkthrough | Onboarding S creen Design

Our android app can have a range of users from a novice to an experinced Picard user.

For a novice who didn’t use Picard, a quick guide would really be a helpful tool.Also once

the app has been updated a quick guide will help the user to land in the right place.

Keeping all these in mind and realising the need to have a proper guide on how to use the Android application efficiently, I propose the following design :

I will make an Android Walkthrough/Onboarding screen Design which will have the basic details in very brief so as to guide the user Step by Step

For example: The Tagging facility requires the user to know

  1. Grant MB the storage permission so that it can show you all the audio files that you have

  2. From the listed audio files,pick your audio

  3. Select scan to get all the metadatas about your audio file

  4. Compare, fix the tags and save to internal storage

MOCK UI Is shown below

MOCK UI: https://youtu.be/34HlHvSVzLE

For this I will use the ViewPager layout container from updated v7 Support Libraries .

Combined with this, I will include Animations Created in Adobe After Effects.

Comparison Table

Once the Metadata is fetched and ready to be compared, The Activity Layout will prompt a panel to be lifted inorder to see Comparison Table

The comparison table implementation will be same as the Picard implementation.The table will have 3 columns. One for the tags and other two for the old and new tag values.

The Slidepane Feature will be implemented using SlidingPaneLayout of the Special-purpose layout containers from updated v7 Support Libraries .

MOCK UI https://youtu.be/8sjMMaWnYJE

Improving Theme Support(Experimental)
This includes implementing the Dark(Night) and Light(Day) theme.
This feature prevents the user from straining their eyes when there is not sufficient light around.
Night and Day mode will be implemented across all the activities/user navigation pages.

This will be implemented using the AppCompatDelegate package from AndroidX support libraries.

Optional Deliverables

If time permits:

Moving the codebase to Kotlin will be taken into action. As this requires the entire codebase to be formatted,this is a time intensive task and may be beyond the scope of Gsoc Period. But I will continue to work on this project even after GSoC to make this purpose served as Android is heading towards Kotlin with a decent pace.Also, I have a prior experience with working in Kotlin which may prove helpful in this case.

Documentation

I will be documenting the code side by side keeping the following things in mind:

Introducing the changes I’ve made.

There may be certain edge-case exceptions (because of the huge variation in mobile screen sizes) that are noticed. Future developers will have to be notified about these as well.

Community feedback, interaction and suggesting future enhancements.

Current Progress

Since, I have been working on the app independently of GSoC, a part of the work is completed.

Basic functionality test of taglib in a C++ compiler is complete.

Basic NDK and JNI frameworks are designed.

Launcher Icon Issue is fixed and merged to the Master Branch of MusicBrainz’s Github Repo

Tag Icon issue is fixed and merged to the Master Branch of MusicBrainz’s Github Repo.

Splash screen and Onboarding are completely design and code ready to be linked to the Dashboard in terms of all aspects.

Estimated project timeline

Throughout the GSoC period, I’ll be working on a branch and merging as I go. Based on my mentors’ suggestion, it would be wise to submit the smaller fixes as patches to master to avoid giant merge conflicts at the end. I’ll also be debugging as I go.

Community Bonding Period(2-3 weeks)

Study MusicBrainz repo code and Taglib related projects more intensively.

Create drawable resources for ldpi,mdpi,hdpi,xdpi,xxdpi screen sizes.

Create UI mock-ups and discuss them with the community.

Fix bugs.

Coding Period

Weekly Schedule

An estimated schedule is provided alongside the detailed description of the tasks to be undertaken. Here I provide, a summary of it.

Week 1: Setting up the Android Debug Bridge.Setting up Chromaprint and fpCalc and Extracting Audio fingerprint.

Week 2: Setting up interaction with Server and fetch data from database using AcoustID Fingerprint

Week 3-4: Completion of JNI,Native Library setup. Extracting metadata using Taglib.
*Week 5:**Completing Table view with datas,Allowing user to overwrite data on internal storage.

Week 6: Integrating Spalsh Screen and Onboarding to the Dashboard, Complete the Dashboard design, Update the App icon.

Week 7: Robust navigation between 6 features mentioned in the dashboard, fix bugs and Complete Documentation of tasks completed so far .

Week 8: Work on proposed File Selection Strategy and improve user login experience. Complete Documentation of tasks completed so far .

Week 9-10(mid): Redesign all 6 Activity’s GUI and improve based on community’s feedback

Week 10(mid)-11: Implement Dark Mode Feature

Week 12-13: Bug fixes, testing and completing documentation. Buffer Period.

Detailed Information About Myself

Tell us about the computer(s) you have available for working on your GSoC project!
I have a HP laptop with 8GB RAM, 1TB Hard disk and 2GB graphics card. The main tool required for the task is Android Studio which runs smoothly on this system.

When did you first start programming?
I started programming when I was in Grade 10th as a part of school curriculum.I started by learning Python. After joining College I’ve Learned Data Structures and Algorithm using C++.Then I began learning Android using Kotlin and then moved to Java

What type of music do you listen to?

I got introduced to Music with Indian Classical Music. I received training for 3 years in INdian Classical Music. I’ve served my school’s Music Society as the President. Currently, I’m an Executive member of the Music Club,National Institute of Technology Warangal. I listen to pop music,Rock and perform them after preparing them with the designated band for 1 month or so.

What aspects of the project you’re applying for (e.g., MusicBrainz, AcousticBrainz, etc.) interest you the most?
As a part of the Music Club our work includes listening to intensive amount of music. The whole club is very dedicated and do detailing of the song to be put on stage.The prospects of cleaning up entire music library on an individual’s mobile automatically is a great help. Also I prefer mobile apps to websites, so I am making work easier for my whole community of musicians with this project.

Have you ever used MusicBrainz to tag your files?
Yes.Few years back when I was in need of this kind of software, I started Googling and landed up using this beautifully crafted Picard version.

Have you contributed to other Open Source projects? If so, which projects and can we see some of your code?
I’m currently a developer at Web and Software Development Cell team of my college which handles all Student and Faculty required tasks.It works for registering students,rendering ID-passwords,Allocating Accomodation,Processing Fee payment,Updating results.
Apart from that, I’ve contributed to the MusicBrainz(Android) repo.I have fixed 2 issues,raised 2 issues and have 2 pull requests merged to the Master Branch of MusicBrainz repo.

What sorts of programming projects have you done on your own time?
I’ve created a BloodBank Database using SQL and DBMS Concepts. I have implemented the Pokemon Catch app in Android. I’m working on a project to create a Buy&Sell system in Android as part of Makeathon. I’m also in a team of my seniors who are working on improving Drone Automation which will go onboard after this summer.

How much time do you have available, and how would you plan to use it?
I will be working full time on this project in GSoC and I’ll utilize the time as mentioned in the above weekly schedule

Do you plan to have a job or study during the summer in conjunction with Summer of Code?
I plan to study the Kotlin in more details so as to be able to contribute to move the Project’s codebase to Kotlin.

3 Likes

Or even better ask on irc freenode #metabrainz for someone (likely @mr_monkey) to design a derived icon proper to the Android app.

3 Likes

This proposal is very attractive, Android app deserves love, and proposed topics are rather well-chosen.
Overall, you got a :+1: from me.

Few remarks though:

  • it would be great to think about portability of the app to Apple iOS, are there tools around to easily generate an iPhone compatible-app?
  • keep in mind MusicBrainz website is on the move to massively use React, which paths the way to a much better compatibility with portable devices. At this point, current accessibility of the website from smartphones and tablets is just awful, so a dedicated app is welcomed but it will have to evolve along the website, avoiding to duplicate it in-app.

Whatever, that’s a solid and interesting proposal we have here.

Did you think about anyone’s to be your mentor on this? (I will pass on it, because Android ecosystem isn’t enough familiar to me and I’m not in Java-ish stuff)

4 Likes

@zas Its not easy to create a good port for iOS. React was not used in the android app because of the tagging facility requiring native libraries which are troublesome to use with react on Android. In all probability, @gr0uch0mars will be mentoring the project this as well.

The proposal looks good but I have few specific things to point @Somal01 regarding the proposal, I’ll leave detailed comments as soon as possible.

2 Likes
  • Extracting the audio fingerprint: As far as I recall, the current implementation matches that of Picard. Can you describe what do you feel is missing ?

  • Taglib: The overall comparison is generic. Add some specific points as to how you plan to integrate taglib in the application.

  • Basic NDK and JNI frameworks are designed.

I believe this task will be the hardest part and the backbone of the entire app. Please update the proposal with whatever code or technical document you have prepared on this front.

  • I like the UI you propose for the tagging screen but keep in mind that on most of the screens the metadata won’t fit on a single line and will be wrapped to multiple lines. If that’s what you intend to do, it is fine otherwise you should reconsider the mockup.

As a general rule of thumb, remember that a proposal is a technical document. You should write in a way that it guides you throughout the summer on how to proceed on your project. It should be as comprehensive as possible but not redundant.

@Somal01 These are the major contentions I have. Overall, a good first draft but it needs improvements. Rest you should coordinate with @gr0uch0mars, as he will be mentoring the project.

1 Like

Thank you for your valuable feedback.
I’ll surely approach @mr_monkey to design a proper icon.
Also,@gr0uch0mars will be mentoring the project.

As rightly pointed out by amCap1712,creating a good port for iOS is a handy task.
Also,A critical point has been addressed regarding evolving the app along with the website.
We’ll surely mark an initiation to that.

@amCap1712
Thank you for your valuable feedback.
It clarified the doubts that I had about what domain to study further to make the proposal an apt guide.
The draft was proposed to check the relevance of the ideas budding in my head.Therefore,I put into it fairly brief technicalities of the ideas.

I’ll update the proposal with more detailed explanation on these.

For this I’ll use ,multiline text fields.If that doesn’t fit well,I’ll consider making it more compact and reduce the layout_weight updating the padding along side.

@Somal01Congrats for this proposal!!! Alghouth it can be improved as per @amCap1712 suggests, you showed good intention and with good ideas!

I’d like to sum all the ideas together to organize the development of the app, as maybe we could split between the Picard Android feature, and the app itself (redesign, and so). This way you each can work on a project but the efforts will revert on the same MB Android app.


@amCap1712 could you please describe the technical challeges that would prevent us from making Picard an android library and using it within a React app? If we could split both features, would the library need special things from the system? (thinking in keeping the Infrastructure —Android APIs— as far from the Domain —Picard tagger— as possible)

@gr0uch0mars
Thank you! :smiley:
I’ll update the proposal as pointed out by @amCap1712 and improve it further

This section has been updated.