Personal Information
Name: Somal Chakraborty
Email : somalchakrabortyy@gmail.com
Location: Delhi, India
Timezone: UTC+5:30 (IST - India)
Educational Institution : National Institute of Technology,Warangal.
IRC nick: SomalRudra
GitHub: SomalRudra · GitHub
Project Background and Overview
MusicBrainz for Android was first created in 2010-11. In the year 2019,the app went through a significant update part of GsoC’19 .The complete codebase was shifted to adhere to the new Android Ecosystem.The relevant updates had been made.The core of the app is now clean and in a much better condition. However,basic feaures like the Tagging facility is yet to be implemented.Also,the app’s UI must be completely redesigned as it needs to have a responsive layout in order to stay relevant in the current Android Environment and to keep up to the current Android Users’ need.
Proposal
My Project is aimed at:
1. Fixing existing bugs and feature issues.
2. Implementing the Tagging Feature completely.
3 Creating and integrating a responsive UI for users for which redesigning of some existing features and adding new features are proposed.This will be done in 2 steps. In the first one, we will add backend code for them and in the second one, the necessary GUI will be added.
Step:1-Tag Feature
Along with fixing the existing bugs,I would like to start working on the Core of the MusicBrainz app-The Tag feature, from the very first week.
Going by the last year’s wrap on this project: The app needs to achieve complete usability with the desktop version of Picard. Many audio formats are not suppored yet. There is no support for Cover Art. In the supported formats, not all tags are currently be allowed. At many points, the JAudioTagger library being used underneath to interact with tags limits the usability. Either work has to be done at developing at a new library for JAVA which fulfills the needs or otherwise a suitable C/C++ can be used with the help of Andrid NDK and JNI.
Hence,The Tagging facility implementation is complete in fragments/partially i.e. the users can not Edit/Update/Save the metadata fields, namely, Title, Album, Artist, Track Number,Total Tracks etc.The app just Shows a Toast message when the user sends a request to perform any operation on the Metadata.
The Tag feature will be implemented in Two steps:
Extracting The Audio Fingerprint
After the request for reading the audio file has been made by the user, the app needs to perform a lookup for the metadata assets using the fingerprint.The fingerprint will be a key tool to the server in order to fetch the correct metadata from the MB database pertaining to the file.
The blueprint of the implementation will match desktop Picard’s implementation.
In order to achieve that,I will use the following tools
1. Chromaprint This library will be used for generating audio fingerprints, mainly to be used with the AcoustID service.
—Input: It needs raw audio stream (16-bit signed int) on input. The audio can have any sampling rate and any number of channels. We’ll use a native library for decoding compressed audio files and feed the result into Chromaprint.
—Output: We’ll use the 64 bit encoded String output to send the fingerprint to the required service.
2. Inorder to use Chromaprint in Android, we’ll be using the FpCalc-android wrapper.
It will produce JSON output, which would then be parsed into Java in order to render it to the User View.
Implementation details
The AcoustID fingerprint will be calculated using the FpCalc Android wrapper through Chromaprint algorithm. The audio fingerprint will be sent to the AcoustID webservice. A lookup request will be performed on MBIDs of the recordings returned by AcoustID. The result will be displayed to the user in a tabular format and the user can save the metadata.
Edit | Update | Save Metadata
Which library to be used?
The existing MB Android app has JAudioTagger implemented in order to perform existing metadata operations.The desktop picard uses Mutagen.
As discussed with the community members, Mutagen is buggy to implement and to use when it comes to Android Environment.
Also,there are some limitations with the currently implemented JAudioTagger for which Taglib is a better alternative.
Why Taglib over Jaudiotagger?
| File Formats | TAGLIB | JAudioTagger |
|---|---|---|
| Mp3 | √ | √ |
| Mp4 | √ | √ |
| Ogg Vorbis comments | √ | √ |
| Vorbis Comments in Flac | √ | √ |
| MPC | √ | X |
| Speex | √ | X |
| WavPack | √ | X |
| TrueAudio | √ | X |
| ASF | √ | X |
| AIFF | √ | X |
| WAV | √ | √ |
Features of Taglib that makes it suitable for this application (Reference: https://taglib.org/)
TagLib is fast - tests have shown it to be about 6 times faster than id3lib and 3 times faster than libvorbisfile at reading tags (CPU time)
TagLib is clean - TagLib is written in clean, object oriented C++ using a programming style common in KDE and Qt library programming
TagLib is easy - TagLib offers an abstraction layer that makes it easy to ignore the differences between the different file formats and their implementations
TagLib is powerful - for those that want to look under the hood TagLib provides access to the implementations of the individual file formats and provides a toolkit for doing advance manipulation of audio meta-data
TagLib is well documented - Every class, namespace, function and enumeration in TagLib is documented
TagLib supports Unicode - both ID3v2 and Ogg Vorbis standards are designed to support Unicode, so is TagLib (many tagging libraries and frontends are not)
TagLib is extensible - TagLib does not implement every feature of ID3v2, but does instead make it possible for application authors to extend TagLib to support the specific features that they need in their applications
TagLib is not toolkit specific - while TagLib does use KDE-style programming conventions and C++ by default, it does not link to either Qt or Glib (in fact it has no external dependencies) and does provide Glib-style C bindings (currently for the generic API only)
Concluding on the suitability of Taglib :
The wide range of audio file formats supported by Taglib makes it a better candidate over JAudioTagger.
The dynamic growth of the development of this library and it’s significant pace to adapt to the current requirements makes it a better alternative keeping in mind the future prospective of the MB android app.
How The Library will help us accomplish our task ?
Implentation of C++ library in Java based Android Studio will be done By using Java Native Interface (JNI) having the library as a Native Method.
Essentially, The file’s existing metadata needs to be extracted and displayed in order to compare with the metadata received from MB server database.
Metadata Extraction Using Taglib
TagLib uses the CMake build system.
To use Taglib as a native library in this project,we need to make sure that the following 2 tasks are accomplished successfully:
1. Taglib is linked with the Target Library:
Target Library is the library which will contain all the instructions related to the implementation of the fundamentals such as reading the file format passed from java, implementing Taglib’s functions,sending the extracted metadatas back to the java class etc. In essence,Taglib(imported library) needs to be linked to the Target (native library) in order to accomplish these tasks.
Here I’m attaching the GitHub repo where I’ve developed the Cmake build script and configured the Cmakelists.txt file and formatted the native library’s C++ codebase to ensure a successful binding between the native library and the imported library.
2.Successful Data transfer between Java and C++:
After the metadata’s are being extracted using Taglib in the native methods, it needs to be sent to the Java Class in order to put the data into the View Table.
The datatypes we will be extensively using are :
1. To pass metadatas like Artist details,Track details,links etc: Strings
2. To pass metadatas like Year,Rating etc: Integer,Float
3. To pass metadatas like List of tracks : Arrays
Here I have attached my github repo where I’ve demonstrated the transfer of String,Int,float,Array,Java object to C++, C++ object to Java.
Implementation details of Taglib in brief
As proposed earlier: First the metadata of the track will be read. The metadata fields, namely, Title, Album, Artist, Track Number , Total Tracks etc will be included in the search query to MusicBrainz Webservice. Each item in the search result, will be processed to calculate a Similarity score using the Weighted Levenshtein Distance . The best match will be displayed to the user and there will be an option to view all results and select an alternative as well. Then, upon clicking the save button the metadata will be written to the audio file.
Step:2-Fixing and Redesigning Existing Design Issues
.
Dashboard Design(Frontend and Backend):
Currently:The current Dashboard uses Simple LinearLayout based UI.
Proposed Update:For better appeal and a user friendly interactive Design, I propose the following Mock UI which is based on the CardView widget and a Collapsible Action Bar . Cardview is part of the updated v7 Support Libraries .
MOCK UI: Dashboard design - YouTube
(MAKE SURE TO WATCH ALL THE MOCK UI VIDEOS IN FULL SCREEN MODE)
After the UI has been implemented in the frontend:
Previous codebase needs to be modified completely in order to integrate with the frontend design proposed. Also,Since “Search” Feature will be moved to a different activity page,it requires a new Java class and associated layout in .xml file to be implemented.
Redesigning the File Selection Strategy(Backend):
Currently: Tag->Select a FIle->Select file from the files listed of the directory already set .
Proposed Update: Tag->Select a File-> Option to choose directory ->list all the files in the directory->Select audio file from the files listed in the directory.
Which will enable the user to change the directory in Runtime.
Login(Backend)
After the user has successfully logged in, The redirected page doesn’t show the update on successful login.
Therefore a feature to show the status of the login or a new redirected activity navigation will be implemented.
Updating the App Icon
The current app icon will be replaced by the Picard icon
New icon:
Old icon:
Step3-Adding New Features
SplashScreen
A Splash Screen will be designed.
A new handler will be implemented to allow to process the Runnable object i.e. the Splash Screen UI associated with the thread’s MessageQueue . After the allowed Splash screen time limit has been reached the applicaton process will be communicting back with UI thread from the background thread .
(MAKE SURE TO WATCH ALL THE MOCK UI VIDEOS IN FULL SCREEN MODE)
MOCK UI: Mock UI is attached with the Walkthrough MOCK UI link
Android Walkthrough | Onboarding S creen Design
Our android app can have a range of users from a novice to an experinced Picard user.
For a novice who didn’t use Picard, a quick guide would really be a helpful tool.Also once
the app has been updated a quick guide will help the user to land in the right place.
Keeping all these in mind and realising the need to have a proper guide on how to use the Android application efficiently, I propose the following design :
I will make an Android Walkthrough/Onboarding screen Design which will have the basic details in very brief so as to guide the user Step by Step
For example: The Tagging facility requires the user to know
-
Grant MB the storage permission so that it can show you all the audio files that you have
-
From the listed audio files,pick your audio
-
Select scan to get all the metadatas about your audio file
-
Compare, fix the tags and save to internal storage
MOCK UI Is shown below
MOCK UI: Onboarding MusicBrainz - YouTube
For this I will use the ViewPager layout container from updated v7 Support Libraries .
Combined with this, I will include Animations Created in Adobe After Effects.
Comparison Table
Once the Metadata is fetched and ready to be compared, The Activity Layout will prompt a panel to be lifted inorder to see Comparison Table
The comparison table implementation will be same as the Picard implementation.The table will have 3 columns. One for the tags and other two for the old and new tag values.
The Slidepane Feature will be implemented using SlidingPaneLayout of the Special-purpose layout containers from updated v7 Support Libraries .
MOCK UI Slide up - YouTube
Improving Theme Support(Experimental)
This includes implementing the Dark(Night) and Light(Day) theme.
This feature prevents the user from straining their eyes when there is not sufficient light around.
Night and Day mode will be implemented across all the activities/user navigation pages.
This will be implemented using the AppCompatDelegate package from AndroidX support libraries.
Optional Deliverables
If time permits:
Moving the codebase to Kotlin will be taken into action. As this requires the entire codebase to be formatted,this is a time intensive task and may be beyond the scope of Gsoc Period. But I will continue to work on this project even after GSoC to make this purpose served as Android is heading towards Kotlin with a decent pace.Also, I have a prior experience with working in Kotlin which may prove helpful in this case.
Documentation
I will be documenting the code side by side keeping the following things in mind:
Introducing the changes I’ve made.
There may be certain edge-case exceptions (because of the huge variation in mobile screen sizes) that are noticed. Future developers will have to be notified about these as well.
Community feedback, interaction and suggesting future enhancements.
Current Progress
Since, I have been working on the app independently of GSoC, a part of the work is completed.
Basic functionality test of taglib in a C++ compiler is complete.
Basic NDK and JNI frameworks are designed.
Launcher Icon Issue is fixed and merged to the Master Branch of MusicBrainz’s Github Repo
Tag Icon issue is fixed and merged to the Master Branch of MusicBrainz’s Github Repo.
Splash screen and Onboarding are completely design and code ready to be linked to the Dashboard in terms of all aspects.
Estimated project timeline
Throughout the GSoC period, I’ll be working on a branch and merging as I go. Based on my mentors’ suggestion, it would be wise to submit the smaller fixes as patches to master to avoid giant merge conflicts at the end. I’ll also be debugging as I go.
Community Bonding Period(2-3 weeks)
Study MusicBrainz repo code and Taglib related projects more intensively.
Create drawable resources for ldpi,mdpi,hdpi,xdpi,xxdpi screen sizes.
Create UI mock-ups and discuss them with the community.
Fix bugs.
Coding Period
Weekly Schedule
An estimated schedule is provided alongside the detailed description of the tasks to be undertaken. Here I provide, a summary of it.
Week 1: Setting up the Android Debug Bridge.Setting up Chromaprint and fpCalc and Extracting Audio fingerprint.
Week 2: Setting up interaction with Server and fetch data from database using AcoustID Fingerprint
Week 3-4: Completion of JNI,Native Library setup. Extracting metadata using Taglib.
*Week 5:**Completing Table view with datas,Allowing user to overwrite data on internal storage.
Week 6: Integrating Spalsh Screen and Onboarding to the Dashboard, Complete the Dashboard design, Update the App icon.
Week 7: Robust navigation between 6 features mentioned in the dashboard, fix bugs and Complete Documentation of tasks completed so far .
Week 8: Work on proposed File Selection Strategy and improve user login experience. Complete Documentation of tasks completed so far .
Week 9-10(mid): Redesign all 6 Activity’s GUI and improve based on community’s feedback
Week 10(mid)-11: Implement Dark Mode Feature
Week 12-13: Bug fixes, testing and completing documentation. Buffer Period.
Detailed Information About Myself
Tell us about the computer(s) you have available for working on your GSoC project!
I have a HP laptop with 8GB RAM, 1TB Hard disk and 2GB graphics card. The main tool required for the task is Android Studio which runs smoothly on this system.
When did you first start programming?
I started programming when I was in Grade 10th as a part of school curriculum.I started by learning Python. After joining College I’ve Learned Data Structures and Algorithm using C++.Then I began learning Android using Kotlin and then moved to Java
What type of music do you listen to?
I got introduced to Music with Indian Classical Music. I received training for 3 years in INdian Classical Music. I’ve served my school’s Music Society as the President. Currently, I’m an Executive member of the Music Club,National Institute of Technology Warangal. I listen to pop music,Rock and perform them after preparing them with the designated band for 1 month or so.
What aspects of the project you’re applying for (e.g., MusicBrainz, AcousticBrainz, etc.) interest you the most?
As a part of the Music Club our work includes listening to intensive amount of music. The whole club is very dedicated and do detailing of the song to be put on stage.The prospects of cleaning up entire music library on an individual’s mobile automatically is a great help. Also I prefer mobile apps to websites, so I am making work easier for my whole community of musicians with this project.
Have you ever used MusicBrainz to tag your files?
Yes.Few years back when I was in need of this kind of software, I started Googling and landed up using this beautifully crafted Picard version.
Have you contributed to other Open Source projects? If so, which projects and can we see some of your code?
I’m currently a developer at Web and Software Development Cell team of my college which handles all Student and Faculty required tasks.It works for registering students,rendering ID-passwords,Allocating Accomodation,Processing Fee payment,Updating results.
Apart from that, I’ve contributed to the MusicBrainz(Android) repo.I have fixed 2 issues,raised 2 issues and have 2 pull requests merged to the Master Branch of MusicBrainz repo.
What sorts of programming projects have you done on your own time?
I’ve created a BloodBank Database using SQL and DBMS Concepts. I have implemented the Pokemon Catch app in Android. I’m working on a project to create a Buy&Sell system in Android as part of Makeathon. I’m also in a team of my seniors who are working on improving Drone Automation which will go onboard after this summer.
How much time do you have available, and how would you plan to use it?
I will be working full time on this project in GSoC and I’ll utilize the time as mentioned in the above weekly schedule
Do you plan to have a job or study during the summer in conjunction with Summer of Code?
I plan to study the Kotlin in more details so as to be able to contribute to move the Project’s codebase to Kotlin.
from me.