Name: Xianyi Nie
IRC: Xianyi
Github: Xianyi Nie
Email: niexianyi01@gmail.com
Timezone: GMT-4 hours
Integrate more music services for recording listens and playing music
Project Overview
ListenBrainz has a number of music discovery features that use BrainzPlayer to facilitate track playback.BrainzPlayer (BP) is a custom React component in ListenBrainz that uses multiple data sources to search and play a track. As of now, it supports Spotify, Youtube and Soundcloud as a backend. ListenBrainz zalso supports linking a Spotify account to record listening history. Integrate the external music service(Apple Music) in ListenBrainz to make adding other music services easier.
Implementation
1. Allow the user to link the Apple Music to record history
- The front-end page of the functional mock-ups is shown in the following figure:
1.1 Modify the database and add functions which can get data we need through the sqlalchemy
-
Modify the USER ListenBrainz database, for example: I would modify the
admin/sql/create_types.sql, add new typeALTER TYPE external_service_oauth_type ADD VALUE 'appleMusic'; -
Add the function which can returns a list of users whose listens should be imported from Apple Music;
def get_active_users_to_process() -> List[dict]: -
Add the function which can return user’s Apple Music linking details to display on connect services page;
def get_user_import_details(user_id: int) -> Optional[dict]: -
Add the function get the recent listened list of the Apple Music;
def get_user(user_id: int) -> Optional[dict]:
1.2 Set up the Apple Music API configuration and save the user Oauth token to the Backend for storage
- Generating Developer Tokens
- Authorize users to connect the AppleMusic service through the Apple Music Documentation API:
const appleMusic = MusicKit.getInstance(); // Check authorization before accessing user's iCloud Music Library: await appleMusic.authorize(); const result = await appleMusic.api.request('v1/me/library/played'); // User's iCloud Music History console.log(result.data.data); //test the data -
Backend: Add the following functions which can save the token in the database
def get_user(self, user_id: int) -> Optional[dict]: # get the message of the user return applemusic.get_user(user_id) def add_new_user(self, user_id: int, token: dict) -> bool: external_service_oauth.save_token(user_id=user_id, service=self.service, access_token=access_token, refresh_token=refresh_token, token_expires_ts=expires_at, record_listens=active, scopes=scopes) -
WebServer: Set an Endpoint to accept the setting from the user and delivered the user message
@profile_bp.route('/music-services/details/', methods=['GET']) @login_required def music_services_details(): applemusic_service = AppleMusicService() applemusic_user = applemusic_service.get_user(current_user.id) // Implement the Apple Music Permission here return render_template( 'user/music_services.html', applemusic_user=applemusic_user, current_appleMusic_permissions=current_applemusic_permissions, )
1.3 Deal with the user authorization process
-
Backend: Add the following functions which can get the authorization information:
def get_authorize_url(self, permissions: Sequence[str]): def fetch_access_token(self, code: str): """ Get a valid Apple Music Access token given the code. Returns JS: a dict with the following keys { 'access_token', 'token_type', 'scope', 'expires_in', 'refresh_token', } """ -
Webserver: Set an Endpoint for the music service callback to finish the authorization process
@profile_bp.route('/music-services/<service_name>/refresh/', methods=['POST']) @api_login_required def refresh_service_token(service_name: str): """ Returns JS: a dict with the following keys { 'access_token' }
1.4 Deal with the process of refreshing the access token
-
Backend: Add the following functions which can revoke the new issued token and ask the user to authenticate again
def refresh_access_token(self, user_id: int, refresh_token: str): """ Refreshes the user token for the given applemusic user. Returns: user (dict): the same user with updated tokens -
Webserver: Set an Endpoint to deliver the new access token
@profile_bp.route('/music-services/<service_name>/refresh/', methods=['POST']) @api_login_required def refresh_service_token(service_name: str): """ Returns JS: a dict with the following keys { 'access_token' }
1.5 Deal with the process of revoking the access token
-
Backend: Add the following functions which can revoke the new issued token and ask the user to authenticate again
def revoke_user(self, user_id: int): def _revoke_token(self, user_id: int, access_token: str): """ Returns JS: a dict with the following keys { 'access_token' } -
Webserver: In order to allow user changing permissions in a single step, we need to set an Endpoint to disconnect the services.
@profile_bp.route('/music-services/<service_name>/disconnect/', methods=['POST']) @api_login_required def music_services_disconnect(service_name: str):
1.6 Add front-end components to allow users to link the Apple Music
-
Noted: This is the demo of the part HTML
<div class="music-service-selection"> <form action="{{ url_for('profile.music_services_disconnect', service_name='applemusic') }}" method="post"> {{ service_permission_button("applemusic", current_applemusic_permissions, "both", "Activate both features (Recommended)", "We will record your listening history permanently and make it available for others to view and explore. " "Discover and play songs directly on ListenBrainz, and import/export your playlist to and from Apple Music.") }} -
The recent listens history can be displayed in the list
1.7 Continue updating recently played resources from Apple Music API and submit them to ListenBrainz
-
Set the update interval
Update_Interval = 60 -
Add the function which can make an request to the Apple Music API for particular user at specified endpoint
def make_api_request(user: dict, endpoint: str):-
Request URL:
https://api.music.apple.com/v1/me/recent/played -
Query Parameters:
Parameters Type Content l String The localization to use, specified by a language tag limit Integer The number of objects or number of objects in the specified relationship returned. offset String The offset to use for a paginated request. include String Additional relationships to include in the fetch. extend String A list of attribute extensions to apply to resources in the response.
-
-
Add the function which can update recently played songs for this user and submit them to ListenBrainz, set the update interval.
def process_user(user: dict, service: SpotifyService) -> int: def submit_listens_to_listenbrainz(user: Dict, listens: List, listen_type=LISTEN_TYPE_IMPORT):- Main response properties of the resource:
Parameters Type Content id String (Required) Persistent identifier of the resource. type String (Required) The type of resource. href String A URL subpath that fetches the resource as the primary object. This member is only present in responses. meta Resource.Meta Information about the request or response. The members may be any of the endpoint parameters.
- Main response properties of the resource:
2. Integrate the AppleMusic in BrainzPlayer
- The front-end page of the functional mock-ups is shown in the following figure:
2.1 Add the AppleMusicPlayer Interface in BrainzPlayer, for example:
export default class AppleMusicPlayer
extends React.Component<AppleMusicPlayerProps, AppleMusicPlayerState>
implements DataSourceType{
}
- Implement the constructor in export default class;
- Add the function that can get the trackID from the listen;
- Add the function that can get the link from the listen, etc;
2.2 Implement RefObject of React for Apple Music
- Update the AppleMusic Player in different functions.
- Update the render with AppleMusic Player.
2.3 Add front-end components to allow users to link the Apple Music:
TimeLine
Before the GSoC (Before 19th May)
- Set up ListenBrainz local Server development environment
- Read the related tickets and solve the good-first-bug.
- Discuss with the mentor and community members about my project and ask for suggestions.
Community Bonding Period
- Discuss more details with my mentor and community members, bonding with other developer.
- Fix more issues in Jira related to ListenBrainz and read more documents to understand the project more better.
Phase 1: Week 1 - Week 6
Week 1
- I will modify the database and finish the class which can get data we need through the sqlalchemy
- Get familiar with the class in the whole project.
Week 2
- I will set up the AppleMusic API configuration and save the user Oauth token to the Backend for storage
Week 3
- I will complete the process of the authorization information.
- Write document about the current part.
Week 4
- I will deal with the process of refreshing and revoking access token. and finish both of the Backend part and the Webserver part.
Week 5
- I will finish the class can continue updating recently played resources from Apple Music API and submit them to ListenBrainz
Week 6
- I will add front-end components and test the part of the link works fine.
- Write document about the current part.
Milestone: Preliminary Deliverables
- Support linking a Apple Music account to record listening history and display them in the website.
Phase 2: Week 7 - Week 12
Week 7-8
- I will write JSX to add the AppleMusicPlayer Interface in BrainzPlayer.
- Integrate the AppleMusic Player as a RefObject of React in BrainzPlayer and work on the render part.
Week 9-10
- I will update the front-end components of the BrainzPlayer and discuss with my mentor and community members about the suggestions of my current part.
- Write document about the current part.
Week 11
- Fix related bugs, and write and combine the document of the whole part.
Week 12
- Buffer for unexpected delay and issues, and discuss with my mentor and community members about the suggestions of my final evaluation deliverables
Final Evaluation Deliverables
- A new music service with AppleMusic integration for users to play and record listens on ListenBrainz.
After the GSoC
- I would continue contributing to ListenBrainz in the future and add more future of the BrainzPlayer
- Try to become a mentor at the next GSoC.
Information about Myself
My name is Xianyi Nie, I’m currently a graduate student at Georgia Institute of Technology and my major is Computer science & Engineering. This is the first time I’m participating in GSoC and I’m so excited. I am new to open source contributions, but I am well aware of Java,Python, data structures and algorithm, HTML, CSS, and JavaScript, SQL, React. Besides, I have experience in full-stack development.
Tell us about the computer(s) you have available for working on your SoC project!
I have a MacBookPro with the M1 chip and 16 GB RAM
When did you first start programming ?
I began to program in C language in the fourth grade of my primary school
What type of music do you listen to ?
I usually listen to pop from Taylor Alison Swift , Jay Chou . Besides, I also listen to the Classical Guitar Music and Classical Piano Music.
What aspects of the project you’re applying for interest you the most?
In fact, I’m a music enthusiast, and I’m also a programming enthusiast. I have played the guitar for more than five years, and my piano skill has reached level 10. Once I found that this project is related to my technology stack, and the ListenBrainz aims to keeps tracks of what music you listen, I can’t wait to devote myself to this project and the community of MetaBrainz. Because I was trying to find an integrated website can collect all the music I loved, instead of being divided in different music app. I really intersted in this ListenBrainz project so I want to make it better through my efforts.
Have you ever used MusicBrainz to tag your files?
I haven’t used it because I didn’t know it before. And now I start to use it.
If you have not contributed to open source projects, do you have other code we can look at?
Yes, I did various full-stack projects like Travel Reservations Service System, Parking Spaces Detection Based on Machine Learning and many other projects which are available on my GitHub.
What sorts of programming projects have you done on your own time?
I have done some full-stack projects on my own time, like Travel Reservations Service System, Customer Feedback-Collection Application.
In the project of Flight Reservations System:
- Developed a full-stack travel reservation project for users and administrators, which can help users manage travel itineraries in an integrated manner and allow administrators to manage flights and customers.
- Implemented stored procedures to modify the database state and implemented views to provide the data presentation based on MySQL workbench.
- Designed the user interfaces by React frameworks in JavaScript to complement front-end web integrated with back-end database system by Express.
In the project of Customer Feedback-Collection Application:
- Developed a feedback-collection application for start-up company, which can be applied to collect survey responses from apps and generate a visualization report to customers.
- Applied NodeJS in JavaScript to construct the server on Heroku and build reusable user inputs with Redux Form, completed with navigation, handled credit cards, and received payments from users with Stripe.
- Utilized Email provider recorded the feedback, stored the collected data in MongoDB by Express backend.
- Enhanced authentication flows in app with Google OAuth authentication.
How much time do you have available, and how would you plan to use it?
I plan to use 30 hours per week to the project. The total duration of the 12-week project is about 350 hours. And I can contribute more time to the community if the project needed.
