Integrate Remote Playback and Listening now Proposal

Introduction

Contact Information

  • Name: Anuj
  • IRC nick: anuj_
  • Email: aanuj8619@gmail.com
  • GitHub: anuj990
  • LinkedIn: Anuj | LinkedIn

I am Anuj, a sophomore at the Indian Institute of Information Technology Jabalpur (IIITDM Jabalpur), pursuing B.Tech in Electronics and Communication Engineering.

I started my coding journey in my first semester with C++ for competitive programming. Through my college’s open source programme BSOC (BitByte Summer of Code) I found open source and Android development interesting. Since then my stack has been Kotlin, Jetpack Compose, and MVVM.

I am particularly interested in music focused products, as I regularly use them during long coding sessions. This makes ListenBrainz a project I naturally connect with, both as a developer and as a
user


Project Overview

The ListenBrainz Android app has two incomplete features. First, Listening Now which shows what the user is playing right now was removed from the UI at some point, even though the API integration is still in the codebase. Second, Remote Playback through Spotify or YouTube can only play one track. There is no queue, no auto next, nothing.

My proposal is to fix both by shared KMP module that watches all three playback sources BrainzPlayer, the LB WebSocket, and the Remote Playback queue and decides what the mini player should show


Deliverables

  1. Integrate Listening Now into the existing playback system, ensuring real-time updates
  2. Listening Now and Remote Playback into a single state model that drives the UI consistently
  3. Implement priority handling:
    • BrainzPlayer state overrides Listening Now
    • Listening Now shown when BrainzPlayer is inactive
  4. Add device detection:
    • Detect whether Listening Now is from the current device or another device using Listen Submission Service
    • Show queue only for current device playback
  5. Implement Remote Playback queue support
  6. Persist playback queue and state using Room to handle process death and app restarts
  7. Implement robust WebSocket handling
  8. Revamp Scaffold UI:
    • Display BrainzPlayer / Listening Now
    • Show queue when applicable
  9. Modularize core logic into a Kotlin Multiplatform module for reuse across platforms

Proposed System:

System Visualization

  • BrainzPlayer is active → override Listening Now from server, show BrainzPlayer state in mini player
  • BrainzPlayer is inactive → show Listening Now state only
  • Listening Now is active from this Android device → show playlist queue that are auto played by Remote Playback integrations
  • Listening Now is active from another device → show the track info only, no queue, since playback is happening elsewhere

PlaybackUiState

All UI states are represented in sealed class in commonMain, it receives one of four states and renders accordingly

sealed class PlaybackUiState {
    object Idle : PlaybackUiState()
    data class BrainzPlayerActive(val track: Track) : PlaybackUiState()
    data class RemotePlaybackActive(
        val track: Track, val queue: List<Track>
    ) : PlaybackUiState()
    data class ListeningNowOtherDevice(val track: Track) : PlaybackUiState()
}

ListeningNowViewModel

The ViewModel extends KMP ViewModel() , it combines three flows BrainzPlayer state , ListeningNow stream and RemotePlayback queue into a single StateFlow. The resolveUiState() function applies the BrainzPlayer win priority described as above, each flow given a default value so that combine() can emit on first without waiting for all three to be ready avoiding UI freeze up

combine(
    brainzPlayerRepository.playerState,       // default: Inactive
    listeningNowRepository.listeningNowFlow,  // default: null
    remotePlaybackController.queueState       // default: Idle
) { playerState, listeningNow, queue ->
    resolveUiState(playerState, listeningNow, queue)
}.stateIn(viewModelScope, SharingStarted.WhileSubscribed(5000), PlaybackUiState.Idle)

ListeningNowRepository

Lives in commonMain opens a Ktor websockets to LB socket API when the flow is collected(cold flow : no connection if nothing is observing), exponential backoff starting at 1 second, capping at 30 seconds, on every reconnection ListeningNowRepository makes one REST call to fetch the current Listening Now state before letting the WebSocket stream take over. This covers the gap. REST for the initial state on connect, WebSocket for updates after that is how the LB Socket API is designed to be used anyway, the connection lifetime is tied to ViewModelScope so it cleans up automatically when the ViewModel is cleared

RemotePlaybackController (expect/actual)

This interface lives in commonMain, androidMain provides SpotifyRemotePlaybackController, ios main provides no-op stubs that returns RemotePlaybackState.Unavailable A factory in androidMain picks the correct implementation based on the playlist’s track source

// commonMain
interface RemotePlaybackController {
    val queueState: StateFlow<RemotePlaybackState>
    suspend fun playPlaylist(tracks: List<Track>)
    suspend fun skipToNext()
}

expect fun createRemotePlaybackController():RemotePlaybackController

Spotify Queue Manager and Track-End Detection

The spotify app remote sdk already present in spotify-app-remote gradle module as spotify-app-remote-release-0.8.0.aar,ListenServiceManager which already runs as a NotificationListenerService and monitors active MediaSessions on the device is queried for the active Spotify MediaController

The MediaController exposes an onPlaybackStateChanged() callback that fires with real state constants from the Android media framework. When Spotify finishes a track, the session transitions to PlaybackState.STATE_NONE or PlaybackState.STATE_STOPPED. This is a genuine state event from the Android system, not an inferred heuristic, so it reliably distinguishes a natural track end from a user pause

ListenServiceManager exposes getActiveMediaController(packageName) , a small addition to its existing session monitoring logic, returning the MediaController for the given package if one is currently active. The Spotify App Remote SDK still handles the actual play commands, the MediaController is only used for state observation. If Spotify is not installed, SpotifyAppRemote.connect() fails and RemotePlaybackState moves to Unavailable, so the UI renders gracefully rather than crashing

// androidMain inside SpotifyRemotePlaybackController
val spotifyController = listenServiceManager.getActiveMediaController(SPOTIFY_PACKAGE)

spotifyController?.registerCallback(object : MediaController.Callback() {
    override fun onPlaybackStateChanged(state: PlaybackState?) {
        if (state?.state == PlaybackState.STATE_NONE ||
            state?.state == PlaybackState.STATE_STOPPED) {
            scope.launch { playNext() }
        }
    }
})

Device Detection via ListenServiceManager

The LB server doesn’t tag ListeningNow events with a device identifier, to determine whether an event is from this device or another ListenServiceManager is modified to expose the last submitted track as internal state ,when an event arrives it will be compared to exposed state(same track name , same artist name,within a 10 second timestamp window that covers the three possible delays : device to server latency , server processing and websocket push back)

Queue Persistence with Room DB

The active ListenBrainz playlist queue including the full track list and current index will be persisted using Room in androidMain to ensure continuity across process death and app restarts.

On each queue update (e.g., play, skip), both the queue and current index will be stored atomically using Room transactions to maintain consistency. On app launch, this state is restored and playback resumes from the last known position.

The schema will be designed to remain flexible:

  • A single row approach (storing the queue as a serialized JSON with current index) offers simplicity and fast read/write
  • A normalized approach (separate table for tracks with position + a queue state row for current index) provides better scalability and supports future operations like reordering or partial updates.

Koin Wiring

// commonMain
val playbackModule = module {
    single<ListeningNowRepository> { ListeningNowRepositoryImpl(get()) }
    viewModelOf(::ListeningNowViewModel)
}

// androidMain
val androidPlaybackModule = module {
    single<RemotePlaybackController> { createRemotePlaybackController() }
    single { QueuePersistenceDao(get()) }
}

YouTube Intent Launch, Foreground Only

YouTube has no App Remote SDK,Youtube is Handled via Android Intents only, When a playlist track has Youtube Source, the app fires an Intent to open You tube directly, there is no in app playback , no queue , no background playback for YT tracks , this is TOS safe and require no API key or Library

SoundCloud Background Queue Playback

SoundCloud is the right replacement for background continuous playback for yt. LB server already resolves tracks to SoundCloud stream URLs; there is no SoundCloud SDK involved on the client side. When a playlist track has a SoundCloud source, the app requests the resolved stream URL from the LB API and passes it directly to ExoPlayer, which is already present in the project inside BrainzPlayer

ExoPlayer handles background playback natively. Track end detection uses Player.Listener.onPlaybackStateChanged with STATE_ENDED a clean callback, no heuristics required unlike Spotify. When a track ends, the queue manager fetches the next resolved URL from the LB API and passes it to ExoPlayer.same will be implemented for iOS

A boolean flag buildQueue on the playlist model controls whether the queue manager activates for a given playlist. When buildQueue = true, continuous background playback runs. When false (YouTube case), the controller is bypassed entirely

Known Hurdles

Hurdle 1 : Spotify SDK Has No Track-End Event

Spotify has no onTrackEnd() using ListenServiceManager.getActiveMediaController() to observe PlaybackState.STATE_NONE / STATE_STOPPED from the Android media framework directly, as described above

Hurdle 2 : Device Detection Timestamp Window

Three delays stack between the moment a track starts on this phone and the moment the WebSocket event arrives back,network latency to the LB server, server processing, and the WebSocket push back to the app. On a good connection this is 1-2 seconds; on a slow connection it can reach 8-9 seconds. The 10-second detection window covers realistic worst cases. The known tradeoff is a false positive when two devices start playing the same track within 10 seconds of each other; this is an accepted limitation since the server does not provide device identifiers. Clock skew between the device clock and the LB server clock is handled by comparing relative elapsed times rather than raw timestamp subtraction, making the comparison clock independent. The window can be made configurable via a preference.

Hurdle 3: WebSocket State Gap After Reconnection

When the WebSocket reconnects after a network interruption or app backgrounding, the server pushes the current Listening Now state, but the app may have missed state changes during the gap. On every reconnect, ListeningNowRepository makes one REST call to the LB API to fetch the current Listening Now state before handing off to the WebSocket stream. This covers the gap cleanly and follows the intended usage of the LB Socket API, REST for the initial state, WebSocket for incremental updates.

Hurdle 4: Race Condition at Startup

Kotlin’s combine() waits for all three input flows to emit at least once before producing any output. At startup, BrainzPlayer state may emit immediately while the WebSocket connection is still opening, which would leave the ViewModel waiting and the mini player blank. The fix is to give each source a default initial value Inactive for BrainzPlayer, null for Listening Now, and Idle for Remote Playback queue state so resolveUiState() has something to work with from the very first emission and returns PlaybackUiState.Idle until real data arrives.

Timeline

Community Bonding (May 8-Jun 1)

Discuss Room DB queue schema design with mentor. Finalize Scaffold front layer mockups in Figma. Map the exact changes needed in ListenServiceManager to expose the last submitted track as internal state. Study ExoPlayer integration points inside BrainzPlayer and the Spotify App Remote SDK MediaSession API. Get familiar with the LB Socket API’s expected REST + WebSocket usage sequence.

Week Priority Timeline Detail
Jun 2 - 8 High Define PlaybackUiState sealed class in commonMain. Set up RemotePlaybackFactory interface. Create Koin module skeletons in commonMain and androidMain
Jun 9 - 15 Critical Implement Ktor WebSocket to LB Socket API. Exponential backoff 1s to 30s cap. One REST call on every reconnect before WebSocket stream takes over, tie connection lifetime to viewModelScope
Jun 16 - 22 Critical Implement combine() with default initial values. Implement resolveUiState() with BrainzPlayer wins priority. Wire to Koin via viewModelOf()
Jun 23 - 29 Critical Add getActiveMediaController(packageName). Implement SpotifyRemotePlaybackController. Handle Spotify not installed via Unavailable state
Jun 30 - Jul 6 Critical Fetch resolved SoundCloud stream URLs from LB API, passed to ExoPlayer. Use Player.Listener STATE_ENDED for clean track-end detection. Add YouTube intent launch. Add buildQueue flag on playlist model
Jul 7 - 13 High Modify ListenServiceManager to expose the last submitted track. Implement isThisDevice() with clock-skew-safe relative elapsed time comparison and 10-second configurable window. Implement Room DB schema queue track list and current index

Midterm Evaluation (July 14 - 18)

Submit midterm evaluation. Use this week as a buffer to address any slippage, fix blockers found by mentor, and stabilise the core flows before touching UI.

Week Priority Timeline Detail
Jul 19 - Aug 1 High Two weeks most visible deliverable. Redesign mini-player Composable to render all four PlaybackUiState variants. Implement queue list UI. Wire ListeningNowViewModel via koinViewModel()
Aug 2 - 8 High Connect all pieces end-to-end. Fix wiring issues. Verify Room DB restore on app restart. Test Spotify not installed, SoundCloud URL failure, WebSocket drop mid session, queue exhausted, YouTube intent
Aug 9 - 15 Medium Unit tests for resolveUiState() and isThisDevice(). One integration test for WebSocket reconnect. Fix anything broken
Aug 16 - 22 High KDoc for all public interfaces and ViewModels. Code cleanup. Final mentor review. Buffer for last-minute fixes before final evaluation

Extras

These are additional features I would like to work on if time permits:

  1. Configurable device detection window : expose the 10-second isThisDevice() threshold as a user preference in Settings.Users on consistently slow mobile connections could raise it; users on fast WiFi could tighten it to reduce false positives

  2. Queue reordering UI:The Room DB schema stores tracks with a position index. If the schema is normalized, drag-to-reorder becomes a natural extension of the queue list UI.

Post GSOC Plan

  1. Listening Together feature: the entire real time infrastructure built here Ktor WebSocket to LB Socket API, device detection, ListeningNowViewModel is the exact foundation needed for a “listen along” feature where friends can follow each other’s playback live. This is a natural product extension that reuses everything built in GSoC with minimal new infrastructure.

  2. Widget For easy access to current state : I am planning to implement Widget for Easy access to current state

Community Affinities

Tell us about the computer(s) you have available for working on your SoC project?
I will be using my Asus TUF Gaming A15 running Fedora KDE/Win 11 on dual boot. It handles Android Studio and Gradle builds comfortably.

When did you first start programming?
I started with C++ in my first semester of college for competitive programming. I then found Android development through BSOC, my college’s open source programme. Kotlin is now my primary language.

What type of music do you listen to?
Sufi and Ghazal music my most-played artists are Nusrat Fateh Ali Khan and Jagjit Singh ,some of my Mbid’s are

  • a39a2243-3f86-4c02-82f0-1200fa611d3e
  • a78b6b9a-d7ea-4f59-81df-20ddc3046f9e
  • 64d905e4-7d08-4dc7-976a-7bbcfe653f3c
  • 19f55b19-2ec3-41c3-9a8c-b55e89774ca3

What aspects of the project interest you the most?
The architectural challenge,one coherent UI. The API integrations already exist, the Spotify SDK module is already in the codebase, ExoPlayer is already in BrainzPlayer. The problem is design: how do you unify them without creating a tangled mess, in a KMP module that also works on iOS? That is the kind of problem I want to solve.

Have you contributed to other Open Source projects?
Yes, I contributed to open source earlier through my college’s open source programme BSOC (BitByte Summer of Code)

How much time do you have available, and how would you plan to use it?
I can dedicate 20-25 hours per week to the project. College exams end before the coding period begins so there will be no academic conflicts. I will keep stay active on IRC and the MetaBrainz forums to keep the mentor updated on progress and blockers.
Link to Proposal PDF

1 Like

Hi Anuj, thanks for the proposal. Good work overall.

Things to address:

  • Room DB queue persistence is underspecified. More importantly, Room lives in androidMain but the ViewModel and queue logic live in commonMain. Also we intend to support SoundCloud on both platforms which means we won’t be able access the DB while it just sits on one platform only. How does commonMain access the persisted queue? The DB should remain in commonMain (even if iOS will not be supported).

  • SoundCloud URL resolution failure path is missing. What happens when the LB server doesn’t return a SoundCloud URL for a track? Skip or fallback to YouTube intent or maybe an rrror state? The happy path is clear but the failure path needs design.

  • No UI description for the four PlaybackUiState variants.

    • What does each state actually look like in the mini-player?

    • Where does the queue appear?

    • How does the user distinguish between sources?

      Figma during community bonding is fine but the proposal should give a sense of the UI intent. Rough sketch of the whole idea should work, it need not be pixel perfect. This helps me know that we are on the same page.

  • Background behavior is unaddressed. Cold flow tied to viewModelScope means no WebSocket when nothing observes.

    • But what about when BrainzPlayer or SoundCloud is playing in the background?
    • Should Listening Now events still be processed?
  • Testing plan needs more depth. Three async input sources with priority resolution and device detection heuristics need more than a handful of unit tests. Think about mocking flow combinations, testing edge cases in resolveUiState(), queue exhaustion, Spotify-not-installed degradation.

  • “Same will be implemented for iOS” for SoundCloud. ExoPlayer is Android-only, iOS would need AVPlayer, which is non-trivial. You can always explore libraries that help you setup playback for both platforms making it simpler for your sake, while also allowing us to extend to other platforms if that library supports so in future.

1 Like

@Jasjeet Thanks for the detailed feedback, addressing each point below

Room DB Queue Persistence

I had the DB in androidMain which means commonMain has no way to access it. Moving the DB to commonMain fixes this entities, DAOs and the database class all go in commonMain so the ViewModel can access the queue directly without crossing the platform boundary. The Room KMP infrastructure is already in place on the cmp branch so this is straightforward to do.

The only part that stays platform-specific is the database builder, since the file path is different on Android and iOS. That uses expect/actual . Everything else including the queue read/write logic lives in commonMain and works on both platforms.

SoundCloud URL Resolution Failure

When the LB server doesn’t return a SoundCloud URL for a track, the queue manager skips that track, marks it as failed in the queue, and moves to the next one. The user sees a small indicator in the queue that the track was skipped rather than it silently disappearing.

YouTube fallback won’t work here because YouTube is foreground only using it would break the background queue session.

If more than 3 consecutive tracks fail to resolve, playback stops and the ViewModel moves to Idle with an error shown to the user. This handles the case where something is systematically wrong like a network issue or server problem, rather than silently skipping through the entire queue.

UI Description for PlaybackUiState Variants

The Scaffold front layer has two parts a collapsed mini player strip always visible at the bottom, and an expanded area the user pulls up to see the queue.

Idle

Mini player strip is hidden ,Nothing shown above the bottom navigation bar

BrainzPlayerActive

Mini player shows album art, track name, artist, and playback controls (play/pause/skip)
Pulling up the front layer reveals the BrainzPlayer queue below

RemotePlaybackActive

Mini player shows album art, track name, artist, and a small source badge,No playback controls since we are not controlling the player directly ,Pulling up shows the LB playlist queue with remaining tracks

ListeningNowOtherDevice

Mini player shows track name, artist, and a small “listening on another device” label
No controls, no queue since playback is happening elsewhere .Front layer cannot be expanded
The source badge on RemotePlaybackActive is how the user distinguishes between source. The “other device” label is how they distinguish remote listening from local playback.

Background Behavior

Currently the WebSocket is tied to viewModelScope, which means it closes when the UI is not visible. However, BrainzPlayer and SoundCloud run inside a foreground service that stays alive during background playback. As a result, Listening Now events stop being processed exactly when playback is most likely happening.

The fix is to move the WebSocket connection into the existing foreground service instead of viewModelScope. The service keeps the connection alive as long as playback is active, regardless of whether the UI is visible. When the user returns to the app, the ViewModel reads from the stream already maintained by the service instead of opening a new connection.

Testing Plan

Will cover the following specific cases:

resolveUiState()

Every combination of the three input flows mocked and tested ,BrainzPlayer active overrides everything ,RemotePlayback only shows when BrainzPlayer is inactive ,OtherDevice only shows when device detection returns false,All flow tests written using Turbine (already in the project

Device Detection

Match within window,Match outside window,Clock skew case,False positive where two devices play the same track within 10 seconds

Queue Exhaustion

Last track ends,ViewModel transitions to Idle cleanly

Spotify Not Installed

SpotifyAppRemote.connect() fails,State moves to Unavailable,UI degrades without crashing

SoundCloud URL Failure

Single track skip tested,Consecutive failures hitting threshold moves to Idle with error shown to user

WebSocket Reconnect

REST fallback state emits correctly before WebSocket stream resumes

SoundCloud on iOS and ExoPlayer

ExoPlayer is Android only. The fix is to abstract audio playback behind an expect/actual interface in commonMain. Android provides an ExoPlayer implementation in androidMain and iOS provides an AVPlayer implementation in iosMain. The queue logic and URL feeding stays entirely in commonMain and works on both platforms unchanged.

I will also explore KMP libraries that wrap both ExoPlayer and AVPlayer behind a single interface, which would simplify the implementation and make it easier to extend to other platforms in future.

I wanted figma rough mockups but given time left for deadline, lets just go with behaviour definition itself. Update the original proposal with this new information.

Submit the proposal to GSoC website.

1 Like