Customizable music recommendations: a framework where users define 'similar'

Hi everyone,

I’m Tommaso, a data scientist/developer based in Italy. I’m building an open-source framework for music recommendation, and I think it could be a good fit for ListenBrainz.
I’d like to know if it’s something that could be useful here.

The idea in short: it’s a framework where the user (and developers) can define what “similar” means. Technically it works in two steps:

  1. Fuse multiple models (audio, lyrics, tags, etc.) into a single embedding space using pretrained encoders (CLAP, SBERT, …). In practice, that means having one system that contains all this information in one place.

  2. Fine-tune these embeddings with contrastive learning, where positive/negative pairs are generated from the user’s rule. In practice, it means that it will be possible to take out of the previous system any information desired and create recommendation systems based on genre, dj transitions, mood, etc… or a mix of them. Some recommendation algorithms will be created in advance, but the framework will also give the possibility to developer and users to create their own.

I think this aligns with what ListenBrainz already does, giving people more ownership over their listening data and extending it to how music is discovered, rather than using closed recommendation algorithms that platforms give them.

ListenBrainz data would also integrate perfectly, since you have listening history and co-occurrences from users, data that only a platform like this has.

I would like to hear if any of this interests you. In that case, I’m happy to share more of course.

Cheers,

Tommaso