ListenBrainz - problem with import


#1

Recently I created an account on MusicBrainz and I imported my scrobbles from Last.fm to ListenBrainz. Operation was successful and everything went OK. Then several times in the last days I repeated the operation and added a scrobble manually - every time without problems - number scrobbles from Last.fm equals with ListenBrainz.

Last Wednesday, there was a problem. Scrobbles have been added but their number from Last.fm does not match with ListenBrainz - some of them have been duplicated. For example, Last.fm has 78,351 and ListenBrainz 78,508.

Several times I tried to reset import timestamp but no effect - the problem still occurs.
Is there any way to remove all listens from ListenBrainz and import them again? Because apparently, mirror synchronization does not work.

P.S. I have to add that I have never been a user of ListenBrainz Alpha.


#2

This seems to be a known problem, https://tickets.metabrainz.org/browse/LB-152 the thing I’m not sure about is if there are duplicates in LB (shouldn’t be possible theoretically) or if it’s a problem on last.fm side, when you look at your profile, can you see any listens that are obvious duplicates?


#3

I checked all the listens until 8th September back (first time I imported) - there are no duplicates.
In the last few days I watched the number of scrobbles on last.fm and on LB and I wrote down the numbers - the number at last.fm always agrees - the problem seems to be on the LB side.

I have a few more observations:

  1. I do not know if this is important but whenever I add a scrobble manually, the message “Successfully submitted 50 listens to LB” appears. No matter if I added 6, 27 or 48 scrobbles is ALWAYS 50.

  2. After the reset, the message “Successfully submitted 78357 listens to LB” will appear - the number is ALWAYS exactly the same as last.fm. But when I close the window and go to the LB profile page, the number is inflated and duplicated.

Correct me if I’m wrong, but I imagine it works by mirroring. It seems that after the LB reset does not clean up to the end of the statistics and leaves something in its base. The number of re-imported scrobbles will always match the number at last.fm. But after going to the profile page LB is overstated (in my case about 150 listens)


#4

I know this is not the complete solution to the problem, but I guess you have to start with the reset import timestamp. Today it looks like the reset is not full - it does not clean the data completely and leaves something in the base LB.


#5

I know that is caused by Lastfm’s API inconsistency, but I have two questions:

  1. number of duplicated listens is still growing (in my case today is over 270 - few days ago was about 150). So what will be in a few months?
  2. at the beginning everything was OK and the number of scrobbles was always equal to the number of listens (I’ve imported few times). Why?

#6

Hi, I’m not exactly sure why this is happening, I’ve been kinda busy but I’ll take a look as soon as possible.


#7

Hi, thanks for your reply.
I hope you will find a solution to this problem. Fingers crossed :grinning:

And thank you for your work - ListenBrainz is a very good and needed idea.


#8

Hi,

In case it’s useful, I did an import this morning and I’ve noticed a a few specific listens that have been duplicated (compared to last.fm).

https://listenbrainz.org/user/JigsawPieces
go back to 2017-11-03T19:31:07Z and earlier, a set of tracks by Steven Wilson

On last.fm I have:
To The Bone, Nowhere Now, Nowhere Now, Pariah, The Same Asylum as Before, Refuge

On LB I have (bold for the dupes):
To The Bone, Nowhere Now, Nowhere Now, Nowhere Now, Pariah, Pariah, The Same Asylum as Before, The Same Asylum as Before, Refuge, Refuge

(I haven’t reset my timestamp, since registering - just done imports to update.)


#9

Playing around with playing music on new device that has no ListenBrainz listen submitting add on yet (Kodi). After playing some tunes and realizing I was creating a hole in my ListenBrainz history I decided to click import on website due to “will not create duplicate entries” statement. But after a quick imort process I have a bunch of dupes that arent on last.fm and the total scrobbles are off by a fair margin now. I have done this before (last summer I think it was) without duplication issues noticed but sure wen’t wrong today. Screen grabs of dupes and them not at last.fm attached. My scrobble total was the same on ListenBrainz and Last.FM before this import click I did, off by a few thousand now.


#10

Last.fm Picture (New users only one pic per post)


#11

Looks like the importer is stuck some four days ago. It don’t import anything more recent than that.


#12

Yes, that’s a known issue currently. I asked on IRC, the team is working on this. @ruaok told me “they are being accepted, but the ingestion queue is stuck.”, so the listens should not be lost.


#13

Listens seem to be getting through again, at least mine are now shown.


#14

I still have way more listens on ListenBrainz than scrobbles on Last.fm. I’ve tried to reset the import timestamp and import again, but I think the problem got even worse. The difference is 1,754 scrobbles now.


#15

@Alsweider There seems to be some issue with deduplication, see


#16

I have deleted my account and opened it again. After importing my Last.fm again, the total number of listens is doubled. So I think there are no actual double listens, it’s just that the total counter doesn’t reset to zero after deleting the account.


#17

It’s been over a year and unfortunately nothing has changed.

Today I reset timestamp and imported scrobble: the number on LB is still higher than on Last.fm (about 1860 difference). Interestingly, the total number of artists is smaller on LB.

I am almost sure that the issue is a reset, which does not completely clean the LB base.

Because the reset does not seem to work - is it possible to completely remove all scrobbles from the LB account and start again?