Incomplete Last.fm import and wrong last import timestamp

Hi All,

I’m having trouble with importing my Last.fm data to ListenBrainz.

I ran the importer and it seemed to work. It imported 9059 pages of data in about 1.5 hours. In the end it reported a success, with the number of imported listens matching the number of scrobbles in my Last.fm profile:

Screenshot_2022-09-23_01-39-51

However, ListenBrainz listen count is not equal to the imported number (the same incorrect number is displayed in my profile page and reported by API):

$ curl -s https://api.listenbrainz.org/1/user/Alioth/listen-count | jq
{
  "payload": {
    "count": 395268
  }
}

The number of listens exported from ListenBrainz to JSON file is again different, looks like it’s sort of rounded down from the number above:

$ jq '. | length' Alioth_lb-2022-09-23.json 
395000

Finally, the latest import timestamp is wrong. It should correspond to early hours of 2022-09-23, but it is zero:

$ curl -s https://api.listenbrainz.org/1/latest-import?user_name=Alioth | jq
{
  "latest_import": 0,
  "musicbrainz_id": "Alioth"
}

I tried importing my Last.fm data from scratch several times in the last three weeks, and I got similar results with every attempt. The number of listens was different each time, but always too small. After each import I waited a few days to check if the number would become correct later, but that never happened.

My expectations are that after importing data from Last.fm:

  • The number of listens displayed in ListenBrainz profile page or returned by API should match the number of scrobbles in Last.fm;
  • the number of listens exported to JSON should match the number displayed in the profile page;
  • the latest import timestamp should not be zero.

This is obviously not the case. Could someone check why is it happening? Or maybe my expectations are wrong?

1 Like

Thanks for the report. I’ll investigate.

2 Likes

I went through open tickets in Jira, and at first glance it looks like it can be related to these two issues (although this doesn’t explain wrong last import timestamp):

Is there a plan to prioritize fixing them? If we want ListenBrainz to become more popular, having a working Last.fm importer is a must.

This has now been reported by other people as well: Importing from Last.fm doesn’t import all scrobbles

2 Likes

Exactly, for me this is holding me back from adopting Listenbrainz completely. I’m in favor of migrating from Last.fm to Listenbrainz but this is holding me back. I don’t want to abandon my accumulated listens.

Hi! Yes, there are plans to improve the existing last.fm importer. We intend to completely rework it so that it works in the background like the spotify connector. That will allow us to deal with some of these transient errors better. However, that means its not a simple fix will take some time to do.