I have submitted listens to both Last.fm and ListenBrainz and afterwards imported listens from Last.fm into ListenBrainz. This gave me duplicates, although both submissions have exactly the same play time, same recording_mbid and same basic tags.
I looked into the Last.fm and ListenBrainz API results, and this is what I have in Last.fm for one track:
{
"artist": {
"#text": "Nine Inch Nails",
"mbid": "b7ffd2af-418f-4be2-bdd1-22f8b48613da"
},
"name": "16 Ghosts II",
"streamable": "0",
"mbid": "e0632d22-f355-41dd-ae01-9bcd87aaacf6",
"album": {
"#text": "Ghosts I–II",
"mbid": ""
},
"url": "https://www.last.fm/music/Nine+Inch+Nails/_/16+Ghosts+II",
"image": [{
"#text": "",
"size": "small"
}, {
"#text": "",
"size": "medium"
}, {
"#text": "",
"size": "large"
}, {
"#text": "",
"size": "extralarge"
}],
"date": {
"uts": "1517300642",
"#text": "30 Jan 2018, 08:24"
}
}
And in ListenBrainz I have those two listens (I assume the top one is the Last.fm import and the bottom one the original ListenBrainz import):
{
"listened_at": 1517300642,
"recording_msid": "c1fb61b3-02d1-45d1-9c7b-10bea8f30686",
"track_metadata": {
"additional_info": {
"artist_mbids": [],
"artist_msid": "69ab35a3-c877-484a-8db9-4569010e72ed",
"artist_name": "Nine Inch Nails",
"dedup_tag": "1",
"recording_mbid": "e0632d22-f355-41dd-ae01-9bcd87aaacf6",
"release_mbid": null,
"release_msid": "918e002b-ed12-4345-ba0c-867e728150a4",
"release_name": "Ghosts I\u2013II",
"tags": [],
"track_name": "16 Ghosts II"
},
"artist_name": "Nine Inch Nails",
"track_name": "16 Ghosts II"
}
}, {
"listened_at": 1517300642,
"recording_msid": "c790d968-f37d-4ceb-b793-feba74a2f93c",
"track_metadata": {
"additional_info": {
"artist_mbids": [],
"artist_msid": "69ab35a3-c877-484a-8db9-4569010e72ed",
"artist_name": "Nine Inch Nails",
"recording_mbid": "e0632d22-f355-41dd-ae01-9bcd87aaacf6",
"release_mbid": null,
"release_msid": "918e002b-ed12-4345-ba0c-867e728150a4",
"release_name": "Ghosts I\u2013II",
"tags": [],
"track_name": "16 Ghosts II"
},
"artist_name": "Nine Inch Nails",
"track_name": "16 Ghosts II"
}
}
See how the MBIDs, listened_at and the readable tags are identical. Shouldn’t the deduplication catch this?