Mirror mb.videolan.org cluttering duckduckgo search results

MusicBrainz
1

My web searches are more and more cluttered by mb.videolan.org MusicBrainz mirror.
Since recently.
I don’t know if you have noticed that.
I use duckduckgo: After 5 results, I see half videolan results, half MusicBrainz results.

I know I can filter them out with site:musicbrainz.org, which is more efficient anyway. But I feel it’s quite a pity.

It seems that google is not (yet?) affected by this problem.
I wonder why.

Related to:

1 Like
2

Not sure if it’s the root cause, but the robots.txt on that mirror are messed up. It looks like the server tries to execute the file as JavaScript/NodeJS and fails immediately, outputting an error message robot crawlers can’t parse.

4 Likes
3

I poked thresh in the VideoLAN IRC channel and they’ve fixed the robots.txt now, so hopefully the mirror will stop showing up in search results soon. :slight_smile:

1 Like