Notes from #MetaBrainz Meeting 2023-06-26

[aerozol]

MetaBrainz Meeting 2023-06-26

Meeting start: IRC Logs for #metabrainz | MetaBrainz Chatlogs
Meeting wiki page: MetaBrainz Meeting - MusicBrainz Wiki

Agenda

• Reviews
• LLM/‘AI’ code of conduct/guidelines

Last meeting: 2023-06-19
Next meeting: 2023-07-03

@reosarevok: Welcome to How is it Monday again and Why is June Ending

Reviews

@snaekboi

[mailed in] Last week was quite productive!
So much so that I am going to bunk this meet and go complete the sleep I sacrificed!

• I was able to setup musicbrainz-docker locally to setup tests, which was a challenge in itself. (super धन्यवाद to bitmap and atj)
• Had the opportunity to learn a lot of new stuff about Docker, Python Packages, postgres, and testing!
• After a few discussions and I was left confused af and decided to make a “problem tree” to visualize the issues - which unfortunately happens to be growing faster than bamboo.
• However, if we keep up this level of quality, I am positive the AreaBot would come out better than we envisioned! Complete with some generic improvements to the MusicBrainz-Bot (potentially as a complete python package as opposed to just a repository), as well as a new stable AreaBot with better tests, CI/CD, and deployment using docker
• Looking forward to some PR reviews soon, to be able to move ahead with some of the other caveats in the testing setup.

@aerozol

[mailed in] I mailed this to reo last week, so it didn’t get read out at the last meeting, because he is a slacker, but I will be on holiday from 7 - 27 July.
I will still have my laptop and will do the usual bits and pieces, but if you have something bigger for me to do let me know ASAP, and let’s get it done.
Last week I polished off some new MB icons with reo, might try to do some more before I go, but those are the main ones that have bugged me. Preview the new ones here: MBS-13130 [Update pencil/edit icon], MBS-12942 [Indicate pending edits for an entity in the relationship editor], MBS-12951 [Make (+) and (x) icons match]
I spent a while discussing and pencilling out possible guidelines for LLM/’AI’ use in MeB, which you can all have fun arguing about later this meeting.
Quite a bit of moderation on CB (very much related to the above)
Worked on some app mockups with jasje and akshaaatt, looking forward to doing more.
Made some fun ’Not made with AI’ stickers for mayhem’s current LB project, LB-1304 [Create button that says “Made with Postgresql, not AI!”]. Can be used until we replace mayhem with a LLM.
Made mockups for Arsh’s Artist Similarity LB page. I think I’ve come up with something that looks nice in any colour combo, but I welcome any feedback.
Disclaimer: arsh will work towards the above mockup, but some of it might not be feasible

@catquest

well Im on vacation so not much
but I’m going to do some instrument images (working on that now reo infact)

@zas

last week, we received new server (jimmy), it will replace one of floyd or pink soon
I reworked a tool called mbstats (which is used to generate statistics about our websites in near realtime)
also worked on Picard issues recently reported (after beta2)
apart that, usual maintenance, supervision, edits, user support

@bitmap

last week I was mostly working on configuration to run a pg standby container on jimmy. I still need to set up ssh keys in the repo so that repmgr works
otherwise I did a little code review and tried to help out Pratha-Fish with docker
I also started a branch for upgrading Flow in musicbrainz-server, since we’re behind again and they added more useful features

@yvanzo

Since last week I resumed working on the translation platform,
more particularly documenting the state of internationalization.
Weblate is mostly set up but still in experimental phase.
Otherwise some tickets triage and support.

@mr_monkey

Last week I was off work, recovering from a very nice wedding and having a very little honeymoon in Girona.
Barely found a half hour to look at GSOC projects
Now I’m back with a vengeance.

@akshaaatt

We made a prod release of the LB app!
I will post a blog soon about it
Other than that I’ve been reviewing PRs. jasje has been going with amazing work
Also, theflash_ has been working on our iOS app and it’s coming out great!
Hopefully we can release it on TestFlight in a couple of months or so
Other than that, I’ve been learning things as usual.

@rob

last week was the usual BS. also bought a plane ticket for akshaaatt, preparing for the summit.
then I spent a lot of time working on LB radio, which now has a better parser thanks to the nudge from atj – the code for the new parser is sexy – it took me a day to learn and implement it, but there are far fewer bugs now.
https://datasets.listenbrainz.org/lb-radio
is anyone wants to try.
see note at the top for syntax hints.
but you can do:
artist:u2:2 tag:(pop, hip hop):1:or
that will give you two interleaved streams from U2 and either tags pop or hip hop. the U2 stream is weighted 2 and the tag is weighted 1, so you’ll get twice as much U2 than tag radio.
its been fun to play with. : )

@kellnerd

During the last week I’ve jumped through various hoops to get the old importer project from 2018 to run.
Among them a bit of conversion from flow types to TypeScript, changing the babel setup to ouput ESM instead of CJS modules (because that was required for the newer versions of a dependency) and migrating to the latest, error-free version of a logging library.
All of that only to realize that the producer script just logs a few lines before it idles indefinitely : (
To ease debugging of that mess, I wanted to save the separate transpilation step by using a command like ts-node which directly executes TypeScript, only to find out that it’s no longer working with v20 (current) of Node.js…
Neither did babel-node.
Today I’ve finally given up on that and with the help of monkey I managed to get the project running in Deno, which supports TS out of the box.
Spoiler: It still doesn’t work as expected, but that was already the case before, and now development iterations are a lot smoother.
At least I had some fun with another Deno project over the weekend
snap_storage@v0.2.0 | Deno is kind of a cache for API responses which will be used in Harmony, my MB importer, soon : )

@lucifer

last week i worked on SoundCloud player integration and popularity datsets.
also discussed pixelpenguin’s project implementation

@reosarevok

Last week I was half-off so I didn’t work much
I’ve mostly looked into genre requests and new genres from RYM
And I wrote some tests and fixed some issues with my PR to let admins see the history of edit note changes for a note
and I discussed LLMs, to come at a later step today

@Vishalftw

I added couple changes for Pydantic Support in Data set hostler project (PR)

@vardansaini

I have the Apple Music player integration draft PR opened .
I have to fix a bug where the preview starts even when account is not connected
I had a minor hiccup with type handling as the the library was not working but I have a custom file where I used stuff from that library and now it is working
I will we be adding the preview video soon

@jasjeet

last week I enjoyed a wedding and did work side by side
im at that point that its much more thinking than writing code itself which i unexpected like
beta for search user is ready and undergoing testing
Just made the follow button
and im really proud of it
really instantaneous and its updated even before app makes api call
im thinking of making it more reusable and will work forward doing that
plus I made an error system to cover all edge cases as well
adhering to minimum standards

@ShivamAwasthi

last week I worked on finishing up the Admin Panel PR, and finally it has been merged and deployed on test.bb (Log in - MusicBrainz)
I also started working on creating an Admin Logs for BB

@atj

last week i continued working on implementing ansible roles for consul-template and openresty, made some pretty good progress
also worked with zas to install a new server (jimmy) with root on ZFS, so bitmap can test performance
did a bit of investigation into the weird issues on rudi
discovered some bizarre corruption in consul, so zas got hetzner to replace the motherboard, CPU and RAM
rudi is now a bit of a “Trigger’s broom” (enjoy that obscure british TV reference)
i’m on holiday from the 30th until the 8th, so I won’t be around for next weeks meeting

@riksucks

The week is mostly uneventful, but rn setting up vscode_'s branch so that I can review it better.

LLM/‘AI’ code of conduct/guidelines

@aerozol

[via a proxy called @reosarevok]
This has come up because we have had a mini-flood of LLM written reviews in CB in the last few days. One of the reviewers claims the reviews are mainly his work, but it is impossible to see how much input the user has had into the output. Though I would say they have at substantial chatGPT content (when compared to reviews I had chatGPT write for the same artists/albums). So, we have a decision to make.

With this having come up in the forums as well this year (example and example) we could consider trying to cover all our projects instead of just CB.

Two options: Cover all our projects in the MeB CoC (Code of Conduct - MetaBrainz Foundation). Or just address the CB reviews with a update to the CB Guidelines for now (About the project - CritiqueBrainz)

I have put a set of drafts for each below, with three options: A. No LLM. B. Allow some LLM. C. Allow LLM, with a disclaimer.

NB: These drafts assume that we are happy to allow tracking of LLM data (e.g. LLM composed albums) in the database. [reo’s aside: I can’t see any other choice than allowing it if we want to document all music]

Drafts for MetaBrainz CoC: [that is, if we decide to set rules for LLM stuff globally]

A. 10. Do not add LLM (‘AI’) text, including in community chat and notes. Note that we do allow the tracking of LLM data in our databases, for instance LLM generated albums.

B. 10. Do not add LLM (‘AI’) text, including in community chat and notes. You may wish to use LLM as a tool, but be aware that obviously LLM generated content will be removed. Note that we do allow the tracking of LLM data in our databases, for instance LLM generated albums.

C. 10. Do not add LLM (‘AI’) text, including in community chat and notes, without adding a disclaimer, e.g. ‘Post written/revised by chatGPT’. Note that we do allow the tracking of LLM data in our databases, for instance LLM generated albums.

Drafts for CB guidelines: [that is, if we decide to do CB only]

A. “Your submissions must be your own original work. Do not submit content that you do not hold the copyright to, or is not your own, including LLM (‘AI’) generated content.”

B. “Your submissions must be your own original work. Do not submit content that you do not hold the copyright to, or is not your own, including primarily LLM (‘AI’) generated content. You may wish to use LLM as a tool, but be aware that obviously LLM generated content will be removed.”

C. “Your submissions must be your own original work. Do not submit content that you do not hold the copyright to, or is not your own. Primarily LLM (‘AI’) generated content must have a clear disclaimer at the top, e.g. ‘Review written/revised by chatGPT’”.

Discussion:

This is a summary only! In the order that these comments were brought up. Please read the chat logs for the full discussion.

• the semantics of using the term ‘LLM’. Is it a future proof term?
• do we need to cover other stuff, like AI that isn’t a LLM
• @rob: “…I think we should limit it to CB for now. I feel that applying this rule to everything is a bit overreaching when we dont understand how it might impact non CB projects.”
• @mr_monkey: “I think AI-assisted writing is an inevitable reality of the near future. Especially relevant for people who want to write in english (for reach) but who are not native speakers. IMO that’s acceptable, so I’m personally more tempted by options C”
• @reosarevok: “We do have issues in the forums with posts that don’t sound like a real human wrote them” “But we can have that separately in the forum rules”
• @reosarevok: “We also had a super long, obviously LLM generated edit note in MB recently”
• discussion around whether we should be using the term AI or LLM [@aerozol note, at the time of this post: I’m not sure if people noticed that I used both in the draft CoC/guidelines - LLM/‘AI’]
• @atj: Annotation - MusicBrainz - “You should never add copyrighted content copied from other resources, be they online or printed.”
• @reosarevok: “Yeah, what atj brings up is another issue - who does even own the copyright?”
• @yvanzo: “But I agree with the general draft otherwise: Do not submit AI-generated content without adding a clear disclaimer.”
• discussion around whether translating text falls under ‘primarily written by LLM’, or it’s the same as using Google Translate
• @yvanzo: I think that the upcoming EU regulation goes in the same direction: add watermarking to AI-generated images.
• @atj: i feel that this opens up MeB to legal risks, obvious mayhem can speak to that, but it seems somewhat similar to the CAA situation
• @yvanzo: Even if using Google Translate, it would be sane to mention the translation tool being used.
• @rob is asked if he could ask MeB’s lawyers if anyone knows what to make of the legal issue, and he already knows the answer: “the answer is: no one does. its all too early to tell.”
• @atj: well in that case i’d say blanket ban and review in a year or something, but it depends on the attitude to risk
• @reosarevok: I’d personally prefer a ban but I can see the point of allowing it for people who don’t have great English or whatnot (I’d much rather they posted in their own language for us to translate tbh since they might not know the bots are changing their meaning, but)
• @reosarevok: I think it 100% needs to be disclaimed and at least that much could be part of the general CoC
• @yvanzo: Ideally I would vote A but I’m not sure we can enforce it, so C would be at least something.

No conclusion, but it’s also mentioned that it’s a lot to take in a meeting session with no prep. It’s agreed to revisit the topic next week, and aerozol is (politely) volunteered to make a forum post and collect community feedback.

[this discussion continues a bit after meeting end]

absent: arsh (1)

[aerozol]

Please leave your comments re LLM/‘AI’ policy in this very special thread:

LLM/‘AI’ Code of Conduct/Guidelines

This topic was raised at the last MetaBrainz meeting, 2023-06-26. This thread will duplicated content from the meeting summary - please keep your discussions on the topic to this thread. Summary This topic will be brought up again at the next meeting (2023-07-03), and we are seeking community input before then. There have been instances of LLM/‘AI’/chatGPT written and/or assisted content in forum posts, CritiqueBrainz reviews, and MusicBrainz annotations. Three examples of how we can deal …

[jesus2099]

@aerozol

AI or LLM

What does it mean, in full text?
AI must be artificial intelligence, but what is LLM?

[elomatreb]

“Large Language Model”