How to disable the rate control for local (new) SOLR search server?

[InvisibleMan78]

Thanks to @yvanzo I can actually use the newest (beta) SOLR search server.

If I enter the local IP address of the running virtual machine and the port 5000 into my official Picard v2.1.3, the returned data from my SOLR looks very good.

There is only one “limiting” thing that I would like to adjust:

D: 16:25:11,987 webservice.ratecontrol._out_of_backoff:225: ('192.168.1.61', 5000): oobackoff; delay: 1000ms -> 1000ms; slow start; window size 1.000 -> 2.000

Where can I tune my local SOLR search server NOT to rate control my local access?

[outsidecontext]

This is basically a Picard question, it is unrelated to your search server install. Picard by default rate limits all network requests. This can be changed by plugins. I had some example somewhere to not rate limit a specific local server, but can’t find it right now.

There are some threads discussing this:

Availablity of 64-bit version of Picard on Windows

That’s not it, it’s Picard’s webservice module. The default minimum time between any request is 1 second. See https://github.com/metabrainz/picard/blob/master/picard/webservice/ratecontrol.py REQUEST_DELAY_MINIMUM

New test VM available

You can change the Picard rate limiting for your host only in Picard by adding a custom plugin. See the comments on https://github.com/metabrainz/picard/blob/3a868739984e603c6c38dd713d312bb717a2d924/picard/webservice/init.py#L64

Picard session threads

Not sure what you mean, do you mean the rate limiting and requests to your private instance? The rate limiting has been overhauled completely, but the default is still 1 request per second with exceptions for AcoustId and Cover Art Archive. You can customize these limits using a simple plugin, but the code for this changed quite a lot recently, I have to look up how it works in the current stable release.

[InvisibleMan78]

Thank you @outsidecontext

Would be fantastic to have a simple entry in the picard.ini in %appdata%\Musicbrainz - valid only for local/private IP ranges like
192.168.0.0 to 192.168.255.255
172.16.0.0 to 172.31.255.255
10.0.0.0 to 10.255.255.255
With this limit, no one “shoots in his own foot” if he tries to reduce the rate limit for the official server. It just doesn’t work

And to be honest:
I still doesn’t now how to make such a plugin working Maybe you could contact me by PM if this public way is too risky?

For the record:

PICARD-784

[InvisibleMan78]

This situation is crazy.

The actual local search server would be extremely fast, but Picard tells me on a local Query search

D: 21:58:01,061 ui.mainwindow.set_statusbar_message:326: Looking up the metadata for cluster Guitar Rock - The Early 70’s: The Hard Stuff…

D: 21:58:01,066 webservice.ratecontrol.get_delay_to_next_request:114: (‘192.168.1.61’, 5000): Last request was 78287 ms ago, starting another one

D: 21:58:01,066 webservice.ratecontrol.increment_requests:134: (‘192.168.1.61’, 5000): Incrementing requests to: 1

D: 21:58:01,068 webservice.ratecontrol.decrement_requests:142: (‘192.168.1.61’, 5000): Decrementing requests to: 0

D: 21:58:01,069 webservice._handle_reply:411: Received reply for http://192.168.1.61:5000/ws/2/release?limit=25&query=artist%3A(lynyrd skynyrd%29 release%3A%28guitar rock %5C- the early 70%27s%5C%3A the hard stuff%29 tracks%3A%2818%29: HTTP 200 (OK) (CACHED)

D: 21:58:01,081 ui.mainwindow.set_statusbar_message:326: No matching releases for cluster Guitar Rock - The Early 70’s: The Hard Stuff

D: 21:58:01,082 webservice.ratecontrol._out_of_backoff:225: (‘192.168.1.61’, 5000): oobackoff; delay: 1000ms → 1000ms; slow start; window size 226.000 → 227.000

D: 21:58:02,473 ui.mainwindow.set_statusbar_message:326: Looking up the metadata for cluster Guitar Rock - The Early 70’s: The Hard Stuff…

D: 21:58:02,480 webservice.ratecontrol.get_delay_to_next_request:114: (‘192.168.1.61’, 5000): Last request was 1414 ms ago, starting another one

D: 21:58:02,481 webservice.ratecontrol.increment_requests:134: (‘192.168.1.61’, 5000): Incrementing requests to: 1

D: 21:58:02,491 webservice.ratecontrol.decrement_requests:142: (‘192.168.1.61’, 5000): Decrementing requests to: 0

D: 21:58:02,492 webservice._handle_reply:411: Received reply for http://192.168.1.61:5000/ws/2/release?limit=25&query=artist%3A(lynyrd skynyrd%29 release%3A%28guitar rock %5C- the early 70%27s%5C%3A the hard stuff%29 tracks%3A%2818%29: HTTP 200 (OK) (CACHED)

D: 21:58:02,515 ui.mainwindow.set_statusbar_message:326: No matching releases for cluster Guitar Rock - The Early 70’s: The Hard Stuff

D: 21:58:02,516 webservice.ratecontrol._out_of_backoff:225: (‘192.168.1.61’, 5000): oobackoff; delay: 1000ms → 1000ms; slow start; window size 227.000 → 228.000

D: 21:58:03,776 ui.mainwindow.set_statusbar_message:326: Looking up the metadata for cluster Guitar Rock - The Early 70’s: The Hard Stuff…

D: 21:58:03,778 webservice.ratecontrol.get_delay_to_next_request:114: (‘192.168.1.61’, 5000): Last request was 1296 ms ago, starting another one

D: 21:58:03,778 webservice.ratecontrol.increment_requests:134: (‘192.168.1.61’, 5000): Incrementing requests to: 1

D: 21:58:03,780 webservice.ratecontrol.decrement_requests:142: (‘192.168.1.61’, 5000): Decrementing requests to: 0

D: 21:58:03,780 webservice._handle_reply:411: Received reply for http://192.168.1.61:5000/ws/2/release?limit=25&query=artist%3A(lynyrd skynyrd%29 release%3A%28guitar rock %5C- the early 70%27s%5C%3A the hard stuff%29 tracks%3A%2818%29: HTTP 200 (OK) (CACHED)

D: 21:58:03,791 ui.mainwindow.set_statusbar_message:326: No matching releases for cluster Guitar Rock - The Early 70’s: The Hard Stuff

D: 21:58:03,792 webservice.ratecontrol._out_of_backoff:225: (‘192.168.1.61’, 5000): oobackoff; delay: 1000ms → 1000ms; slow start; window size 228.000 → 229.000

Even 2 minutes later:

D: 22:01:15,346 ui.mainwindow.set_statusbar_message:326: Looking up the metadata for cluster Guitar Rock - The Early 70’s: The Hard Stuff…

D: 22:01:15,348 webservice.ratecontrol.get_delay_to_next_request:114: (‘192.168.1.61’, 5000): Last request was 191569 ms ago, starting another one

D: 22:01:15,348 webservice.ratecontrol.increment_requests:134: (‘192.168.1.61’, 5000): Incrementing requests to: 1

D: 22:01:15,351 webservice.ratecontrol.decrement_requests:142: (‘192.168.1.61’, 5000): Decrementing requests to: 0

D: 22:01:15,351 webservice._handle_reply:411: Received reply for http://192.168.1.61:5000/ws/2/release?limit=25&query=artist%3A(lynyrd skynyrd%29 release%3A%28guitar rock %5C- the early 70%27s%5C%3A the hard stuff%29 tracks%3A%2818%29: HTTP 200 (OK) (CACHED)

D: 22:01:15,363 ui.mainwindow.set_statusbar_message:326: No matching releases for cluster Guitar Rock - The Early 70’s: The Hard Stuff

D: 22:01:15,364 webservice.ratecontrol._out_of_backoff:225: (‘192.168.1.61’, 5000): oobackoff; delay: 1000ms → 1000ms; slow start; window size 229.000 → 230.000

Therefore I have to use the “Analyze”-Button, hammer the external AcoustID-Server to get a release ID and the details.

@outsidecontext I would be very happy if you could help me to get around this local rate limiting thing. I really only use it on my LOCAL setup.

[outsidecontext]

I think this should work. Create a file myratecontrol.py (or whatever you want to name it) in the Picard plugin folder with the following content:

PLUGIN_NAME = u'Customize rate control for own MB server'
PLUGIN_VERSION = '0.1'
PLUGIN_API_VERSIONS = ["2.0"]

from picard.webservice import ratecontrol
# Hostname and port of your MB server, 0 is no delay
ratecontrol.set_minimum_delay(('192.168.1.61', 5000), 0) 

Restart Picard for this to take affect.

[InvisibleMan78]

Thank you VERY MUCH!

You are my