# AcoustID downtime
**URL:** https://community.metabrainz.org/t/acoustid-downtime/522087
**Category:** AcoustID
**Created:** [March 2, 2021, 2:36pm UTC](https://community.metabrainz.org/t/acoustid-downtime/522087 "2021-03-02T14:36:04Z")
**Posts on this page:** 19
**Page:** 1
### Author:  [@lukz](https://community.metabrainz.org/u/lukz)
#### Post date: [March 2, 2021, 2:36pm UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/1 "2021-03-02T14:36:04Z")
The AcoustID service is currently experiencing serious technical problems and will have to stay down for several day and some recent data will be lost.
If you are user of the free service, I apologize for the inconvenience and hope that I’ll be able to repair the service soon, so that you can start using it.
If you are a paying customer, supporting the open source project, you future invoices will be cancelled. When the service will be functional again, I’ll get in touch with you whether you would like to stay a customer.
Will post more updates as things get better.
---
### Author:  [@outsidecontext](https://community.metabrainz.org/u/outsidecontext)
#### Post date: [March 2, 2021, 4:07pm UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/2 "2021-03-02T16:07:24Z")
---
### Author:  [@rob](https://community.metabrainz.org/u/rob)
#### Post date: [March 2, 2021, 4:13pm UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/3 "2021-03-02T16:13:26Z")
Would a donation from MetaBrainz help? Drop a line to rob@meta… and let’s chat!
---
### Author:  [@outsidecontext](https://community.metabrainz.org/u/outsidecontext)
#### Post date: [March 4, 2021, 5:02pm UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/4 "2021-03-04T17:02:23Z")
See also the status page at [https://status.acoustid.org/](https://status.acoustid.org/)
---
### Author:  [@rob](https://community.metabrainz.org/u/rob)
#### Post date: [March 4, 2021, 5:07pm UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/5 "2021-03-04T17:07:35Z")
Well, I’ve not heard anything from ya, so I donated 500eur. I hope that helps.
---
### Author:  [@Alfg](https://community.metabrainz.org/u/Alfg)
#### Post date: [March 4, 2021, 5:38pm UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/6 "2021-03-04T17:38:45Z")
thank you
works again 😀 😀 😀 partly 🙄
---
### Author:  [@lukz](https://community.metabrainz.org/u/lukz)
#### Post date: [March 4, 2021, 5:44pm UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/7 "2021-03-04T17:44:11Z")
Sorry for not replying, Rob, I was super busy with fixing things. This is purely a technical problem, not a financial one, which is kind of how I got into the situation. So far, I was always able to fix things just by renting more servers, but that is now no longer effective and AcoustID needs bigger changes.
The service is now partially restored, but heavily rate-limited at least for the next week or so, until I manage to separate some parts of the backend.
---
### Author:  [@lukz](https://community.metabrainz.org/u/lukz)
#### Post date: [March 5, 2021, 7:15am UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/8 "2021-03-05T07:15:04Z")
I’ve increased the rate limits, as things seem more stable now, but I’m seeing nowhere near the amount of traffic I was seeing before the incident, so something must be still wrong. If the service is not working for you, please let me know and I’ll try to find out why it’s not working.
---
### Author:  [@outsidecontext](https://community.metabrainz.org/u/outsidecontext)
#### Post date: [March 5, 2021, 8:04am UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/9 "2021-03-05T08:04:01Z")
Not sure if this had already been resolved before the downtime or if it is a result of the redeployment, but the issue of sometimes randomly getting empty results is now also ok again.
---
### Author:  [@lukz](https://community.metabrainz.org/u/lukz)
#### Post date: [March 5, 2021, 3:45pm UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/10 "2021-03-05T15:45:47Z")
The service is now fully restored. You can read more about what went wrong in this blog post. More details will be posted later. You can expect some changes to the APIs, to make sure the service stays fast and a situation like this does repeat.
[https://blog.acoustid.org/posts/2021-03-05-postmortem-of-the-march-2-2021-outage/](https://blog.acoustid.org/posts/2021-03-05-postmortem-of-the-march-2-2021-outage/)
---
### Author:  [@outsidecontext](https://community.metabrainz.org/u/outsidecontext)
#### Post date: [March 6, 2021, 8:08pm UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/11 "2021-03-06T20:08:46Z")
Thanks a lot for getting this fixed and for the insights.
I actually see the positive side of this: The problem the service ran into are a direct result of its astonishing success over the last 10 years (nearly 11 really) and the continuous growth this is causing. Thanks a lot, @lukz, for the service itself and its continued support.
I am looking forward to the changes and to see the service improve and prosper. Let me know if there is anything that should be done on the Picard side.
---
### Author:  [@ApeKattQuest\_MonkeyPython](https://community.metabrainz.org/u/ApeKattQuest_MonkeyPython)
#### Post date: [March 6, 2021, 10:28pm UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/12 "2021-03-06T22:28:47Z")
I’m pretty much echoing @outsidecontext here, and acoustid is a great service. Thank you!
---
### Author:  [@sound.and.vision](https://community.metabrainz.org/u/sound.and.vision)
#### Post date: [March 7, 2021, 11:43am UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/13 "2021-03-07T11:43:59Z")
Thank-you for writing the blog post on what happened, always interesting to see the reason behind outages.
I love the AcousticID service, and hopefully the changes coming in 2021 will make things a bit more stable 😃
---
### Author:  [@Tche333](https://community.metabrainz.org/u/Tche333)
#### Post date: [March 15, 2021, 11:45am UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/14 "2021-03-15T11:45:42Z")
Thanks @outsidecontext for sharing that URL.
As service is now restored, I’ve tested that API & status URLs.
API URL: I can (quickly) search & find from (Linux) Picard
Status URL: “AcoustID services are unavailable”
From my point of view, the status webpage doesn’t seem to reflect the API status.
---
### Author:  [@Tche333](https://community.metabrainz.org/u/Tche333)
#### Post date: [March 15, 2021, 12:00pm UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/15 "2021-03-15T12:00:02Z")
Hi @lukz, thanks for your work to restore the service !
FYI
The API lookup function is working fine as the submission\_status function but I can see many of my submissions are still pending from before the technical issue.
Is it related to your explanation ? Does this mean we’d better stop new submissions until further noticed ?
> “AcoustID is now back up, but still not in ideal condition as we were having major performance problems for the last few months, which are unfortunately no longer solvable just by adding more servers”
I’ve pushed a donation at same time 🙂
Thanks for the great work to have this service still online for soooo long (seems an eternity in digital world/age) 👏
---
### Author:  [@regagain](https://community.metabrainz.org/u/regagain)
#### Post date: [March 15, 2021, 2:30pm UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/16 "2021-03-15T14:30:46Z")
I’d like to donate to the project but the link to Patreon on [https://acoustid.org/donate](https://acoustid.org/donate) doesn’t seem to work (it redirects me to the main page on [patreon.com](http://patreon.com)). Is it just me?
---
### Author:  [@sound.and.vision](https://community.metabrainz.org/u/sound.and.vision)
#### Post date: [March 20, 2021, 4:34pm UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/17 "2021-03-20T16:34:49Z")
Seems to be working, there is a PayPal badge and that takes you to the donation page
---
### Author:  [@aerozol](https://community.metabrainz.org/u/aerozol)
#### Post date: [March 20, 2021, 9:22pm UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/18 "2021-03-20T21:22:33Z")
**Patreon** link doesn’t work for me either, redirects to the home page.
---
### Author:  [@sound.and.vision](https://community.metabrainz.org/u/sound.and.vision)
#### Post date: [April 27, 2021, 8:15pm UTC](https://community.metabrainz.org/t/acoustid-downtime/522087/19 "2021-04-27T20:15:20Z")
I dont like to bring old threads back to life, but was there any further diagnosis on this?