FYI—seeing packet loss this morning


#1

Noticed the site was going really slow this morning, so ran mtr.

Two mtr runs, the second one started a little after the first. The packet loss on Einstein.home is just ICMP echo response rate limiting.

To: community.metabrainz.org

 Host                                Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. Einstein.home                    96.3%   658    0.2   0.2   0.1   0.3   0.0
 2. lo0-100.WASHDC-VFTTP-315.verizon  0.0%   658    4.0  10.4   2.2 280.9  19.4
 3. B3315.WASHDC-LCR-22.verizon-gni.  0.0%   658    5.5  11.4   3.0 232.3  20.3
 4. ???
 5. 0.ae4.BR2.IAD8.ALTER.NET          0.0%   658    5.8  15.6   2.8 155.0  18.0
 6. ae17.edge1.washingtondc12.level3 88.0%   658    7.3 1136.   3.6 6185. 1883.
 7. ae-1-51.ear2.LosAngeles1.Level3. 97.0%   658  23574 23175 21825 24144 630.3
 8. DIGITAL-WES.ear2.LosAngeles1.Lev  0.0%   658   74.5  77.8  70.4 314.0  19.1
 9. xe-0-0-0-5011.core01.dwni.pin.sb  0.0%   658   74.2  79.2  70.8 297.9  18.0
10. ae1-2-dist09.dwni.pin.sbp1.digit  0.0%   658   73.4  79.4  72.7 249.6  16.1
11. ae0-17-dist10.dwni.pin.sbp1.digi  0.0%   657   74.9  78.3  71.2 200.1  13.6
12. lists-mx.musicbrainz.org         13.0%   657  135.7 114.6  71.1 592.6  67.1

To: musicbrainz.org

Host                                Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. Einstein.home                     0.0%   720    0.3   0.1   0.1   2.8   0.2
 2. lo0-100.WASHDC-VFTTP-315.verizon  0.0%   720    3.6   9.2   2.1 152.5  13.8
 3. B3315.WASHDC-LCR-22.verizon-gni.  0.0%   720    7.5   9.5   2.9 190.5  14.8
 4. ???
 5. 0.ae4.BR2.IAD8.ALTER.NET          0.0%   720   28.0  17.0   2.9 246.6  22.5
 6. ae17.edge1.washingtondc12.level3 89.3%   720    7.0  90.6   3.1 1065. 222.1
 7. ae-1-51.ear2.LosAngeles1.Level3. 98.3%   720  24262 23161 22288 24262 644.6
 8. DIGITAL-WES.ear2.LosAngeles1.Lev  0.0%   720   72.2  76.2  70.3 198.4  13.6
 9. xe-0-0-0-5011.core01.dwni.pin.sb  0.0%   720   74.6  78.5  71.7 242.9  15.3
10. ae1-2-dist09.dwni.pin.sbp1.digit  0.0%   720   73.7  78.6  72.9 284.3  17.8
11. ae0-17-dist10.dwni.pin.sbp1.digi  0.0%   720   75.1  78.9  72.8 335.7  17.6
12. stimpy.musicbrainz.org           13.9%   719   73.2 119.1  70.5 776.8  76.4

Not sure what to make of the loss (and epic latency) at 6/7, but it’s probably not a real issue since 8-11 have 0% and sane latency.


#2

Our servers are hammered by disrespectful bots, situation improved a bit since few hours. Thanks for the report.