Discussion about this post

User's avatar
Raviraj Achar's avatar

Incidents are a great way to learn and improve! Sounds like manual remediation for multiple days was a miss but I am sure it left a mark on you :)

Expand full comment
Anjana Patil's avatar

Thanks for sharing your story. We have a 3rd party api to draw a chart in an email. So when the schedule task sent the email, emails went with no chart. Of course this didn’t block operations but the business value was lost. The logs reported 429 failure code too.

I wrote directly to the api provider to know their rate limit. I would have even called them. 😀

In another similar incident, api provider black-listed us and that was some fun.

Expand full comment
8 more comments...

No posts