Grafana OnCall Reviews & Insights

Grafana OnCall organically fit into our IT infrastructure (as it supports Prometheus and Zabbix we already had in production use), performing the role of Incident Response & Management for developers and engineers. We found this tool is really useful to collaborate different departments (in our company we are using Teams), escalate issues and provide fast and informational alerting.

One of the greatest drawback in any big companies is that different teams within an IT departments do not know what the others do. The worst case scenario is than Service Desk doesn't know where the issue resolution is and can not provide business with estimation recovery time as well as any update about the root cause of an issue. Grafana OnCall is the best tool that solved this misscolaboration in our company.

We use Grafana Cloud for observability , with Testing & Synthetics for the alerting and oncall-duty from our service team we use this for incidents to send the alerts to the dedicated duty scrum team after business working hours and the weekends and holidays. During the day business hours, incidents are handled by the responsible application scrum teams.

Setting up the Grafana OnCall configuration is from the UI is a little bit complex, you need to create an integration, from there an escalation chain ann after that you need to change the template for the notificaion. Also the documentation is not entirely clear, particularly the section on provisioning with Terraform, because production stack is provisioned with infra structure as code.