MS Teams Alerts for Kubernetes Cluster: Setting Up Alertmanager with Prometheus-MSTeams Helm Chart

Introduction

Integrating Alertmanager with Microsoft Teams enables you to receive alerts directly in Teams channels, facilitating swift collaboration with your team to address issues promptly. This guide will use Prometheus-MSTeams, a lightweight Go Web Server designed to bridge the gap between Prometheus Alertmanager and Microsoft Teams.

Alertmanager by itself cannot format notifications correctly for the Microsoft Teams webhook. This limitation necessitates the use of Prometheus-MSTeams, which converts the POST alert messages from Prometheus Alertmanager into a format compatible with Teams. In our setup, we will deploy Prometheus-MSTeams using a Helm chart in Kubernetes, ensuring a seamless integration and efficient alert management within your Kubernetes cluster.

Step 1: Download and Install Prometheus-MSTeams Chart

Download the chart:

helm repo add prometheus-msteams https://prometheus-msteams.github.io/prometheus-msteams/

Install the chart:

helm upgrade --install msteams prometheus-msteams/prometheus-msteams -n monitoring

After the installation, it’s crucial to verify that the pods and deployment are running correctly. You can do this by executing the following commands:

Check the status of the deployment:

kubectl get deployments -n monitoring

Check the status of the pods:

kubectl get pods -n monitoring

Check the status of the services:

kubectl get svc -n monitoring

Step 2: Create a New Channel in MS Teams

To configure the new channel, simply navigate to “Connectors,” add an “Incoming Webhook,” and specify a name of your choice.

Copy the URL and save it.

Step 3: Add Connector Configuration for Teams Channel in Prometheus-MSTeams

We need to edit the ConfigMap for Prometheus-MSTeams Configuration:

kubectl edit cm prometheus-msteams-config -n monitoring

In this step, we configure a new connector named “alertmanager-teams” in Prometheus-MSTeams, using the incoming webhook URL previously created in Microsoft Teams.

Step 4: Configure Alertmanager to Use the New Connector as Receiver

Add http://prometheus-msteams:2000/alertmanager-teams as а receiver for MS Teams.

Explanation of the URL:

The URL “http://prometheus-msteams:2000/alertmanager-teams” specifies how Alertmanager sends alerts to Prometheus-MSTeams formatted for Microsoft Teams notifications.:

http://prometheus-msteams: This is the hostname of the Prometheus-MSTeams service in your Kubernetes cluster, assuming it’s accessible via the service name prometheus-msteams.

:2000: This is the port number where Prometheus-MSTeams listens for incoming alerts, configured here as port 2000.

alertmanager-teams: This path represents the name of the connector (endpoint) in Prometheus-MSTeams where it expects to receive alerts from.

Step 5: Confirm Alertmanager is Sending Alerts to Teams Channel

After configuring Alertmanager with the new connector as a receiver, it is important to verify that alerts are properly being sent to your Microsoft Teams channel.

Conclusion

Integrating Alertmanager with Microsoft Teams through Prometheus-MSTeams ensures that alerts are formatted correctly and delivered efficiently to your Teams channels. This integration enhances your team’s ability to respond to issues promptly within your Kubernetes cluster.