Setting up Monitoring for DigitalOcean Managed Databases with Prometheus and Grafana

Introduction

Monitoring your DigitalOcean Managed Databases is crucial for ensuring their performance, stability, and security. By implementing monitoring solutions like Prometheus and Grafana, you gain valuable insights into your database clusters’ health and performance metrics, enabling proactive management and troubleshooting. You can programmatically access your database cluster’s metrics through the metrics endpoint, providing access to over twenty times the metrics compared to what’s accessible in the Insights tab within the cloud control panel.

This tutorial outlines the steps to set up monitoring for Managed Databases on DigitalOcean (except MongoDB) using Prometheus and Grafana. We use a script here that utilizes the scrapable metrics endpoint to export logs, enabling comprehensive monitoring of your DigitalOcean Managed Databases. We’ll cover accessing the metrics endpoint, configuring Prometheus to scrape metrics, and visualizing the data in Grafana.

Prerequisites

• A DigitalOcean account with Managed Databases set up (except MongoDB).
• Basic understanding of bash scripting.
• A Prometheus Droplet deployed from DigitalOcean Marketplace.
• A Grafana Droplet deployed from DigitalOcean Marketplace
• Installed dependencies: jq
• A personal access token created with database read access. To create one for yourself, you can follow How to Create a Personal Access Token.

Preparing Prometheus Droplet

Step 1: SSH into the Prometheus Droplet.

Step 2: Download the script scrape.sh using the wget command:

wget https://solutions-files.ams3.cdn.digitaloceanspaces.com/DBaaS-Scrape-Metrics/scrape.sh

Step 3: Once the script is downloaded, ensure it has executable permissions by running:

chmod +x scrape.sh

Step 4: Execute the script scrape.sh by running:

./scrape.sh

root@prometheus:~# ./scrape.sh
Checking dependencies...
Error: jq is not installed. Please install it before running this script.

In that case, you can install jq on the Droplet by running the following commands:

sudo apt-get update && sudo apt-get install jq

If you get the below error when installing jq, then remove the lock file by running the command sudo rm /var/lib/dpkg/lock*

root@prometheus:~# sudo apt-get install jq
E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?

root@prometheus:~# sudo dpkg --configure -a
dpkg: error: dpkg frontend is locked by another process

root@prometheus:~# sudo rm /var/lib/dpkg/lock*

root@prometheus:~# sudo apt-get install jq
Reading package lists... Done
Building dependency tree       
Reading state information... Done

Run the script again after installing jq.

Step 5: Select the managed database you want to manage by entering its number.

Step 6: Enter your DigitalOcean token.

Step 7: On the Main Menu, choose the required option and select the database.

Step 8: The script will add or remove the databases for monitoring.

Step 9: Once added or removed, check whether the targets are updated on prometheus_hostname:9090

Adding databases to monitoring

Removing databases from monitoring

Preparing Grafana Droplet

Step 1: Log into the Grafana dashboard by visiting your browser’s URL Grafana_IP:3000.

Step 2: Go to Configuration > Data Sources.

Step 3: Click on Add data source.

Step 4: Search and Select Prometheus.

Step 5: Enter Name as Prometheus, URL (Prometheushostname:9090) and click “Save & Test”. If you see “Data source is working”, you have successfully added the data source. Once done, go to Create > Import.

Step 6: You can manually configure the dashboard or import the dashboard by uploading the JSON file. Some sample dashboard JSON files are below:

MySQL | PostgreSQL | Redis | Kafka

Step 7: Fill in the fields and Import.

Step 8: The Grafana dashboard is ready. Select the host and check if the metrics are visible. You can modify and edit the dashboard as needed.

Conclusion

Setting up monitoring for DigitalOcean Managed Databases using Prometheus and Grafana enhances your ability to ensure your database clusters’ performance, stability, and security.

You gain comprehensive insights into database health and performance metrics by leveraging Prometheus to scrape metrics and Grafana to visualize them.

This proactive approach allows for efficient management and timely troubleshooting, which is crucial for maintaining optimal database operations. With the outlined steps in this documentation, you can easily implement robust monitoring tailored to your specific database needs, ensuring smooth operations of your DigitalOcean Managed Databases.