Setting up Promethaus, Grafana and Spring Boot

The fastest way to get Prometheus and Grafana up and running is by using Docker. This saves a lot of time when it comes to configuration and security, as Docker provides a solid foundation. The provided Docker images are already well preconfigured, making it very easy to get started.
If you haven’t installed Docker Compose yet, take a look at the official documentation here: https://docs.docker.com/compose/install/linux/.

To get started, simply create a new docker-compose.yml file on your server where Docker and Docker Compose are already installed:

WARNING: You should definitely not leave the password set to “admin” if the server is accessible from the outside. Otherwise, script kiddies and similar attackers may gain access to your system.

In the directory where your docker-compose.yml is located, you also need to create a file named prometheus.yml. Then add the following configuration:

Here we have defined a scrape job that accesses the service at localhost:9090, which is the Prometheus server itself. With this configuration, Prometheus loads various expressions that you can use to read metric data related to the Prometheus server.

Once the configuration is in place, you need to start the Docker containers using the following command:

If no firewall or similar restrictions are in place, Prometheus and Grafana should now be accessible.

To access Prometheus, use the following URL:
<server-ip>:9090

Since we defined a scraper in the configuration file that reads metrics from the Prometheus container, you can, for example, display the number of HTTP requests using this link:
<server-ip>:9090/graph?g0.expr=prometheus_http_requests_total&g0.tab=0&g0.display_mode=lines&g0.show_exemplars=0&g0.range_input=1h

To access Grafana, use the following URL:
<server-ip>:3000

Log in using the password defined in your docker-compose.yml file.

As you can see, Grafana is still quite empty. Before defining your own dashboard, you need to connect Prometheus as a data source. To do this, open the following URL:
<server-ip>:3000/connections/datasources/new

Select Prometheus and use the following URL:
<server-ip>:9090

Save the configuration.

In Grafana, metric data is defined and visualized in dashboards. To create a dashboard, open the following URL:
<server-ip>:3000/dashboards

Create a new dashboard by clicking “New” in the top-right corner. Then choose “Add visualization” and select Prometheus as the data source.

Now let’s take monitoring to the application level.

Imagine you have a Spring Boot application with high traffic and plenty of logs stored across different areas. In this section, I’ll show you how to set everything up using a Prometheus dependency and Promtail to ship logs to Loki. Loki acts as an interface that Grafana uses to visualize log files.

We use Maven as the build tool here, but the same approach works with Gradle.

Maven:

Add the following dependency to your pom.xml file:

<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

Gradle:

For Gradle, adjust your build.gradle file and add the following line:

implementation 'io.micrometer:micrometer-registry-prometheus'

Then rebuild your project to apply the changes.

To make the metrics accessible, you must add the following to your application.properties or application.yml:

management.endpoints.web.exposure.include=prometheus
management.endpoint.prometheus.enabled=true

This ensures that Prometheus metrics are exposed via the appropriate endpoint.

Warning: If Spring Boot Security is enabled, make sure to explicitly allow this endpoint so you can access the metrics.

Next, you need to configure the scraper in the Prometheus configuration.

Add the following section to your prometheus.yml file:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'spring-boot'
    metrics_path: "/actuator/prometheus"
    static_configs:
      - targets: ['<springboot-server-ip>:8080']
  

After applying the changes, restart the container.

If everything worked correctly, you should now find new expressions in Prometheus or Grafana, such as jvm_info.

Reading Spring Boot logs in Grafana is particularly useful because it allows you to monitor the performance and behavior of your application in real time. Centralized log visualization helps you identify and analyze issues more quickly, enabling more efficient debugging and optimization processes. Grafana lets you search logs interactively and correlate them with metrics, significantly simplifying error diagnosis. This way, you not only keep an overview of your logs but can also make well-founded decisions to improve your application.

For this setup, Docker must also be installed in order to run the Promtail Docker image on the server where Spring Boot is running. Promtail is used to ship logs to Loki. Loki can also be installed as a Docker image and is queried by Grafana, as Grafana already provides a Loki data source to visualize logs. This part is completely independent of Prometheus.

To do this, create a docker-compose.yaml file on the Spring Boot server with the following content:

version: '3.8'
services:
  promtail:
    image: grafana/promtail:latest
    container_name: promtail
    volumes:
      - ../tomcat/logs:/var/log
      - ./promtail-config.yml:/etc/promtail/promtail-config.yml
    command: -config.file=/etc/promtail/promtail-config.yml
    restart: always
  

Warning: The path ../tomcat/logs is only an example. Make sure to adjust the path to match the actual location of your log files.

Create a new file named promtail-config.yml and use the following template as its content:

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: <grafana-server-ip>/loki/api/v1/push

scrape_configs:
  - job_name: "tomcat-logs"
    static_configs:
      - targets:
          - localhost
        labels:
          job: "tomcat"
          __path__: /var/log/*.log
  

Make sure to adjust the url entry to point to your Grafana server.
You can then start the container.

Loki should be installed on the same server as Grafana. To do this, extend the docker-compose.yml on your server as follows:

version: '3.7'

services:
  prometheus:
    image: prom/prometheus
    container_name: prometheus
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    ports:
      - "9090:9090"

  grafana:
    image: grafana/grafana
    container_name: grafana
    ports:
      - "3000:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=admin

  # add loki
  loki:
    image: grafana/loki:2.9.10
    container_name: loki
    command: -config.file=/etc/loki/local-config.yaml
    restart: always
    ports:
      - "3100:3100"
  

You can now start the Loki container as well.

Navigate again to:

<server-ip>:3000/connections/datasources/new

Select Loki as the data source this time and use the following URL:

<server-ip>:3100

You can then create a new dashboard and use Loki as the data source.

And there we have it! The configuration is finally complete. You can now enjoy visualizing both logs and metric data from your Spring Boot application in Grafana. Having everything in one place makes it much easier to monitor application performance and react quickly to potential issues.

Grafana gives you a powerful way to present collected data effectively. You can create interactive dashboards that provide instant insight into the health of your application. Whether it’s performance monitoring or error analysis, the combination of Prometheus, Grafana, and Spring Boot gives you the control and visibility you need.

Have fun experimenting with metrics and logs!