Page History

If you're using Checkmk version 2.1, use data source release v2.0.3.

Checkmk 2.1 uses Web API, while version 2.2+ uses REST API. Upgrading to version 2.2+ and using data source v2.0.3 may cause issues.

To avoid this, upgrade the data source to v3.0.0.

Data sources can be downloaded from: https://github.com/tribe29/grafana-checkmk-datasource/releases

Web vs Rest API

Checkmk version 2.1 and below use the Web API while 2.2 and greater uses REST API.

You can verify which API Checkmk uses by going to SETUP > GLOBAL SETTINGS > and searching for API at the top. 

Web API Testing (checkmk 2.0 & 2.1)

If testing the Web API via Grafana is needed, you will first want to open the Network Analyze with your internet browser. We can test the Web API using Curl, but we will need to form the API request first.

Start the Network Monitor

• Ctrl + Shift + I
• For Firefox follow this manual: https://developer.mozilla.org/en-US/docs/Tools/Network_Monitor
• For Google Chrome follow this manual: https://developer.chrome.com/docs/devtools/network/

After you open the Network Analyzer tool, you will want to:

1. Click on "Network" within the top bar
2. Click on "Save & Test" on the Grafana Data Source page
3. Select the "webapi.py" response on the left sidebar of the Inspector page
4. Click on "Headers" on the response bar
5. Copy the "Request URL" line to your text editor so adjustments can be made. Copy everything from HTTP:// through webapi.py?
   1. ie: "http://localhost:3000/api/datasources/proxy/uid/BZ7iqAa4z/cmk/check_mk/webapi.py?"
   2. This provides us with the Unique ID "BZ7iqAa4z" which will be different on every installation. 

• To form your API request, you will need several pieces of information that will be required.
  1. Grafana Username & Password
  2. URL Copied from Internet Inspector
  3. Checkmk Automation User & Password
  4. API commands for the information that you are trying to query
     1. Checkmk Web API Reference Guide

In the following example, we will make an API request to list all of the known hosts currently added to our Checkmk instance.

bash-5.1# curl "http://grafanaUsername:grafanaPassword@localhost:3000/api/datasources/proxy/uid/BZ7iqAa4z/cmk/check_mk/webapi.py?action=get_host_names&_username=checkmkAutomationUser&_secret=checkmkAutomationSecret&output_format=pythyon" 
{"result_code": 0, "result": ["host001.domain.com", "host002.domain.com", "host003.domain.com", "host004.domain.com"]}

• To break this command down, we used the following:
  1. Grafana Username & Password
     • username:password
  2. URL Copied from Internet Inspector
     • @localhost:3000/api/datasources/proxy/uid/BZ7iqAa4z/cmk/check_mk/webapi.py?
  3. API Action or Command
     • ?action=get_host_names 
       • Checkmk Web API Reference Guide
  4. Checkmk Username & Password
     • &_username=checkmkAutomationUser&_secret=checkmkAutomationSecret
  5. API Output (Python or JSON) 
     •  &output_format=pythyon
       • More information can be found in Section 12 of this document.

Visual understanding of the above API command:

REST API Testing (checkmk 2.2 & above)

If testing the Web API via Grafana is needed, you will first want to open the Network Analyze with your internet browser. We can test the Web API using Curl, but we will need to form the API request first.

Start the Network Monitor

• Ctrl + Shift + I
• For Firefox follow this manual: https://developer.mozilla.org/en-US/docs/Tools/Network_Monitor
• For Google Chrome follow this manual: https://developer.chrome.com/docs/devtools/network/

After you open the Network Analyzer tool, you will want to:

1. Click on "Network" within the top bar
2. Click on "Save & Test" on the Grafana Data Source page
3. Select the "Version" response on the left sidebar of the Inspector page
4. Click on "Headers" on the response bar
5. Copy the "Request URL" line to your text editor so adjustments can be made. Copy everything from HTTP:// through /1.0/
   1. ie: "http://localhost:3000/api/datasources/proxy/uid/BZ7iqAa4z/rest/check_mk/api/1.0/"
   2. This provides us with the Unique ID "BZ7iqAa4z" which will be different on every installation. 

• To form your API request, you will need several pieces of information that will be required.
  1. Grafana Username & Password
  2. URL Copied from Internet Inspector
  3. API commands for the information that you are trying to query
     1. Checkmk REST API Reference Guide

In the following example, we will make an API request to list all of the known hosts currently added to our Checkmk instance.

bash-5.1# curl -X 'GET' 'http://grafanaUsername:grafanaPassword@localhost:3000/api/datasources/proxy/uid/BZ7iqAa4z/rest/check_mk/api/1.0/domain-types/host_config/collections/all?effective_attributes=false' -H 'accept: application/json'

{
  "links": [
    {
      "domainType": "link",
      "rel": "self",
      "href": "http://localhost/sitename/check_mk/api/1.0/domain-types/host_config/collections/all",
      "method": "GET",
      "type": "application/json"
    }
  ],
  "id": "host",
  "domainType": "host_config",
  "value": [
    {
      "links": [
        {
          "domainType": "link",
          "rel": "self",
          "href": "http://localhost/sitename/check_mk/api/1.0/objects/host_config/host001_domain",
          "method": "GET",
          "type": "application/json"
        },
...full response truncated

• To break this command down, we used the following:
  1. Grafana Username & Password
     • username:password
  2. URL Copied from Internet Inspector
     • @localhost:3000/api/datasources/proxy/uid/BZ7iqAa4z/rest/check_mk/api/1.0/
  3. API Action or Command
     • domain-types/host_config/collections/all?effective_attributes=false' 
       • Checkmk REST API Reference Guide
       • A very useful resource can be found within the help documentation on your Checkmk 2.2+ site
         • 
  4. API Output 
     • -H 'accept: application/json'

Visual understanding of the above API command:

Checkmk 2.1 using Data Source v2.0.3

With this example, you can see that the Data Source is saved and is working.

Checkmk 2.1 using Data Source v3.0.0

Fortunately, data source v3.0.0 is backward compatible with Checkmk 2.1. To use it, when configuring Checkmk in Grafana, change the version from ">= 2.2" to "< 2.2". See the example below:

Checkmk 2.2 using Data Source v2.0.3 (not supported)

Using this combination produces a "Could not read API response" error.

Checkmk 2.2 using Data Source v3.0.0 (recommended)

Recommended configuration for Checkmk 2.2.

SSL Configuration and Testing

During the configuration process, you may encounter errors related to a Bad Gateway. SSL issues between Checkmk and Grafana typically cause this. To prevent this, verifying that SSL communication is functioning correctly between the two sites is recommended.

Curl can be used to help troubleshoot SSL issues. If SSL works correctly, Curl should respond like the one below. This should be run on the Grafana server that you're using. 

bash-5.1# curl https://localhost/checkmksite/

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="http://127.0.0.1:5008/checkmksite/check_mk/">here</a>.</p>
</body></html>

An unsuccessful attempt may look like the following:

bash-5.1# curl https://localhost/checkmksite/ 
curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

If you are using a Self Signed Certificate for your Checkmk server and receive an error of "Could not read  API response" like the below: 

You may need to copy your self-signed .crt file to /usr/local/share/ca-certificates/ and run update-ca-certificates on the Grafana server.

If you have no SSL certificates installed, you will want to review the Securing the web interface with HTTPS document to create and install your certificates.