Skip to end of metadata
Go to start of metadata

This article only applies to the CEE!

This article provides a workaround for a bug that is currently under review.

Problem

Modifying site specific global settings of the central site, can cause a dead Livestatus connection for the remote site. 

The reason for this is the order of precedence of the setting "Trusted certificate authorities for SSL" inside the "Global Settings" and the "Site specific global settings" of the central site. 

Let's go to Setup → General → Distributed Monitoring to the state of the connection:


It looks like the certificate for the remote site is not verified. If you click the shield, you will receive the message that this is already trusted.


A temporally and not recommended workaround is to trust the certificate again. This will add the certificate twice in the global settings and the Livestatus connection will go the next time again dead, when trying to save some site specific settings.

The main issue here is that you can store the certificates for the master at two different points (global and site specific global settings). The certificate will be added to the "Global Settings", if you add it through the web interface of "Distributed Monitoring". Checkmk is then running into troubles if you store additionally certificates in the site specific, as now only the site specific are considered and the certificates added through the web interface (stored in the Global Settings) are now ignored.

OMD[workshop]:~$ cat etc/check_mk/multisite.d/wato/ca-certificates.mk |wc -l
24
OMD[workshop]:~$ cat etc/check_mk/multisite.d/wato/ca-certificates_sitespecific.mk|wc -l 
4

Solution

If you run into this issue, please follow these points:

  1. Compare the "Trusted certificate authorities for SSL" inside global settings or the site specific global settings of the central site
    1. let's start at the command line:

      # Trusted certificate authorities for SSL - global settings
      OMD[workshop]:~$ cat etc/check_mk/multisite.d/wato/ca-certificates.mk |wc -l
      24
      # Trusted certificate authorities for SSL - site specific global settings
      OMD[workshop]:~$ cat etc/check_mk/multisite.d/wato/ca-certificates_sitespecific.mk|wc -l 
      4
      

      Here we see, that there is a difference between both files. It looks like we have fewer certificates configured on the site specific global settings (central site)

    2. let's check both setting on the GUI 

      Global settings

      Site specific global settings (central site)


  2. A recap: What does this mean?
    1. This means, that Checkmk knows of two certificate stores. In this example, one stores two certificates (global settings) and the other one stores three certificates (site specific global settings of the central site)
    2. If you compare both screenshots, you will notice that the certificate of the site "ws5" is missing in the site specific global settings
    3. But only one of these two stores are used. Usually, the "Site specific global settings" are superseding the "Global settings". But if you add a certificate through the web interface, the precedence will change temporarily (and obscure the problem).
    4. Every time you change the certificates in the site specific global settings of the central site, the Livestatus connection will go to status "DEAD".  At this point, Checkmk has trouble with the validation of the certificate stores
  3. What can you do to solve the problem?
    1. Decide a single store for all your certificates. Use the "Global settings" as long as there is no compelling reason, that you must use the "Site specific global settings".
    2. If you use, the "Global settings", you can use Checkmk without any restrictions.
    3. If you are forced to use the "Site specific global settings", you must not use the user interface button to add the certificate in the Certificate details, but instead copy and add any new certificate of remote instances manually to the "Site specific global settings" of your central instance.

At the moment, we're under consideration how to proceed with this issue!