Checkmk Knowledge Base

Space shortcuts

Page tree

This article explains how to debug special agents used by vSphere.

LAST TESTED ON CHECKMK 2.0.0P1

Basic Debugging

  1. Example with Special Agent of vSphere

    • Find out the detailed special agent command

      OMD[mysite]:~$ cmk -D <vcenter-host> | more

vcenter 
Addresses: x.x.x.x
Tags: [add_ip_addresses:add_ip_addresses_1], [address_family:ip-v4-only], [agent:special-agents], [criticality:prod], 
[ip-v4:ip-v4], [networking:lan], [piggyback:auto-piggyback], [site:nagnis_master], [snmp_ds:no-snmp], [tcp:tcp]
Labels: [cmk/vsphere_object:vm]
Host groups: check_mk
Contact groups: all
Agent mode: No Checkmk agent, all configured special agents
Type of agent: 
Program: /omd/sites/mysite/share/check_mk/agents/special/agent_vsphere -u 'user' -s 'password' -i hos
tsystem,virtualmachine,datastore,counters,licenses -P --spaces cut --snapshot_display vCenter --no-cert-check 'x.x.x.x'
Process piggyback data from /omd/sites/mysite/tmp/check_mk/piggyback/vcenter
Services:
checktype item params

      An easier way would be this command: /bin/sh -c "$(cmk -D vcenter | grep -A1 "^Type of agent:" | grep "^ Program:" | cut -f2 -d':')"

      Please note that if a line matching "^Type of agent:" followed by a line matching "^  Program:" exists more than once, the output might be messed up.

    • Check if there are options for debugging.

      OMD[mysite]:~$ /omd/sites/mysite/share/check_mk/agents/special/agent_vsphere -h

      • There are two options for debugging the request.

        --debug                       Debug mode: let Python exceptions come through

--tracefile FILENAME          Log all outgoing and incoming data into the given tracefile

    • Modify the special agent command by adding these two options

      OMD[mysite]:~$ /omd/sites/mysite/share/check_mk/agents/special/agent_vsphere  -u 'user' -s 'password' --debug --tracefile $OMD_ROOT/tmp/vcenter.out -i hostsystem,virtualmachine,datastore,counters,licenses -P --spaces cut --no-cert-check '$HOST_ADDRESS' > $OMD_ROOT/tmp/vcenter.debug

      In CMK 1.6.0, you might find the option "--snapshot_display vCenter" in your CMK -D output. If that's the case, you can include this parameter.

    • Run the special agent with no debug options to create an agent output. With this file, we can reproduce your issue.

      root@linux~# /omd/sites/mysite/share/check_mk/agents/special/agent_vsphere -u 'user' -s 'password' -i hostsystem,virtualmachine,datastore,counters,licenses -P --spaces cut --no-cert-check 'x.x.x.x' >/~tmp/agent.output

  2. Please send us all three files. Now we're able to investigate further

    1
    2
    3

    ~/tmp/vcenter.debug      # Debug Output
    ~/tmp/vcenter.out        # Tracefile
    /~tmp/agent.output       # Agent Output

Advanced Debugging Examples

Collect several agent outputs over a period of time:

export t=60; export s=0; while [ $s -le 600 ]; do echo $s; cmk -d $VSPHERE_HOST > /tmp/agent_vsphere_output.$s; let s=$s+$t; sleep $t; done


Collect several trace files over a period of time:

export t=60; export s=0; while [ $s -le 600 ]; do echo $s; ./agent_vsphere --trace /tmp/agent_vsphere_trace.$s $OTHER_COMMAND_PARAMS; let s=$s+$t; sleep $t; done