Skip to end of metadata
Go to start of metadata

Basic Debugging

  1. Example with Special Agent of vSphere
    • Find out the detailed special agent command

      OMD[<sitename>]:~$ cmk -D <vcenter-host> | more

      Addresses: x.x.x.x
      Tags: [add_ip_addresses:add_ip_addresses_1], [address_family:ip-v4-only], [agent:special-agents], [criticality:prod],
      [ip-v4:ip-v4], [networking:lan], [piggyback:auto-piggyback], [site:nagnis_master], [snmp_ds:no-snmp], [tcp:tcp]
      Labels: [cmk/vsphere_object:vm]
      Host groups: check_mk
      Contact groups: all
      Agent mode: No Checkmk agent, all configured special agents
      Type of agent:
      Program: /omd/sites/<sitename>/share/check_mk/agents/special/agent_vsphere -u 'user' -s 'password' -i hos
      tsystem,virtualmachine,datastore,counters,licenses -P --spaces cut --snapshot_display vCenter --no-cert-check 'x.x.x.x'
      Process piggyback data from /omd/sites/<sitename>/tmp/check_mk/piggyback/vcenter
      checktype item params

      An easier way would be this command: /bin/sh -c "$(cmk -D vcenter | grep -A1 "^Type of agent:" | grep "^ Program:" | cut -f2 -d':')"

      Please note that if a line matching "^Type of agent:" followed by a line matching "^  Program:" exists more than once, then the output might be messed up.

    • Check if there are options for debugging

      OMD[<sitename>]:~$ /omd/sites/<sitename>/share/check_mk/agents/special/agent_vsphere -h
      • There are two options for debugging the request

        --debug                       Debug mode: let Python exceptions come through
        --tracefile FILENAME          Log all outgoing and incoming data into the given tracefile
    • Modify the special agent command by adding these two options

      OMD[<sitename>]:~$ /omd/sites/nagnis_master/share/check_mk/agents/special/agent_vsphere  -u 'user' -s 'password' --debug --tracefile ~/tmp/vcenter.out -i hostsystem,virtualmachine,datastore,counters,licenses -P --spaces cut --no-cert-check 'x.x.x.x' > ~/tmp/vcenter.debug

      In CMK 1.6.0 you might find the option "--snapshot_display vCenter" in your CMK -D output. If that's the case, you can include this parameter.

    • Run the special agent with no debbug options, to create an agent output. With this file we can reproduce your issue

      /omd/sites/<sitename>/share/check_mk/agents/special/agent_vsphere -u 'user' -s 'password' -i hostsystem,virtualmachine,datastore,counters,licenses -P --spaces cut --no-cert-check 'x.x.x.x' >/~tmp/agent.output
  2. Please send us all three files. Now we're able to investigate further


    ~/tmp/vcenter.debug      # Debug Output
    ~/tmp/vcenter.out        # Tracefile
    /~tmp/agent.output       # Agent Output

Advanced Debugging Examples

Collect several agent outputs over a period of time:

export t=60; export s=0; while [ $s -le 600 ]; do echo $s; cmk -d $VSPHERE_HOST > /tmp/agent_vsphere_output.$s; let s=$s+$t; sleep $t; done

Collect several trace files over a period of time:

export t=60; export s=0; while [ $s -le 600 ]; do echo $s; ./agent_vsphere --trace /tmp/agent_vsphere_trace.$s $OTHER_COMMAND_PARAMS; let s=$s+$t; sleep $t; done