Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Info

This article helps debug SroreOnce errors that can occur while using Checkmk.

Status
colourGreen
titleLAST TESTED ON CHECKMK 2.0.0P1

Table of Contents

Problem

The storeonce is agent is crashing with  the following message

Code Block
languagebash
themeRDark
<<<storeonce4x_d2d_services:sep(0)>>>
Traceback (most recent call last):
  File "/omd/sites/mysite/lib/python3/requests_oauthlib/oauth2_session.py", line 477, in request
    url, headers, data = self._client.add_token(
  File "/omd/sites/mysite/lib/python3/oauthlib/oauth2/rfc6749/clients/base.py", line 198, in add_token
    raise TokenExpiredError()
oauthlib.oauth2.rfc6749.errors.TokenExpiredError: (token_expired)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "share/check_mk/agents/special/agent_storeonce4x", line 10, in <module>
    main()
  File "/omd/sites/mysite/lib/python3/cmk/special_agents/agent_storeonce4x.py", line 260, in main
    special_agent_main(parse_arguments, agent_storeonce4x_main)
  File "/omd/sites/mysite/lib/python3/cmk/special_agents/utils/agent_common.py", line 159, in special_agent_main
    _special_agent_main_core(parse_arguments, main_fn, sys.argv[1:])
  File "/omd/sites/mysite/lib/python3/cmk/special_agents/utils/agent_common.py", line 141, in _special_agent_main_core
    main_fn(args)
  File "/omd/sites/mysite/lib/python3/cmk/special_agents/agent_storeonce4x.py", line 251, in agent_storeonce4x_main
    writer.append_json(function(oauth_session))
  File "/omd/sites/mysite/lib/python3/cmk/special_agents/utils/agent_common.py", line 51, in append_json
    for l in data:
  File "/omd/sites/mysite/lib/python3/cmk/special_agents/agent_storeonce4x.py", line 154, in handler_simple
    yield from (requester.get(uri) for uri in uris)
  File "/omd/sites/mysite/lib/python3/cmk/special_agents/agent_storeonce4x.py", line 154, in <genexpr>
    yield from (requester.get(uri) for uri in uris)
  File "/omd/sites/mysite/lib/python3/cmk/special_agents/agent_storeonce4x.py", line 142, in get
    resp = self._oauth_session.request(
  File "/omd/sites/mysite/lib/python3/requests_oauthlib/oauth2_session.py", line 496, in request
    token = self.refresh_token(
  File "/omd/sites/mysite/lib/python3/requests_oauthlib/oauth2_session.py", line 446, in refresh_token
    self.token = self._client.parse_request_body_response(r.text, scope=self.scope)
  File "/omd/sites/mysite/lib/python3/oauthlib/oauth2/rfc6749/clients/base.py", line 421, in parse_request_body_response
    self.token = parse_token_response(body, scope=scope)
  File "/omd/sites/mysite/lib/python3/oauthlib/oauth2/rfc6749/parameters.py", line 431, in parse_token_response
    validate_token_parameters(params)
  File "/omd/sites/mysite/lib/python3/oauthlib/oauth2/rfc6749/parameters.py", line 441, in validate_token_parameters
    raise MissingTokenError(description="Missing access token parameter.")
oauthlib.oauth2.rfc6749.errors.MissingTokenError: (missing_token) Missing access token parameter.

Solution

  1. Example with Special Agent of Storeonce4x

    • Find out the detailed special agent command (Type of agent column)

      Code Block
      languagebash
      themeRDark
      OMD[mysite]:~$ cmk -D hostname


      Note

      an easier way would be this command: /bin/sh -c "$(cmk -D k8s | grep -A1 "^Type of agent:" | grep "Program:" | cut -f2 -d':')"

      Please note that if a line matching "^Type of agent:" followed by a line matching "^  Program:" exists more than once, then the output might be messed up.


    • Check if there are some options for debugging

      Code Block
      languagebash
      themeRDark
      OMD[mysite]:~$ ~/share/check_mk/agents/special/agent_storeonce4x -h


      • There are three options for debugging the request

        Code Block
        languagebash
        themeRDark
        --debug, -d           Enable debug mode (keep some exceptions unhandled)
          --verbose, -v
          --vcrtrace TRACEFILE, --tracefile TRACEFILE
                                    If this flag is set to a TRACEFILE that does not exist yet, it will be created and
                                    all requests the program sends and their corresponding answers will be recorded in said file.
                                    If the file already exists, no requests are sent to the server, but the responses will be
                                    replayed from the tracefile. 
        
        



    • Modify the special agent command by adding these three options

      Code Block
      languagebash
      themeRDark
      OMD[mysite]:~$ ~/share/check_mk/agents/special/agent_storeonce4x <OTHER ARGUMENTS> --debug -v --vcrtrace ~/tmp/vcrtrace.txt 2>1 ~/tmp/storeonce4x_with_debug.txt


    • Run the special agent with no debug options to create an agent output. With this file, we can reproduce your issue

      Code Block
      languagebash
      themeRDark
      OMD[mysite]:~$ /omd/sites/mysite/share/check_mk/agents/special/agent_kube <OTHER ARGUMENTS> > ~/tmp/k8s_agent_output.txt


  2. Rename the token file
    the storeonce4x special agent is using username/password for authentication. After the successful login, we obtain the access token. The access token is used for future REST requests. If you want to read more, you can check this out: https://hewlettpackard.github.io/storeonce-rest/#Authentication

    • We save the token file inside the site in

      Code Block
      languagebash
      themeRDark
      ~/tmp/check_mk/special_agents/agent_storeonce4x/<hostname>_oAuthToken.json


    • rename the file to 

      Code Block
      languagebash
      themeRDark
      ~/tmp/check_mk/special_agents/agent_storeonce4x/<hostname>_oAuthToken.json.back


    • run the special agent again

Content by Label
showLabelsfalse
max5
spacesCON
showSpacefalse
sortmodified
reversetrue
typepage
cqllabel in ("special_agent","agent","troubleshooting") and type = "page" and space = "KB"
labelsspecialagents special_agent

Page properties
hiddentrue


Related issues