Splunk Observability Cloud and Kong plugin setup?

[NickyCarlisle]

I'm trying to get the kong plugin to work with Splunk Observability cloud.

Here is my agent_config.yaml relating to kong:

recievers
 smartagent/kong:
  type: collectd/kong
  host: 127.0.0.1
  port: 8000

service:
 pipelines:
  metrics:
   receivers: [hostmetrics, otlp, signalfx, smartagent/signalfx-forwarder, smartagent/kong]

When I start my Splunk OTEL collector I am getting metrics from the server but not the Kong service, checking journalctl I see:

otelcol[25528]: 2023-01-30T17:08:05.855Z        error        signalfx/handler.go:189        Traceback (most recent call last):
otelcol[25528]:   File "/usr/lib/splunk-otel-collector/agent-bundle/lib/python3.8/site-packages/sfxrunner/scheduler/simple.py", line 57, in _call_on_interval
otelcol[25528]:     func()
otelcol[25528]:   File "/usr/lib/splunk-otel-collector/agent-bundle/collectd-python/kong/kong/reporter.py", line 56, in update_and_report
otelcol[25528]:     self.kong_state.update_from_sfx()
otelcol[25528]:   File "/usr/lib/splunk-otel-collector/agent-bundle/collectd-python/kong/kong/kong_state.py", line 63, in update_from_sfx
otelcol[25528]:     self.update_resource_metrics(status['signalfx'])
otelcol[25528]: KeyError: 'signalfx'
otelcol[25528]:         {"kind": "receiver", "name": "smartagent/kong", "pipeline": "metrics", "monitorID": "smartagentkong", "monitorType": "collectd/kong", "runnerPID": 25545, "createdTime": 1675098485.8552756, "logger": "root", "sourcePath": "/usr/lib/splunk-otel-collector/agent-bundle/lib/python3.8/site-packages/sfxrunner/logs.py", "lineno": 56}

I have installed the kong plugin using these instructions: 

https://docs.splunk.com/Observability/gdi/kong/kong.html

[bishida]

The signalfx Kong plugin was written for Kong 2.x. I don't believe it is compatible with Kong 3.x. This may be the problem if you have the newest version of Kong. Kong 3.x deprecated a bunch of lua libraries the plugin was using.