Hi Team,
I am using splunk otel collector daemonset to collect logs from containers and send them to splunk with some transformations. I am trying to achieve converting following log entry which is in string format
Body: Str(2025-03-05T22:46:16.526842773Z stdout F {"workspace":"#1234","service":{"updated_at":1700246094,"log_type":"kong-apilog"}}
Next I need to parse log entry as
{"timestamp": 2025-03-05T22:46:16.526842773Z, "log_entry": "stdout", "log_type": "F", "log": {"workspace":"#1234","service":{"updated_at":1700246094,"log_type":"kong-apilog"}} }
Following is my config
filelog/kong-logs:
include:
- /var/log/containers/kong-*.log
- type: regex_parser
regex: ^(?P<time>[^ ]+) (?P<stream>stdout|stderr) (?P<logtag>[^ ]*) ?(?P<log>.*)$
parse_from: body
storage: file_storage
transform:
error_mode: ignore
log_statements:
- context: log
statements:
- set(attributes["log"], ParseJSON(attributes["log"]))
So far I am able to parse log attribute alone to json but not able to construct full json structure as mentioned above and I am also facing error converting time attribute which is in string format to timestamp field using following transformer
- set(time, Time(attributes["time"], "%Y-%m-%dT%H:%M:%S.%9N%Z"))
Since my timestamp is in nanoseconds I need to parse it in nano seconds
Can someone please help me in achieving the desired output
Thanks,
Vamsi
