I am using the Splunk OTEL Collector Helm chart to send logs from my GKE pods to the Splunk Cloud Platform. I have set `UsesplunkIncludeAnnotation` to `true` to filter logs from specific pods. This setup was working fine until I tried to filter the logs being sent. I added the following configuration to my `splunk` values.yaml:
config:
processors:
filter/ottl:
error_mode: ignore
logs:
log_record:
- 'IsMatch(body, "GET /status")'
- 'IsMatch(body, "GET /healthcheck")'
When I applied this configuration, the specified logs were excluded as expected, but it did not filter logs from the specified pods. I am still receiving logs from all my pods, and the annotation is not taking effect. Additionally, the host is not displaying correctly and is showing as "unknown". (I will attach a screenshot for reference.)
My questions are:
1. How can I exclude these specific logs more effectively?
2. Is there a more efficient way to achieve this filtering?