We are having difficulty getting exclusions of logs that have fields in Camelcase or have entries that have special characters related to OTEL logs. Fields without capitalization and/or special chara...
See more...
We are having difficulty getting exclusions of logs that have fields in Camelcase or have entries that have special characters related to OTEL logs. Fields without capitalization and/or special character values are able to be parsed out, but not others.
Here is an example log that we are looking at (attached as yaml and key portion).
filelog/kube-apiserver-audit-log:
include:
- /var/log/kubernetes/kube-apiserver.log
include_file_name: false
include_file_path: true
operators:
- id: extract-audit-group
type: regex_parser
regex: '\s*\"resourceGroup\"\s*\:\s*\"(?P<extracted_group>[^\"]+)\"\s*'
- id: filter-group
type: filter
expr: 'attributes.extracted_beta == "batch"'
- id: remove-extracted-group
type: remove
field: attributes.extracted_group
- id: extract-audit-api
type: regex_parser
regex: '\"level\"\:\"(?P<extracted_audit_beta>[^\"]+)\"'
- id: filter-api
type: filter
expr: 'attributes.extracted_audit_beta == "Metadata"'
- id: remove-extracted-api
type: remove
field: attributes.extracted_api
- id: extract-audit-verb
type: regex_parser
regex: '\"verb\"\:\"(?P<extracted_verb>[^\"]+)\"'
- id: filter-verb
type: filter
expr: 'attributes.extracted_verb == "watch" || attributes.extracted_verb == "list"'
- id: remove-extracted-verb
type: remove
field: attributes.extracted_verb
The resourceGroup field is compared to something else and failing, verb and level are succeeding.
Here is an example log that would be pulled in.
{"apiVersion":"batch/v1","component":"sync-agent","eventType":"MODIFIED","kind":"CronJob","level":"info","msg":"sent event","name":"agentupdater-workload","namespace":"vmware-system-tmc","resourceGroup":"batch","resourceType":"cronjobs","resourceVersion":"v1","time":"2024-03-14T18:17:11Z"}