I am trying to build some logic for a docker/k8s integration that we are doing through fluentd. Basically we are testing one avenue of this integration utilizing fluentd -> splunk. One of the things we want to be able to do, is to leverage a field in the json payload called 'namespace' to separate the data into its own index. Right now, here is the setup of my inputs.conf
[tcp:1520] connection_host = dns index = app_cpceng sourcetype = fluentd_json acceptFrom = xx.xx.xx.xx
Now i know utilizing transforms we can do something like this:
[routemyhosttomynewindex] SOURCE_KEY = MetaData:Host REGEX = myhost DEST_KEY = MetaData:Index FORMAT = mynewindex
where in this example it reroutes the data to an alertnate index based on the hostname, but is it possible for me to do something like this:
[routemyhosttomynewindex] SOURCE_KEY = MetaData:Raw (not sure if that is the correct meta name) REGEX = \"namespace\":\"(\w+)\" DEST_KEY = MetaData:Index FORMAT = app_cpceng_$1
A very crude example above, but hopefully it illustrates what I am looking for.
The default SOURCE_KEY is _raw so, just exclude that in your transforms.conf stanza, ensure regex is correct and it should work fine.
Reference, (REGEX per your need)
Good to know, thanks for the info. One of my curiosities was if I could use regex group captures within the FORMAT field, but based on : https://answers.splunk.com/answers/326378/can-i-do-named-capture-in-transformsconf-format-ip.html it looks like I can, so i think my question is answered 🙂