All Apps and Add-ons

Split kubernetes opentelemetry logs to differents indexes

Ivansplunk
Observer

Hi,

With have some applications running on kubernetes.
All the logs produced by the application are sent to the standard output of the pod instance.

On those logs, we would like to be able to extract them (based on a pattern for exemple) and send them to a specific index. The others logs would go to a "by default" index.

Can we acheive this with splunk OTEL for kubernetes?
do you have some hints where i should start first ?

thank you

 

Labels (1)
0 Karma

VatsalJagani
Super Champion

@Ivansplunk - You can dynamically change metadata (including Index) at the parsing phase of data ingestion.

You can decide based on source, host, raw event content, write regex upon it, and update the index accordingly.

# transforms.conf

[<transforms stanza name>]
SOURCE_KEY = MetaData:Host OR MetaData:Sourcetype OR if you don't add this attribute regex will apply on _raw event data
REGEX = <write regex>
DEST_KEY = _MetaData:Index
FORMAT = Use $1, $2 from regex group to extract dynamic values, otherwise use static string of index name

 

Reference - https://docs.splunk.com/Documentation/Splunk/8.2.6/Admin/Transformsconf 

https://community.splunk.com/t5/Splunk-Search/How-to-dynamically-set-an-index-based-on-a-string-in-e...

https://community.splunk.com/t5/Dashboards-Visualizations/How-to-set-an-index-dynamically/m-p/262562 

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Customer Survey!

If you use Splunk Observability Cloud, we invite you to share your valuable insights with us through a brief ...

Happy CX Day, Splunk Community!

Happy CX Day, Splunk Community! CX stands for Customer Experience, and today, October 3rd, is CX Day — a ...

.conf23 | Get Your Cybersecurity Defense Analyst Certification in Vegas

We’re excited to announce a new Splunk certification exam being released at .conf23! If you’re going to Las ...