All Apps and Add-ons

Split kubernetes opentelemetry logs to differents indexes

Ivansplunk
Observer

Hi,

With have some applications running on kubernetes.
All the logs produced by the application are sent to the standard output of the pod instance.

On those logs, we would like to be able to extract them (based on a pattern for exemple) and send them to a specific index. The others logs would go to a "by default" index.

Can we acheive this with splunk OTEL for kubernetes?
do you have some hints where i should start first ?

thank you

 

Labels (1)
0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@Ivansplunk - You can dynamically change metadata (including Index) at the parsing phase of data ingestion.

You can decide based on source, host, raw event content, write regex upon it, and update the index accordingly.

# transforms.conf

[<transforms stanza name>]
SOURCE_KEY = MetaData:Host OR MetaData:Sourcetype OR if you don't add this attribute regex will apply on _raw event data
REGEX = <write regex>
DEST_KEY = _MetaData:Index
FORMAT = Use $1, $2 from regex group to extract dynamic values, otherwise use static string of index name

 

Reference - https://docs.splunk.com/Documentation/Splunk/8.2.6/Admin/Transformsconf 

https://community.splunk.com/t5/Splunk-Search/How-to-dynamically-set-an-index-based-on-a-string-in-e...

https://community.splunk.com/t5/Dashboards-Visualizations/How-to-set-an-index-dynamically/m-p/262562 

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...