Getting Data In

What adverse results can occur if using an override index and override sourcetype at the same time?

Log_wrangler
Builder

Just wanted to poll the community as I am currently testing this.

Fyi - a UF on a SYSLOG-NG is not possible at the moment

I have multiple inputs coming in on one tcp port. The logs I want to move to indx_b and sourcetype_b contain > |foo bar| as a pattern which I will regex.

inputs.conf

[tcp://666] 
Disabled = 0
index = indx_A
sourcetype = st_A

I want to try the following or any suggested variation

props.conf

[source::tcp:666]
TRANSFORMS-Indx_B = SEND_TO_Indx_B
TRANSFORMS-ST_B = CHANGE_TO_ST_B

transforms.conf

[SEND_TO_Indx_B]
REGEX = \|foo bar\|
DEST_KEY = _MetaData:Index
FORMAT = Indx_B

[CHANGE_TO_ST_B]
DEST_KEY = MetaData:Sourcetype
REGEX=\|foo bar\|
FORMAT = sourcetype::st_B

Is it possible to override the sourcetype based on the index its going to? e.g. REGEX = Indx_B ?

Will a double override create a performance problem? or any other?

Thank you

0 Karma
1 Solution

jtacy
Builder

Your config seems to work as expected on a Splunk 6.6.3 indexer; what version are you running?

As to your question about performance, you're going to add some amount of CPU load by applying the transforms but unless your event volume is extremely high (at least tens of thousands of events per second) I suspect that you won't even notice the load. You've already done Splunk a favor by limiting the transforms to a specific source; unless your hardware is already overwhelmed I wouldn't worry about performance.

View solution in original post

0 Karma

jtacy
Builder

Your config seems to work as expected on a Splunk 6.6.3 indexer; what version are you running?

As to your question about performance, you're going to add some amount of CPU load by applying the transforms but unless your event volume is extremely high (at least tens of thousands of events per second) I suspect that you won't even notice the load. You've already done Splunk a favor by limiting the transforms to a specific source; unless your hardware is already overwhelmed I wouldn't worry about performance.

0 Karma

Log_wrangler
Builder

Sorry for the very delayed response, I believe my version of indexer is the problem.

0 Karma

Log_wrangler
Builder

If you convert to answer I will accept. Thank you

0 Karma

Log_wrangler
Builder

does not look like you can do a double override... I have tried the above with no luck. It seems that once the event goes thru the transforms and is sent to index_B, the event is gone before the sourcetype transforms can apply.

Any way to write the transforms to change index and sourcetype at the same time?

Thank you

0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...