Getting Data In

What adverse results can occur if using an override index and override sourcetype at the same time?

Log_wrangler
Builder

Just wanted to poll the community as I am currently testing this.

Fyi - a UF on a SYSLOG-NG is not possible at the moment

I have multiple inputs coming in on one tcp port. The logs I want to move to indx_b and sourcetype_b contain > |foo bar| as a pattern which I will regex.

inputs.conf

[tcp://666] 
Disabled = 0
index = indx_A
sourcetype = st_A

I want to try the following or any suggested variation

props.conf

[source::tcp:666]
TRANSFORMS-Indx_B = SEND_TO_Indx_B
TRANSFORMS-ST_B = CHANGE_TO_ST_B

transforms.conf

[SEND_TO_Indx_B]
REGEX = \|foo bar\|
DEST_KEY = _MetaData:Index
FORMAT = Indx_B

[CHANGE_TO_ST_B]
DEST_KEY = MetaData:Sourcetype
REGEX=\|foo bar\|
FORMAT = sourcetype::st_B

Is it possible to override the sourcetype based on the index its going to? e.g. REGEX = Indx_B ?

Will a double override create a performance problem? or any other?

Thank you

0 Karma
1 Solution

jtacy
Builder

Your config seems to work as expected on a Splunk 6.6.3 indexer; what version are you running?

As to your question about performance, you're going to add some amount of CPU load by applying the transforms but unless your event volume is extremely high (at least tens of thousands of events per second) I suspect that you won't even notice the load. You've already done Splunk a favor by limiting the transforms to a specific source; unless your hardware is already overwhelmed I wouldn't worry about performance.

View solution in original post

0 Karma

jtacy
Builder

Your config seems to work as expected on a Splunk 6.6.3 indexer; what version are you running?

As to your question about performance, you're going to add some amount of CPU load by applying the transforms but unless your event volume is extremely high (at least tens of thousands of events per second) I suspect that you won't even notice the load. You've already done Splunk a favor by limiting the transforms to a specific source; unless your hardware is already overwhelmed I wouldn't worry about performance.

View solution in original post

0 Karma

Log_wrangler
Builder

Sorry for the very delayed response, I believe my version of indexer is the problem.

0 Karma

Log_wrangler
Builder

If you convert to answer I will accept. Thank you

0 Karma

Log_wrangler
Builder

does not look like you can do a double override... I have tried the above with no luck. It seems that once the event goes thru the transforms and is sent to index_B, the event is gone before the sourcetype transforms can apply.

Any way to write the transforms to change index and sourcetype at the same time?

Thank you

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.