All Apps and Add-ons

How does splunk process transforms.conf ?

jeffking
Engager

Hello, I have a couple of questions -

I have a splunk forwarder with the SCOM app forwarding events to my splunk indexer. I have a lot of events forwarding and noticed that it takes a few hours before the events show up via search. How can I determine if this is due to the indexer still processing events or the forwarder being slow sending the events to the indexer?

I'm trying to cut down on the events by filtering out some unneeded events. I don't need any events from this sourcetype that contain "Failure Code: 0x19". Here's what I put in my props.conf & transforms.com

props.conf:

[SCOM:Events:Security]
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD=30
LINE_BREAKER = ([\r\n](?=\d{2}/\d{2}/\d{2,4} \d{2}:\d{2}:\d{2} [aApPmM]{2}))
REPORT-MESSAGE = wel-message, wel-eq-kv, wel-col-kv
KV_MODE=none
TRANSFORMS-null = setnull

transforms.conf:

[setnull]
REGEX = (?m)^.*Failure\s+Code:\s+0x19.*$
DEST_KEY = queue
FORMAT = nullQueue

How can I tell this is working if the indexer is still processing events? Does it process the event when it indexes or when it receives the event from the forwarder? Does the regex look okay to you?

Thanks in advance!

0 Karma
1 Solution

lukejadamec
Super Champion

You can test the regex with a search in the search app.

sourcetype=SCOM:Events:Security | regex _raw=(?m)^.*Failure\s+Code:\s+0x19.*$

The Transforms are processed on the indexer unless the forwarder is a heavy forwarder that is cooking the data in which case it is done on the heavy forwarder.

View solution in original post

lukejadamec
Super Champion

You can test the regex with a search in the search app.

sourcetype=SCOM:Events:Security | regex _raw=(?m)^.*Failure\s+Code:\s+0x19.*$

The Transforms are processed on the indexer unless the forwarder is a heavy forwarder that is cooking the data in which case it is done on the heavy forwarder.

lukejadamec
Super Champion

You are welcome. Feel free to accept the answer:)

0 Karma

jeffking
Engager

Thanks for the help, lukejadamec. I set the transform on the heavy forwarder and that seems to do the trick. Also, by stopping the heavy forwarder I noticed events stopped coming into the indexer. So that answers my first question about the bottleneck. It's slow from the forwarder, and adding that nullqueue I believe will help.

0 Karma

lukejadamec
Super Champion

You should put the transform on the heavy forwarder that way if you should make a change in the future to the forwarder function you won't loose this piece.
With transforms, they are done only once - once cooked it cannot be recooked. In other words, you can't transform in both places.

0 Karma

jeffking
Engager

Thanks. That is how I built the regex query and it works in the search app. The forwarder is a heavy forwarder, but I don't think the SCOM app is cooking the data, just forwarding it. I think I need to do some testing where the transforms.conf is being processed.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...