All Apps and Add-ons

How does splunk process transforms.conf ?

jeffking
Engager

Hello, I have a couple of questions -

I have a splunk forwarder with the SCOM app forwarding events to my splunk indexer. I have a lot of events forwarding and noticed that it takes a few hours before the events show up via search. How can I determine if this is due to the indexer still processing events or the forwarder being slow sending the events to the indexer?

I'm trying to cut down on the events by filtering out some unneeded events. I don't need any events from this sourcetype that contain "Failure Code: 0x19". Here's what I put in my props.conf & transforms.com

props.conf:

[SCOM:Events:Security]
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD=30
LINE_BREAKER = ([\r\n](?=\d{2}/\d{2}/\d{2,4} \d{2}:\d{2}:\d{2} [aApPmM]{2}))
REPORT-MESSAGE = wel-message, wel-eq-kv, wel-col-kv
KV_MODE=none
TRANSFORMS-null = setnull

transforms.conf:

[setnull]
REGEX = (?m)^.*Failure\s+Code:\s+0x19.*$
DEST_KEY = queue
FORMAT = nullQueue

How can I tell this is working if the indexer is still processing events? Does it process the event when it indexes or when it receives the event from the forwarder? Does the regex look okay to you?

Thanks in advance!

0 Karma
1 Solution

lukejadamec
Super Champion

You can test the regex with a search in the search app.

sourcetype=SCOM:Events:Security | regex _raw=(?m)^.*Failure\s+Code:\s+0x19.*$

The Transforms are processed on the indexer unless the forwarder is a heavy forwarder that is cooking the data in which case it is done on the heavy forwarder.

View solution in original post

lukejadamec
Super Champion

You can test the regex with a search in the search app.

sourcetype=SCOM:Events:Security | regex _raw=(?m)^.*Failure\s+Code:\s+0x19.*$

The Transforms are processed on the indexer unless the forwarder is a heavy forwarder that is cooking the data in which case it is done on the heavy forwarder.

lukejadamec
Super Champion

You are welcome. Feel free to accept the answer:)

0 Karma

jeffking
Engager

Thanks for the help, lukejadamec. I set the transform on the heavy forwarder and that seems to do the trick. Also, by stopping the heavy forwarder I noticed events stopped coming into the indexer. So that answers my first question about the bottleneck. It's slow from the forwarder, and adding that nullqueue I believe will help.

0 Karma

lukejadamec
Super Champion

You should put the transform on the heavy forwarder that way if you should make a change in the future to the forwarder function you won't loose this piece.
With transforms, they are done only once - once cooked it cannot be recooked. In other words, you can't transform in both places.

0 Karma

jeffking
Engager

Thanks. That is how I built the regex query and it works in the search app. The forwarder is a heavy forwarder, but I don't think the SCOM app is cooking the data, just forwarding it. I think I need to do some testing where the transforms.conf is being processed.

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...