Getting Data In

How to solve setnull issue while filtering required events?

roopeshetty
Path Finder

Hi ,

 

We are getting lot of events into our Splunk which is filling up the our computer disk storage rapidly.

 

Our search query is “index="cloud" sourcetype=sls_sourcetype”

 

 

Upon investigating we decided that we need to capture only below events;

Collect only below events Where field name “sql” contains below 4 values ;

 

logout!

login success!

login failed!

create

 

Actually field name “sql” contains more than 100 values so we need to exclude all of them and capture only those events which has mentioned above 4 values.

We tried below configurations in transforms.conf and props.conf files but none of them are giving desired results. Can some please help us with the correct settings as this is the first time we are working transforms.conf and props.conf settings.

 

 

transforms.conf  as below;

 

[setnull]

SOURCE_KEY=_raw

REGEX = *

DEST_KEY = queue

FORMAT = nullQueue

[setqueue]

SOURCE_KEY=_raw

REGEX = sql=log(out! |in success! |in failure!)

DEST_KEY = queue

FORMAT = indexQueue

 

Props.conf as below;

 

[sourcetype::sls_sourcetype]

TRANSFORMS-set= setnull,setparsing

Labels (1)
0 Karma
1 Solution

PaulPanther
Motivator

@roopeshetty  There is a typo in your props.conf. The stanza setparsing does not exist.

[sls_sourcetype]
TRANSFORMS-set=setnull,setqueue

and rewrite your setnull stanza

[setnull]
SOURCE_KEY=_raw
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

If it still does not work verify your regex with regex101: build, test, and debug regex.

View solution in original post

0 Karma

PaulPanther
Motivator

@roopeshetty  There is a typo in your props.conf. The stanza setparsing does not exist.

[sls_sourcetype]
TRANSFORMS-set=setnull,setqueue

and rewrite your setnull stanza

[setnull]
SOURCE_KEY=_raw
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

If it still does not work verify your regex with regex101: build, test, and debug regex.

0 Karma
Get Updates on the Splunk Community!

Leveraging Detections from the Splunk Threat Research Team & Cisco Talos

  Now On Demand  Stay ahead of today’s evolving threats with the combined power of the Splunk Threat Research ...

New in Splunk Observability Cloud: Automated Archiving for Unused Metrics

Automated Archival is a new capability within Metrics Management; which is a robust usage & cost optimization ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...