Index only events matching with pattern and exclude everyting else

AKG1_old1 — Wed, 16 Sep 2020 16:17:19 GMT

Hello,

In particular sourcetype we are getting huge numbers of events but only some data events are relevant. I am try to take only events with matching string and exclude everything else.

Matching strings : Session initialization | Session initialized (There are few more as well)

I have tried this by refereing this post Link

When I am using this its excluding everyting and when i tried only with setparsing its injesting all data. Not sure what I am missing here.

props.conf

[mx_java_event]
DATETIME_CONFIG =
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
category = Custom
pulldown_type = true
EXTRACT-JavaClass = ,\d+\s\[(?<JavaClass>[^:]*):
EXTRACT-Session = session:(?<Session>\d+)
TRANSFORMS-set = setnull, setparsing

transforms.conf

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = Session initialization | Session initialized
DEST_KEY = queue
FORMAT = indexQueue

FYI : In case REGEX is incorrect, I tried with "REGEX = Session" its not working either.

Sample data: (Only 1st and 3rd line are matching)

2020-08-12 14:08:11,775 [Thread-233 - Worker-54] murex.processing.stp.osp.server.service.OspServer : DEBUG - [session:1758555252] Session initialization - SGITOPS/SG_LAW_MRC
2020-08-12 14:08:12,775 [Thread-233 - Worker-54] murex.processing.stp.osp.server.service.OspServer : DEBUG - [session:1758555252] Excluded - SGITOPS/SG_LAW_MRC
2020-08-12 14:08:11,912 [Thread-233 - Worker-54] murex.processing.stp.osp.server.service.OspServer : DEBUG - [session:1758555252] Session initialized
2020-08-12 14:08:12,912 [Thread-233 - Worker-54] murex.processing.stp.osp.server.service.OspServer : DEBUG - [session:1758555252] Session Excluded2
2020-08-12 14:08:12,912 JUST FOR Testing

Re: Index only events matching with pattern and exclude everyting else

thambisetty — Wed, 16 Sep 2020 16:12:42 GMT

Provide sample events to test regex.

Re: Index only events matching with pattern and exclude everyting else

AKG1_old1 — Wed, 16 Sep 2020 16:18:11 GMT

added sample data

Re: Index only events matching with pattern and exclude everyting else

thambisetty — Thu, 17 Sep 2020 08:53:40 GMT

props.conf

[mx_java_event] TRANSFORMS-set = setnull, setparsing

transforms.conf

NOTE: if event contains "Session initialization" OR "Session initialized" anywhere then the event will be indexed others are ignored.

[setnull] REGEX = . DEST_KEY = queue FORMAT = nullQueue [setparsing] REGEX = (Session initialization|Session initialized) DEST_KEY = queue FORMAT = indexQueue

below is output from your sample events:

topic Re: Index only events matching with pattern and exclude everyting else in Splunk Enterprise

Index only events matching with pattern and exclude everyting else

Re: Index only events matching with pattern and exclude everyting else

Re: Index only events matching with pattern and exclude everyting else

Re: Index only events matching with pattern and exclude everyting else