topic Re: Filter event data using conditional regex in Splunk Search

Filter event data using conditional regex

anshul0915 — Sat, 23 Sep 2017 04:56:09 GMT

HI All,

Below is my raw event data .
{"FormatVersion":"1.1","StartTime":"2017-09-22T01:11:38.565Z","EndTime":"2017-09-22T01:11:39.468Z","EventType":"Login","Result":"Success","UserId":"dmorand","TerminalId":"172.16.3.85","SessionId":"RCIcAM1DxUYmG7WMDMkEuQXyGTpOqcBMtyrGOPpFUPU=","LoginUri":"/login-auth/saml","EventSource":"Platform","ServerHostname":"fe02.hbc.stage.us-west-2.orionsaas"}

I want event indexing like below condition.
1. IF "EventType":"Login" and "LoginUri":"/login-auth/saml" the index those event. means we need to discard those event in which event type = login and login uri != /login-auth/saml
2. if "EventType":"Login" and "LoginUri" is not present then index those event
3. If in logs event type not equal to login then index those events also

please help for making this regex .

Re: Filter event data using conditional regex

Sukisen1981 — Sun, 24 Sep 2017 12:04:28 GMT

what does the last point 'if "EventType":"Login" not equals to Login index those event' mean?

Re: Filter event data using conditional regex

anshul0915 — Sun, 24 Sep 2017 12:10:13 GMT

If in logs event type not equal to login then index those events also

Re: Filter event data using conditional regex

Sukisen1981 — Sun, 24 Sep 2017 15:20:45 GMT

one last question - do you want to extract them in the conf files before getting indexed or you want to write a post search regex to extract ONLY these events using regex for this particular use case?

Re: Filter event data using conditional regex

anshul0915 — Sun, 24 Sep 2017 15:44:14 GMT

Want in conf file before getting indexed

Re: Filter event data using conditional regex

jkat54 — Sun, 24 Sep 2017 16:54:07 GMT

Props.conf

 [sourcetypeName]
 KV_MODE=json
 INDEXED_EXTRACTIONS=true
 TRANSFORMS-toNull=toNull

Transforms.conf

 [toNull]
 REGEX= .*^((?!EventType":"Login".*LoginUri":"\/login-auth\/saml).*)
 DEST_KEY=queue
 FORMAT=nullQueue

Re: Filter event data using conditional regex

jkat54 — Sun, 24 Sep 2017 17:09:12 GMT

Needs to be on the forwarder(s) and indexer(s). Will only apply to new data that is indexed after the settings are in placeZ

Re: Filter event data using conditional regex

anshul0915 — Sun, 24 Sep 2017 17:41:49 GMT

Why on both places configuration required . On only indexer will it not work??

Re: Filter event data using conditional regex

jkat54 — Mon, 25 Sep 2017 18:26:10 GMT

INDEXED_EXTRACTIONS has to be on the forwarder. The queue routing happens on first full Splunk instance (heavy forwarder or indexer).