Filter event data using conditional regex

[anshul0915]

HI All,

Below is my raw event data .
{"FormatVersion":"1.1","StartTime":"2017-09-22T01:11:38.565Z","EndTime":"2017-09-22T01:11:39.468Z","EventType":"Login","Result":"Success","UserId":"dmorand","TerminalId":"172.16.3.85","SessionId":"RCIcAM1DxUYmG7WMDMkEuQXyGTpOqcBMtyrGOPpFUPU=","LoginUri":"/login-auth/saml","EventSource":"Platform","ServerHostname":"fe02.hbc.stage.us-west-2.orionsaas"}

I want event indexing like below condition.
1. IF "EventType":"Login" and "LoginUri":"/login-auth/saml" the index those event. means we need to discard those event in which event type = login and login uri != /login-auth/saml
2. if "EventType":"Login" and "LoginUri" is not present then index those event
3. If in logs event type not equal to login then index those events also

please help for making this regex .

[jkat54 · Accepted solution]

Props.conf

 [sourcetypeName]
 KV_MODE=json
 INDEXED_EXTRACTIONS=true
 TRANSFORMS-toNull=toNull

Transforms.conf

 [toNull]
 REGEX= .*^((?!EventType":"Login".*LoginUri":"\/login-auth\/saml).*)
 DEST_KEY=queue
 FORMAT=nullQueue

[jkat54]

Needs to be on the forwarder(s) and indexer(s). Will only apply to new data that is indexed after the settings are in placeZ

[anshul0915]

Why on both places configuration required . On only indexer will it not work??

[jkat54]

INDEXED_EXTRACTIONS has to be on the forwarder. The queue routing happens on first full Splunk instance (heavy forwarder or indexer).

[Sukisen1981]

what does the last point 'if "EventType":"Login" not equals to Login index those event' mean?

[anshul0915]

If in logs event type not equal to login then index those events also

[Sukisen1981]

one last question - do you want to extract them in the conf files before getting indexed or you want to write a post search regex to extract ONLY these events using regex for this particular use case?

[anshul0915]

Want in conf file before getting indexed