topic Re: Extract field using rex in Splunk Search

Extract field using rex

osasfrancis — Fri, 10 Sep 2021 23:46:56 GMT

I have the below test raw logs

CEF:0|Forcepoint|Forcepoint DLP|8.8.0|55564097|DLP Syslog|2| act=Permitted duser=destuser@gmail.com fname=testfile.PDF.TXT - 11.01 KB msg=EndPoint Operation suser=User, Test cat=Test Category sourceServiceName=Endpoint Printing analyzedBy=Policy Engine testengine loginName=testuser1 sourceIp=N/A severityType=LOW sourceHost=testhost productVersion=8.0 maxMatches=0 timeStamp=2021-09-01 15:58:50.624 destinationHosts=N/A eventId=4762037341417287789

CEF:0|Forcepoint|Forcepoint DLP|8.8.0|55564097|DLP Syslog|2| act=Permitted duser=destuser@gmail.com fname=testfile.PDF.TXT - 11.01 KB msg=EndPoint Operation suser=User, Test cat=Test Category sourceServiceName=Endpoint Printing analyzedBy=Policy Engine testengine loginName=domain\\testuser sourceIp=N/A severityType=LOW sourceHost=testhost productVersion=8.0 maxMatches=0 timeStamp=2021-09-02 15:58:50.624 destinationHosts=N/A eventId=4762037341417287788

CEF:0|Forcepoint|Forcepoint DLP|8.8.0|55564097|DLP Syslog|2| act=Permitted duser=destuser@gmail.com fname=testfile.PDF.TXT - 11.01 KB msg=EndPoint Operation suser=User, Test cat=Test Category sourceServiceName=Endpoint Printing analyzedBy=Policy Engine testengine loginName=tuser sourceIp=N/A severityType=LOW sourceHost=testhost productVersion=8.0 maxMatches=0 timeStamp=2021-09-04 15:58:50.624 destinationHosts=N/A eventId=4762037341417287787

CEF:0|Forcepoint|Forcepoint DLP|8.8.0|55564097|DLP Syslog|2| act=Permitted duser=destuser@gmail.com fname=testfile.PDF.TXT - 11.01 KB msg=EndPoint Operation suser=User, Test cat=Test Category sourceServiceName=Endpoint Printing analyzedBy=Policy Engine testengine loginName=N/A sourceIp=N/A severityType=LOW sourceHost=testhost productVersion=8.0 maxMatches=0 timeStamp=2021-09-03 15:58:50.624 destinationHosts=N/A eventId=4762037341417287786

I am trying to use rex to extract a field called loginName, in which the regex will capture all entries after the "loginName=" text.

I have tried ...| rex field=_raw "(loginName=)(?<loginName>[^\=]+)(?=\s)", but it does not capture all events.

Please assist.

Re: Extract field using rex

richgalloway — Sat, 11 Sep 2021 00:14:24 GMT

Your regex and this simplified version of it

loginName=(?<loginName>[^=]+)\s

works fine with the sample events. Please share an event where the regex fails. Or show the expected results and your actual results.

Re: Extract field using rex

PickleRick — Sat, 11 Sep 2021 16:17:32 GMT

I wouldn't capture [^=]. Maybe in this particular case you don't have users with "=" in the middle of their login but in general, I'd try to come up with a more generalized solution (like capturing up to a first whitespace?).

Re: Extract field using rex

richgalloway — Sat, 11 Sep 2021 16:49:22 GMT

The regex says to capture everything until the first equals sign so, of course, equals signs are not captured. Your sample events don't have equals signs in the loginName field so the existing regex should be fine. If you have examples of loginName values with "=" in them then please share. To capture up to the first whitespace, use \S+.

Re: Extract field using rex

isoutamo — Mon, 13 Sep 2021 08:23:29 GMT

you can play with this by https://regex101.com/r/10Rhs4/1

Rich's regex handle also spaces on name which didn't happen if you are using whitespace as end character.

r. Ismo

Re: Extract field using rex

PickleRick — Mon, 13 Sep 2021 09:24:01 GMT

I don't recall, to be completely honest, what the CEF specification says - where can there be spaces or equal signs. It'd be most reasonable to check with the specs and adjust the regex accordingly. As we can see from the example surely the values can have unescaped spaces. I'm not sure about equal signs in values and spaces in key names. And I'm not 100% sure whether this is a proper CEF 😉

Re: Extract field using rex

osasfrancis — Mon, 13 Sep 2021 14:43:48 GMT

Hi,

This solved my problem. Thanks