topic Re: Need a regex improvement in Splunk Search

Need a regex improvement

tinylund — Wed, 29 Sep 2021 18:21:02 GMT

| rex field=_raw "(?<dscvIP>[^\.]\d+\.\d+\.\d+\.\d+[\s|\:])"

Using the above rex command to try to capture IP addresses, an it works most of the time, but I still get a few false positives for ESX log entries that contain the following

Rcv-tx-10.20.30.45.78.80

The rex field captures 30.45.78.80 as the dscvIP field. I thought by adding the [^\.] to the beginning of the regex match it would not capture an string matching the IP syntax that had a period(.) immediately preceding the string. And I also thought this string would be skipped all together because for it to not start with a period(.) , the match would have to go to the dash(-) following tx, but then would not match because there is not a space or colon after the 4th \d+ match.

What am

Re: Need a regex improvement

ashvinpandey — Wed, 29 Sep 2021 18:27:32 GMT

@tinylund Can you share the raw event ?

Re: Need a regex improvement

tinylund — Wed, 29 Sep 2021 18:33:51 GMT

date FQN date:time FQN vmkernel: cpu16:66435)CpuSched: 694: user latency of 418901 RPC-tx-10.20.30.45.78.80 0 changed by 66435 NFSv3-RemountHandler -6

Re: Need a regex improvement

PickleRick — Wed, 29 Sep 2021 18:45:19 GMT

Well. The typical regex used often to capture IP addresses is \d{1,3}\.\d{1,3}\.\d{1,3}\d{1,3}

Unfortunately, it doesn't validate the validity of the captured addres and accepts values over 255. It can be improved but it quickly gets ugly

((1?\d{1,2})|2([0-4]\d|5[0-5]))\d((1?\d{1,2})|2([0-4]\d|5[0-5]))\d((1?\d{1,2})|2([0-4]\d|5[0-5]))\d((1?\d{1,2})|2([0-4]\d|5[0-5]))

Something like that - I'm writing it on the fly so can't guarantee correctness 😉 Regex is not very well suited for matching IP's

Anyway, if you want to capture just the four octet sequence with a guarantee that it's not preceedee or continued by any dot-delimited sequence, you might want to match (^|[^.]) at the beginning and ($|[^.]) at the end (you don't escape the dot in character set)

So effectively (in the simple - non-validating form) you end up with

(^|[^.])(?<IP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})($|[^.])

Re: Need a regex improvement

ashvinpandey — Wed, 29 Sep 2021 18:46:17 GMT

@tinylund Please try using the below rex:

| rex field=_raw "RPC-tx-(?P<dscvIP>.*?)\s"

Also, If this reply helps you, an upvote would be appreciated.

Re: Need a regex improvement

tinylund — Wed, 29 Sep 2021 18:55:05 GMT

This is just the raw log that fails, the above rex finds the correct combination in other logs, so I don't want to single out the RPC-tx- (this solution doesn't resolve the issue)