topic Problems Evaluating Field After Rex Extraction in Splunk Search

Problems Evaluating Field After Rex Extraction

mwollenweber — Wed, 28 Dec 2011 05:25:20 GMT

I'm trying to evaluate a field after it is extracted at search time using rex. Unfortunately it is failing. An example query:

sslvpn* "Session started" | dedup _raw | rex field=_raw "\[(?<ip>\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b)\]\ (?<netid>.*)\(" | ip != 10.*

Results in an error message:

Search operation 'ip' is unknown. You
might not have permission to run this
operation.

I know IP field is being discovered as the following query works:

sslvpn* "Session started" | dedup _raw | rex field=_raw "\[(?<ip>\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b)\]\ (?<netid>.*)\(" | stats values(netid), dc(netid), values(host) by ip | RENAME dc(netid) as netid_count | WHERE netid_count > 1

What I'm hoping to do is filter based on IP addresses. I've also tried adding a WHERE clause but that didn't work either. Help would be appreciated.

Re: Problems Evaluating Field After Rex Extraction

dwaddle — Wed, 28 Dec 2011 05:39:27 GMT

The first thing after a pipe should be a command, and ip is not a command. Without seeing your search I would say that where should have worked. But we'll try something else instead.

sslvpn* "Session started" 
| dedup _raw 
| rex field=_raw "\[(?<ip>\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b)\]\ (?<netid>.*)\(" 
| search ip != 10.*

Alternately, this is a good place for the cidrmatch function which is part of the where command. So, something like:

sslvpn* "Session started" 
| dedup _raw 
| rex field=_raw "\[(?<ip>\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b)\]\ (?<netid>.*)\(" 
| where cidrmatch("10.0.0.0/8",ip)

Re: Problems Evaluating Field After Rex Extraction

mwollenweber — Wed, 28 Dec 2011 06:01:44 GMT

Excellent, your where cidrmatch solution works. Oddly enough

sslvpn* "Session started" 
| dedup _raw 
| rex field=_raw "\[(?<ip>\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b)\]\ (?<netid>.*)\(" 
| where ip != "10.*"

evaluates correctly but doesn't appropriately filter but

sslvpn* "Session started" 
| dedup _raw 
| rex field=_raw "\[(?<ip>\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b)\]\ (?<netid>.*)\(" 
| where ip != 10.*

is malformed with error:

Error in 'where' command: The expression is malformed. The factor is missing.

ip has a string as a type, so I'm not sure why the two options above fail, but your approach works. Thanks again!