How to combine a range of multiple IP address and compare it to a variable

[commanman]

So, I have multiple ip addresses i want to combine them using regex or normal by supplying dashes and compare them to the variable.

For eg:

This is my existing query:

| search NOT src IN (10.161.5.50 , 10.161.5.51,10.161.5.52, 10.161.5.53,10.161.10.20,192.168.1.120,192.168.1.130 )

I had an output of 15 matched output.

What i have tried doing to get result is:

| search NOT src IN ("10.161.5.5[0-3]", 10.161.10.20,192.168.1.120,192.168.1.130)

Doing this lead to an increase in the matched query up to 30 results. Why was this happening and what can i do to prevent it.

Any solutions?

[ITWhisperer · Accepted solution]

| makeresults 
| eval _raw="src
10.161.5.50
10.161.5.51
10.161.5.52
10.161.5.53
10.161.10.20
192.168.1.120
192.168.1.130
10.161.5.54
10.161.10.21
192.168.1.121
192.168.1.131"
| multikv forceheader=1
| table src



| regex src!="10.161.5.5[0-3]|10.161.10.20|192.168.1.1[2-3]0"

[commanman]

Thanks @ITWhisperer  for helping out.

[PickleRick]

Just remember that performance--wise that might not be a best choice.

OK, if you have a small data set to search, no problem. But if the index grows...

Well, let me show you.

I have a linux_auditd index. I did a search on it:

index=linux_auditd addr=119*

 This is the heading of the job inspector:

This search has completed and has returned 11,223 results by scanning 51,721 events in 3.754  seconds

 For comparison I did another one matching events by regex:

index=linux_auditd 
| regex addr="^119\..*"

 This time the job inspector said:

This search has completed and has returned 11,223 results by scanning 3,569,498 events in 236.314  seconds

As you can see, the difference in run time and number of scanned events is huge.

If you do the similar check for yourself you'll see why - in case of field value matching splunk is checking the provided pattern "intelligently" against the index which holds not only raw data but also a summary of the data split into a form of lexical units. So it only had to verify 51 thousand of occurences of "119" pattern to check whether the involved events parse out the value as the needed field.

If you do a "blank" search and then pipe it to regex, splunk reads every single event from a given timeframe and then tries to match it to the given regex.

That's why the difference in execution time is so huge.

It's best to narrow down the search condition as much as you can and only then try to perform additional operations on the data.

Of course - as I wrote before - if your dataset is small, the difference will be negligible and you're good to go with any option but it's good to know the difference.

[PickleRick]

Firstly, Splunk supports only wildcard pattern-matching on search, not regex.

Secondly, be very careful when using negations. Maybe this is not the case, but remember that "src_ip!=127.0.0.1" is not equivalent of "NOT src_ip=127.0.0.1"

[maciep]

is that your entire splunk search?  or is that just a portion of a larger search that is trying to filter results?  i've been out of the splunk game for a bit, but i don't think the search command supports regex.  if this is filtering results in a larger search, then maybe using the where command with the match function is more appropriate.

[commanman]

Hello @maciep  and @PickleRick ,

it is a portion of my larger query.

So i have a correlation search which detects scanner in on my Infrastructure assets but there are some legitmate ip address assigned to the server which should be excluded from the matching then source.

For eg:

Attacker scans my webserver having IP of 123.2.3.245 

Me scanning the webserver having ip address 10.10.10.10

Internal unauthorized employee scanning for the webserver  10.1.10.20

Coming to my point is that there a search query in place which detect these and stores in a variable called src. 

Now i wont to exclude a range of ip address like 10 -20 ip address for scanning in the same background creating a lookup for that would be a waste of resources.

Hence, I query  looks like this 

| search NOT src IN (10.161.5.50, 10.161.5.51, 10.161.5.52, 10.161.5.53, 10.161.10.20, 192.168.1.120, 192.168.1.130 )

What i want to do is group those near by ip address

| search NOT src IN (10.161.5.5[0-3], 10.161.10.20, 192.168.1.120, 192.168.1.130 )
something on these lines. 

I have seen few regex/ ip commands on the forum which are used. But i a bit confused what would work here


I hope this makes any sense to you guys.