topic Re: run different filter in an index search based on a condition in dropdown in Splunk Search

run different filter in an index search based on a condition in dropdown

LearningGuy — Sat, 23 Sep 2023 00:43:04 GMT

Is it possible to run different filter in an index search based on a condition in dropdown below?
The second filter works for both ipv4 and ipv6, but it is slowing down the search. I don't want ipv4 going through my filter for ipv6.
Thanks

If select IPv4
dropdown box > select 1.1.1.1
ip_token=1.1.1.1

Search:
| index=vulnerability_index
ip="$ip_token$"

if select IPv6
dropdown box > select 2001:db8:3333:4444:5555:6666::2101
ip_token=2001:db8:3333:4444:5555:6666::2101

Search:
| index=vulnerability_index
| rex mode=sed field=ip "s/<regex>/<replacement>/<flags>"
| search ip="$ip_token$"

Re: run different filter in an index search based on a condition in dropdown

yuanliu — Sat, 23 Sep 2023 07:49:21 GMT

Maybe you can first answer the question why does the first search not satisfy your need? In other words, what is that rex is supposed to accomplish? If your data look like the following:

_raw	_time	ip
foo 1.1.1.1 bar	2023-09-23 00:44:01	1.1.1.1
foo 2.2.2.2 bar	2023-09-23 00:44:01	2.2.2.2
foo 2001:db8:3333:4444:5555:6666::2101 bar	2023-09-23 00:44:01	2001:db8:3333:4444:5555:6666::2101
foo 2001:db8:3333:4444:5555:6666::2102 bar	2023-09-23 00:44:01	2001:db8:3333:4444:5555:6666::2102

ip="$ip_token$" should pick up the correct event whether $ip_token$ is 1.1.1.1 (IPv4) or 2001:db8:3333:4444:5555:6666::2101 (IPv6). What am I missing here?

Re: run different filter in an index search based on a condition in dropdown

LearningGuy — Sat, 23 Sep 2023 13:23:11 GMT

Hello,
The first search does not work because ipv6 from the dropdown is in a compressed format from a different data source, while the ipv6 in the index is in not in a compressed format, so it has to go through a regex or function to convert it to a compressed format in the second search.

Thank you for your help

Re: run different filter in an index search based on a condition in dropdown

PickleRick — Sat, 23 Sep 2023 17:18:14 GMT

Also remember that if you do manual extraction with the rex command and only then search on its results it will be much much slower than by simply searching the index because instead of finding the value in the index splunk has to pass every event through the regex extraction and only then find matching events.

Re: run different filter in an index search based on a condition in dropdown

LearningGuy — Sat, 23 Sep 2023 22:41:44 GMT

Hello,
I am using a regex because the ipv6 on the index is not in compressed format. The search with regex is slower than regular search, that is the reason why I want to bypass the regex for ipv4.
Please suggest.

Thanks

Re: run different filter in an index search based on a condition in dropdown

yuanliu — Sat, 23 Sep 2023 23:39:44 GMT

Ah, the same situation as you expressed in Re: How to perform lookup on inconsistent IPv6 for.... The solution is also the same: Use host CIDR expressions instead of host IP address in search. This time, it is right in search command, no lookup required. (Absolutely no regex. Always suppress your urge to manipulate structured data as string.) See CIDR matching.

So, instead of

Instead of	Use
If select IPv4 dropdown box > select 1.1.1.1 ip_token="1.1.1.1"	If select IPv4 dropdown box > select 1.1.1.1 ip_token="1.1.1.1/32"
if select IPv6 dropdown box > select 2001:db8:3333:4444:5555:6666::2101 ip_token="2001:db8:3333:4444:5555:6666::2101"	if select IPv6 dropdown box > select 2001:db8:3333:4444:5555:6666::2101 ip_token="2001:db8:3333:4444:5555:6666::2101/128"

You will be using the same efficient search for both no matter whether the address representation is compressed or not.

Let me guess your next question (because I did answered your follow-up IPv6 questions:-): the tokens are populated by a search, so you need to know which host bitmask to apply to which value. Well, that answer was a hack on ipmask function: https://community.splunk.com/t5/Splunk-Search/How-to-perform-lookup-on-inconsistent-IPv6-format-in-CSV-file/m-p/657104/highlight/true#M226967

Re: run different filter in an index search based on a condition in dropdown

PickleRick — Sun, 24 Sep 2023 05:54:01 GMT

Any way you do it, it _will_ be inefficient (that's the "beauty" of matching ipv6 addresses). In this case it probably would be best to use additional "external" mechanics if possible - maybe try to expand the addresses on ingest to index the full form and have it easier matchable on search later. Or at least add an indexed field with a flag to easily identify the fields having ipv6 field version.

Re: run different filter in an index search based on a condition in dropdown

LearningGuy — Mon, 25 Sep 2023 14:51:19 GMT

Hello,

Can you give an example how to implement your suggestion in the search with cidrmatch?
Assume that the mask already added in the dropdown box. Thank you for your help
ip_token=1.1.1.1/32
ip_token=2001:db8:3333:4444:5555:6666::2101/128

Search
| index=vulnerability_index
``` if cidrmatch then ``` ???
ip="$ip_token$"

-------------------------------------------------------
Note that my search with regex below works for both ipv4 and ipv6 and it's faster than 3rd party ipv6compress function
my original question: is it possible only to bypass regex statement for ipv4 (only use regex for ipv6)?
I was able to use drilldown condition in XML source as a workaround, but it made the code complex and it's not transferrable to Dashboard Studio

Search:
| index=vulnerability_index
| rex mode=sed field=ip "s/<regex>/<replacement>/<flags>"
| search ip="$ip_token$"

Re: run different filter in an index search based on a condition in dropdown

LearningGuy — Mon, 25 Sep 2023 14:56:50 GMT

Hello,

I think my original question was not clear. My apology.
my search with regex below works for both ipv4 and ipv6 and it's faster than 3rd party ipv6compress function
my original question: is it possible only to bypass regex statement for ipv4 (only use regex for ipv6)?
I was able to use drilldown condition in XML source as a workaround, but it made the code complex and it's not transferrable to Dashboard Studio. Thank you for your help.

Search
| index=vulnerability_index
| rex mode=sed field=ip "s/<regex>/<replacement>/<flags>"
| search ip="$ip_token$"

Re: run different filter in an index search based on a condition in dropdown

yuanliu — Tue, 26 Sep 2023 05:22:22 GMT

rex can only happen after scooping up all events. That is why you feel slow with your second search.

When match happens in search command, you only pick up that matching one. The search is just as your first search. No matter whether the token is IPv4 or IPv6, search command is the same

index=vulnerability_index ip="$ip_token$"

Consider the following mock data:

10.10.10.12

50.10.10.17

10.10.10.23

fa00:0:0:0::1

fa00:0:0:0::2

1. $ip_token$ = fa00::1/128

Result:

_time	ip
2023-09-25 22:05:27	fa00:0:0:0::1

| makeresults | eval ip = split("10.10.10.12 50.10.10.17 10.10.10.23 fa00:0:0:0::1 fa00:0:0:0::2", " ") | mvexpand ip | search ip=fa00::1/128 ``` the above emulates index=vulnerability_index ip = fa00::1/128 ```

2. $ip_token$ = 10.10.10.23/32

Result:

_time	ip
2023-09-25 22:13:01	10.10.10.23

| makeresults | eval ip = split("10.10.10.12 50.10.10.17 10.10.10.23 fa00:0:0:0::1 fa00:0:0:0::2", " ") | mvexpand ip | search ip=10.10.10.23/32 ``` the above emulates index=vulnerability_index ip = 10.10.10.23/32 ```

Re: run different filter in an index search based on a condition in dropdown

LearningGuy — Thu, 28 Sep 2023 00:19:01 GMT

Hello,
Thanks for your help.
There was a workaround to use condition value using a drilldown
https://community.splunk.com/t5/Dashboards-Visualizations/Condition-value-using-a-drilldown/m-p/255914
It worked fine when I tested it, but the issue is it's difficult to read and it's not transferrable to Dashboard Studio
<eval token="dmp">if(like($row.VulnerableIPs$,":"), "| search ip=\"" . $row.VulnerableIPs$ . "\" | rex mode=sed field=ip \"s/<regex>/<replacement>/<flags>"", "ip=" . $row.VulnerableIPs$ )
</eval>