topic Re: Dynamically remove events based on field value in Splunk Search

How to dynamically remove events based on field value?

es5 — Thu, 20 Oct 2022 22:49:05 GMT

Hello all,

I have a search that's something like this:

index=* sourcetype=* ID=* (value=1 OR value=2 OR value=3) | list(_raw) as events BY ID value msg | table ID value msg

Next, I utilize a drilldown option that adds the chosen value into a new search. Basically:

index=* sourcetype=* ID=* value=1 | table ID value msg

The point is to group events into one list based on them having the same ID and a specific value. Now, when I click the drilldown sometimes the table will include fields of value=1 that contain a "msg" field that is irrelevant to the data I'm searching for.

Is it possible to do something like:

index=* sourcetype=* ID=* value=1 | table ID value msg | eval msg=if(msg==bad, "Remove From Table", msg)

Sorry for being vague, but I cannot post the actual searches. I hope this makes sense.

Re: Dynamically remove events based on field value

bowesmana — Wed, 19 Oct 2022 22:04:17 GMT

You can either just include that in the search command, e.g.

index=* sourcetype=* ID=* value=1 msg!="bad" | table ID value msg OR ... | where msg!="bad"

Presumably you have tokens from the original panel where the drilldown is made, so your search is something like

index=* sourcetype=* ID=* value=$clicked_token_value$ | table ID value msg

Maybe there is something in the clicked data that you can also use as a filter for msg?

Anyway, basic answer is just exclude from search with search or where commands

Re: Dynamically remove events based on field value

es5 — Wed, 19 Oct 2022 22:29:45 GMT

Hey thanks for the reply. It's been a long day and I'm not sure I explained the question correctly. That being said, your solution will definitely work for this case. I will test as I think it might work in my case too.

The problem is that the ID field is dependent on not being excluded in some cases. For instance:

index=* sourcetype=* (msg=x OR msg=y OR msg=z) (value=1 OR value=2 OR value=3) | list(_raw) AS events BY ID value msg

When value=1, it's okay to exclude msg=y or msg=z, but when value !=1, I still want those values to be included because I might get events where msg=y and value=2.

Does that make sense? I essentially want something like:

| where msg!=x AND value==1

Can you use logic in the where command like that - I don't know but will try it.

Re: Dynamically remove events based on field value

bowesmana — Wed, 19 Oct 2022 22:45:40 GMT

I'm not sure I get the requirements totally - and it's the start of my day 😁 but I'm only 2 coffees in

However, the 'where' clause is very powerful - you can use any kind of eval statement in there.

Note that when comparing strings in where clauses, you can't use wildcard, like in search, e.g.

| where msg!="bad*"

Instead you can use the match statement, which takes a regex, e.g.

| where match(msg, "^bad$") AND value=1

you can put all sorts of AND/OR and other eval logic in there

Re: Dynamically remove events based on field value

johnhuang — Thu, 20 Oct 2022 03:29:17 GMT

If you want to exclude value=1 and msg=y OR msg=z then you should try:

| search NOT (value=1 AND msg IN ("y", "z"))

Re: Dynamically remove events based on field value

es5 — Thu, 20 Oct 2022 15:37:22 GMT

No worries, the problem was with my communication not your understanding! Let me try to explain in a more digestible way.

I have a search that checks for specific commands. IE: I want it to return all information where value = 1, 2, 3

The search specifically looks for values that are 1, 2, or 3, and when it finds those values, they also contain the msg field which can contain x, y, or z. The problem is, if I were to code:

| where value==1 AND msg==x OR msg==y

I'm excluding a large portion of the potential returned results of the search because now I'm only checking for value=1 and msg=x, y.

So for the search:

index=* sourcetype=* (value=1 OR value=2 OR value=3) AND (msg=x OR msg=y OR msg=z)

I want my drilldown to show me specifics for each value, essentially, but not to exclude from the original search the potential for different values to be returned with their respective msg. So, I need the search to dynamically understand that ONLY IF value=1, should it exclude msg=x, y. But, if value=2 the search should still return msg where msg ANY of x, y, z.

Programmatically it'd look like:

if(value==1): exclude(x, y) else: include(x, y, z)

Still not sure if that makes sense and I understand it's hard to conceptualize without the actual search, so I appreciate the help.

Re: Dynamically remove events based on field value

bowesmana — Thu, 20 Oct 2022 21:26:14 GMT

OK, so your main search is

index=* sourcetype=* (value=1 OR value=2 OR value=3) AND (msg=x OR msg=y OR msg=z)

then the drilldown search could be

index=* sourcetype=* value=$value_token$ | where (value=1 AND msg=z) OR value!=1

so, it's effectively adding the where filter to say

value=1 + msg=z (same as your exclude x,y)
OR value=2 or 3 (!=1)

There are other ways, which may be more optimal in that in your <drilldown> part of the dashboard, you could set more than just the clicked value. You could also set the msg constraints with <eval> statements, e.g. (untested)

<drilldown> <set token="value_token">$row.value$</set> <eval token="msg_token">if($row.value$=1, "z", "x,y,z")</eval> </drilldown>

and then the drilldown search will do

index=* sourcetype=* value=$value_token$ msg IN ($msg_token$)

so, in the case you click 1, the msg_token is set to z and for all other values, it will set x,y,z and that is used for the drilldown.

You could also make the first search a base search and the drilldown search use the base search and then you only need * as the non value=1 test, as the base search has already filtered x,y,z.

Note: I have assumed you are drilling down from a table, hence the $row.value$ token setting.

I hope this helps get you where you need to get.

Re: Dynamically remove events based on field value

es5 — Mon, 24 Oct 2022 14:45:46 GMT

Thanks for your replies. I will give this a try today and let you know how it turns out!