topic Re: Dashboard - How to trigger second search based on first search where condition is : first result count in Splunk Search

Dashboard - How to trigger second search based on first search where condition is : first result count

Luciana — Fri, 18 Jun 2021 02:45:46 GMT

Please, Can someone help me here?

Basically , in the first search IF the search stats count >=1 then, a second search would be trigger and then, in this search I could use the sendemail command to send me alert with more information.

so, considering first query:

<condition match="$result.resultCount$>1">

then run a second query:

index = xxx sourcetype=xxx action=Allowed [|inputlookup OTX | search type=url | rename indicator as url | table url]
|dedup user |table _time, hostname, domain, user, serverip, ClientIP |sendemail to="myemail.co.nz" server=smtp.server.co.nz subject="Notification - IOC found by url" message="This is an test message" sendresults=true inline=true format=csv

Do you know if is possible, or How Can I do this?

I thought about set a token with
<condition match="$result.resultCount$>1">

but then , I dont know how to trigger a second search if the condition is true.

the second search will be exclusively for sending alerts with more info, it will not be appearing in the dashboard.

Thank you so much

Re: Dashboard - How to trigger second search based on first search where condition is : first result count

bowesmana — Fri, 18 Jun 2021 03:18:43 GMT

In your second query have a dummy token, e.g.

index = xxx sourcetype=xxx action=Allowed $trigger$ [ |inputlookup OTX | search type=url | rename indicator as url | table url ] |dedup user |table _time, hostname, domain, user, serverip, ClientIP |sendemail to="myemail.co.nz" server=smtp.server.co.nz subject="Notification - IOC found by url" message="This is an test message" sendresults=true inline=true format=csv

then in your condition statement

however, once you set that token, the search will not run a second time if the first search runs again.

You can do this to get around that

<condition match="$result.resultCount$>1"> <eval token="trigger">if(isnull($trigger$),0,$trigger$+1</eval> </condition>

See this simple dashboard example. The first panel will make a search and then increase the trigger value, which will then cause the second panel to rerun its search

<dashboard> <label>trigger</label> <row> <panel> <single> <search> <query>index=_audit earliest=-1m latest=now | stats count </query> <earliest>-24h@h</earliest> <latest>now</latest> <sampleRatio>1</sampleRatio> <done> <condition match="$job.resultCount$>0"> <eval token="trigger">if(isnull($trigger$),0,$trigger$+1)</eval>  </condition> </done> </search> <option name="drilldown">none</option> </single> </panel> <panel> <title>Trigger=$trigger$</title> <single> <search> <query>index=_audit $trigger$ earliest=-2m latest=-1m | stats count</query> <earliest>-24h@h</earliest> <latest>now</latest> <sampleRatio>1</sampleRatio> </search> <option name="drilldown">none</option> <option name="refresh.display">progressbar</option> </single> </panel> </row> </dashboard>

Re: Dashboard - How to trigger second search based on first search where condition is : first result count

Luciana — Fri, 18 Jun 2021 05:28:42 GMT

@bowesmana , thanks for your answer. I really appreciate.

I will give a try in your suggestion, but I have a question:

You said that I should add a token in the second query, but in your dashboard you are adding a drilldown in the first query. Was intentional or just an example?

When I saw your model of dashboard I was in doubt how to integrate this in my current dashboard. Please, let me explain better:

I already have a dashboard that shows the status count.

I can't use the sendemail in the first query, because it only shows the number of IOCs found, but it does not give me any detail 😞 In my current dashboard, I have several searchs looking IOCs across different logs.

However, the only way I found to use the second search was using a drilldown.

so, if the USER clicks in the number (stats count - 1st query) , then the detailed information (second query) will appear in the last panel in the bottom. As soon it appears there, then automatically the dashboard will send an email to me, but with my solution, I am always depending on SOMEONE click in the number in red.

I could not find a way to make the drilldown works independently , meaning, drilldown be activated with second query without ANYONE clicking, only obeying the condition bellow in the second query

( <condition match="'click.value'!= 0">)

Even adding a condition in drilldown I m always forced to CLICK. so annoying.

Then, I thought: ok, I can have the stats number, I can have the drilldown, but I could additionally add my second query again after stats number and send the email alert for me.

You can take a look in my dashboard and all the code in the link below

Re: Splunk dashboard using drilldown showing detai... - Splunk Community

I am wonder if maybe the solution would be not using the drilldown.

Does it make sense more sense to you?

Re: Dashboard - How to trigger second search based on first search where condition is : first result count

bowesmana — Sun, 20 Jun 2021 23:52:21 GMT

@Luciana

There seems to be some confusion.

I have not put ANY drilldown in my example dashboard. It shows two panels - the first panel runs a search and shows a count. When that search finishes, it will trigger the second search.

Please save that example dashboard as a new dashboard and you will see how it works.

There is no requirement to click anything to drilldown, as the second search runs on the completion of the first search.

This seems to provide the solution for your problem, i.e. NO drilldown is needed to make the second search run.

Re: Dashboard - How to trigger second search based on first search where condition is : first result count

Luciana — Mon, 21 Jun 2021 01:01:24 GMT

@bowesmana thanks for your suggestion. I tried and the token works perfectly I think I can adapt in my dashboard 🙂

Sorry, I got confused about the drilldown, now I understood that you had set the drilldown as 'none' 🙂

The only thing is , the second search will be exclusively for sending alerts with more info, so it will not be appearing in the dashboard, Is there a way to use the same logic but now show in the dashboard?.

Re: Dashboard - How to trigger second search based on first search where condition is : first result count

bowesmana — Mon, 21 Jun 2021 03:18:17 GMT

Two ways to hide the UI for that search. Either just set

in the XML for the second panel - in which case it will not show, or you can just take the whole of the

section from that table and put that in its own area in the XML before the first <row>

It will then run and is not part of any visualisation, so is never shown.

I find that it's often useful to use the first form, so if you want to 'debug' that query, it's easy just to define the token (in this case 'undefined') and you can then see how the search is running.

Re: Dashboard - How to trigger second search based on first search where condition is : first result count

Luciana — Mon, 21 Jun 2021 03:20:07 GMT

thanks @bowesmana you helped me a lot! 🙂 Have a nice week

Re: Dashboard - How to trigger second search based on first search where condition is : first result count

Luciana — Mon, 21 Jun 2021 05:54:38 GMT

Thanks @bowesmana I understood and it worked, but in the end, I ended up showing the results from my SECOND search in the bottom panels, I've changed a little the condition match but, now is happening again what you had advised me about once I set the token, the search will not run a second time if the first search runs again 😞 and I dont know how to fix it, because I understood the issue but, I might have not understood the logic. Would you mind to explain to me?

Check my code to be easier to understand:

<search id="main_search1">
<query>index = my index sourcetype=my sourcetype action=Allowed [|inputlookup OTX | search type=URL | rename indicator as url | table url] |dedup user |stats count</query>
<earliest>$time.earliest$</earliest>
<latest>$time.latest$</latest>
<refresh>3600s</refresh>
<done>
<condition match="'result.count'!= "0No event for this table"">
<set token="tkn_first_search">index = my index sourcetype= my sourcetype action=Allowed [|inputlookup OTX | search type=URL | rename indicator as url | table url] |dedup user | table _time, url, user, serverip, ClientIP |sendemail to="myemail@domain.co.nz" server=smtp.server.co.nz subject="OTX - Notification - IOC found by Domain" message="This is an test message" sendresults=true inline=true format=csv</set>
</condition>
<condition>
<unset token="tkn_first_search"></unset>
</condition>
</done>
</search>

for example, the stats count that was in RED , now has changed to green, which means there are 0 IOCs found, but the bottom panel continues to show me the oldest result and sending me emails.

thanks

Luciana, feeling ignorant with Splunk

Re: Dashboard - How to trigger second search based on first search where condition is : first result count

bowesmana — Tue, 22 Jun 2021 00:51:02 GMT

@Luciana

That code you show is not the code I suggested. In my example you see how I set the token - you should use that <eval> mechanism instead as it will always increment the token value, causing the rerun of the second search, rather than the example you are using from your similar question here in community, where you are only ever setting a constant token, so will never trigger the rerun.

Re: Dashboard - How to trigger second search based on first search where condition is : first result count

Luciana — Tue, 22 Jun 2021 04:00:13 GMT

@bowesmana , How are you? First, I want to thank you for helping me. I think there is a misunderstanding here.

So, the issue here is HOW to run a SECOND search IF the results in the FIRST search is =! 0, right?

However, in the example you gave me , you used my first search, instead of the SECOND.

my first search is:

but , my SECOND search is:

index = my index sourcetype= my sourcetype action=Allowed [|inputlookup OTX | search type=URL | rename indicator as url | table url] |dedup user | table _time, url, user, serverip, ClientIP |sendemail to="myemail@domain.co.nz" server=smtp.server.co.nz subject="OTX - Notification - IOC found by Domain" message="This is an test message" sendresults=true inline=true format=csv</set>

What I dont know is HOW to trigger a second search , if the results of the first one is !=0

You said that I should add the token in the SECOND search, which makes sense, but then, I didnt understand HOW to trigger my second search using

eval token="trigger">if(isnull($trigger$),0,$trigger$+1)</eval>

in your example, you used "stats count" in the 2 queries.

Can you clarify?

I feeling ignorant, but I cant help it. so, I tried to get the main idea.

so, in your example,

Re: Dashboard - How to trigger second search based on first search where condition is : first result count

bowesmana — Wed, 23 Jun 2021 04:07:09 GMT

The dashboard I gave you is an example so you could see HOW it works - I don't have your data, so cannot use your search in my example - you will use YOUR queries, not mine in your version of the dashboard.

This is the first query.

<query>index=_audit earliest=-1m latest=now | stats count </query>

Replace this query with your query.

The XML section

<done> <condition match="$job.resultCount$>0"> <eval token="trigger">if(isnull($trigger$),0,$trigger$+1)</eval> </condition> </done>

is the logic that you must use to set the token based on your condition. This is what you would need to test the field 'count' coming from your | stats count, i.e. result.count. This will increment the value of your trigger token.

<done> <condition match="$result.count$>0"> <eval token="trigger">if(isnull($trigger$),0,$trigger$+1)</eval> </condition> </done>

In YOUR second query, you need to include the $trigger$ token as part of your SPL search. That is HOW the second search will run - you don't do anything - Splunk will detect that the $trigger$ token has changed value, so will automatically rerun the search.

The simplest way to include that token in your search is to include the eval statement in your SPL like below

| dedup user | eval _dummy=$trigger$ | table....

Re: Dashboard - How to trigger second search based on first search where condition is : first result count

Luciana — Wed, 23 Jun 2021 05:10:15 GMT

@bowesmana , yes, now I understood, and it worked, thank you so much for your patience.