topic Re: Run command after having result in Dashboards & Visualizations

Dashboard element with runtime search behaviour

lnn2204 — Wed, 01 Jun 2022 09:05:00 GMT

Hi guys,

I'm using ipinfo to check IPs of my system.

<base search> | stats sum(Download) as Download by DestIP | sort 5 -Size | ipinfo DestIP

The problem is it didn't wait for the final result then call the command "ipinfo", it'll make more request than 5 times, depends on how much DestIP it had.

Are there any solutions for this case?

Re: Run command after having result

PickleRick — Tue, 31 May 2022 08:10:29 GMT

Please be a bit more specific because there doesn't seem to be anything wrong with your search.

The "sort 5 - Size" part should give you 5 results and ipinfo should perform calls for those 5 results. If it's making more than one IPinfo API call per result row, that's nothing you can do withouth fiddling with internals of the app.

Re: Run command after having result

lnn2204 — Tue, 31 May 2022 08:25:33 GMT

Ah, you can think about Splunk will do sort 5 ips, but the event will be indexed one by one follow the time of its. And in that moment, the top 5 is the top, but 5s later, the top 5 will be changed, so it will do10 requests instead of 5 request at the final

Re: Run command after having result

ITWhisperer — Tue, 31 May 2022 08:30:57 GMT

Firstly Size doesn't exist so the sort won't do anything (of course, this could just be a typo in your example)

Even so, you should only get 5 events after the sort.

Try inserting a table command

<base search> | stats sum(Download) as Download by DestIP | sort 5 -Download | table DestIP Download | ipinfo DestIP

Re: Run command after having result

lnn2204 — Tue, 31 May 2022 08:36:56 GMT

the table command was put after "ipinfo" commands, so I think it's totally fine.

And yes, I got just 5 row in the table, but the request was made to Ipinfo API is more than 5. Depends on how many DestIP user surfed.

Re: Run command after having result

ITWhisperer — Tue, 31 May 2022 08:57:12 GMT

Even if you think it might be totally fine, it might be worth checking - just saying 😁

Re: Run command after having result

PickleRick — Tue, 31 May 2022 10:13:12 GMT

Honestly, I don't understand the issue.

If you run the same search against different sets of data (for example, because of running it for different time ranges), you might get different results. So those "top 5 IPs" might indeed be different.

There are a few possibilities that could let you store some kind of "global state" but firstly you'd have to think what would be the expected behaviour if - for example - you got 5 IPs at the first run of your search and 3 "old" and 2 "new" IPs at next run.

Re: Run command after having result

lnn2204 — Tue, 31 May 2022 10:52:26 GMT

I don't know how to detail my issue.

Like you search this in 5min:

And when it had run for 2-3 mins, the top 5 will be set, and when it run to the final, the top 5 will be changed. So when top 5 are set at first time, they will call the API with 5 requests, and when it run to the final stage, another 5 requests will be made

Re: Run command after having result

lnn2204 — Tue, 31 May 2022 10:55:30 GMT

Checked it, not work hehe.

Do you have another solutions?

Re: Run command after having result

PickleRick — Tue, 31 May 2022 18:36:54 GMT

I still don't understand. If you change your search, change its conditions or time range, you can get different results. That's how it works. So what's surprising here?

Also if you're running two separate searches with ipinfo command in them - the command will get run separately for each result set.

Again - honestly, I have no idea what other behaviour you expect and why.

Re: Run command after having result

lnn2204 — Wed, 01 Jun 2022 03:42:11 GMT

Okay, tks for your replying. I'll try to figure it out.

Re: Run command after having result

ITWhisperer — Wed, 01 Jun 2022 04:39:18 GMT

Is this search in a dashboard? I noticed that you have a token. Often, searches in dashboards are executed multiple times as tokens are given different values (searchWhenChanged). Is this perhaps what is going on?

Re: Run command after having result

lnn2204 — Wed, 01 Jun 2022 04:52:01 GMT

Yeah, I think it's the problem. It's in dashboard, with realtime search and a drilldown which to change the token then this dashboard can run. But my token will not change when it's running. So in my mind now, maybe about the realtime search.

Re: Run command after having result

ITWhisperer — Wed, 01 Jun 2022 05:22:48 GMT

Realtime searches tie up cpus and should be used sparingly. Does your dashboard need a realtime search? Can you split off the "realtime" part of the dashboard to another dashboard? Can you use short refresh cycles to produce "near realtime" results which you can then tune to an appropriate value? How near to realtime does the dashboard need to be? Do you have someone watching the dashboard every second of every minute of every hour of every day? How quickly do they need to respond to something they see in the dashboard?

Re: Run command after having result

lnn2204 — Wed, 01 Jun 2022 05:34:05 GMT

Thank you, I'll think about that

Re: Run command after having result

PickleRick — Wed, 01 Jun 2022 09:03:39 GMT

So that's a completely different problem. It's not about search as such but about the fact that when you're running a realtime search the results are indeed being updated as the search progresses.

That's "by design". As @ITWhisperer already pointed out, real-time searches are not supposed to be used often. In fact, it's best to avoid them altogether (there are very few use cases when they are indeed justified).

With a normal historical ad-hoc search, ever run of your search will produce a set of results which will be passed to your command which calls external IP and that's it. But remember that if you have a refresh set on your dashboard, every time the dashboard refreshes, the search will get re-run however so your API calls will be performed again. That's simply how it works 🙂