Dashboard element with runtime search behaviour

[lnn2204]

Hi guys,

I'm using ipinfo to check IPs of my system.

<base search>
| stats sum(Download) as Download by DestIP 
| sort 5 -Size
| ipinfo DestIP

The problem is it didn't wait for the final result then call the command "ipinfo", it'll make more request than 5 times, depends on how much DestIP it had.

Are there any solutions for this case?

[ITWhisperer]

Firstly Size doesn't exist so the sort won't do anything (of course, this could just be a typo in your example)

Even so, you should only get 5 events after the sort.

Try inserting a table command

<base search>
| stats sum(Download) as Download by DestIP 
| sort 5 -Download
| table DestIP Download
| ipinfo DestIP

[lnn2204]

the table command was put after "ipinfo" commands, so I think it's totally fine.

And yes, I got just 5 row in the table, but the request was made to Ipinfo API is more than 5. Depends on how many DestIP user surfed.

[ITWhisperer]

Even if you think it might be totally fine, it might be worth checking - just saying 😁

[lnn2204]

Checked it, not work hehe.

Do you have another solutions?

[PickleRick]

Please be a bit more specific because there doesn't seem to be anything wrong with your search.

The "sort 5 - Size" part should give you 5 results and ipinfo should perform calls for those 5 results. If it's making more than one IPinfo API call per result row, that's nothing you can do withouth fiddling with internals of the app.

[lnn2204]

Ah, you can think about Splunk will do sort 5 ips, but the event will be indexed one by one follow the time of its. And in that moment, the top 5 is the top, but 5s later, the top 5 will be changed, so it will do10 requests instead of 5 request at the final

[PickleRick]

Honestly, I don't understand the issue.

If you run the same search against different sets of data (for example, because of running it for different time ranges), you might get different results. So those "top 5 IPs" might indeed be different.

There are a few possibilities that could let you store some kind of "global state" but firstly you'd have to think what would be the expected behaviour if - for example - you got 5 IPs at the first run of your search and 3 "old" and 2 "new" IPs at next run.

[lnn2204]

I don't know how to detail my issue.

Like you search this in 5min:

| eval RespondSize=round(('RespondSize'/1024)/1024,2)
| search SrcIP="$ipDownloadDetail$" 
| stats sum(RespondSize) as Size by DstIP 
| sort 5 -Size
| ipinfo DstIP
| table DstIP, org, hostname, country, Size

And when it had run for 2-3 mins, the top 5 will be set, and when it run to the final, the top 5 will be changed. So when top 5 are set at first time, they will call the API with 5 requests, and when it run to the final stage, another 5 requests will be made

[PickleRick]

I still don't understand. If you change your search, change its conditions or time range, you can get different results. That's how it works. So what's surprising here?

Also if you're running two separate searches with ipinfo command in them - the command will get run separately for each result set.

Again - honestly, I have no idea what other behaviour you expect and why.

[lnn2204]

Okay, tks for your replying. I'll try to figure it out.

[ITWhisperer · Accepted solution]

Is this search in a dashboard? I noticed that you have a token. Often, searches in dashboards are executed multiple times as tokens are given different values (searchWhenChanged). Is this perhaps what is going on?

[lnn2204]

Yeah, I think it's the problem. It's in dashboard, with realtime search and a drilldown which to change the token then this dashboard can run. But my token will not change when it's running. So in my mind now, maybe about the realtime search.

[PickleRick]

So that's a completely different problem. It's not about search as such but about the fact that when you're running a realtime search the results are indeed being updated as the search progresses.

That's "by design". As @ITWhisperer already pointed out, real-time searches are not supposed to be used often. In fact, it's best to avoid them altogether (there are very few use cases when they are indeed justified).

With a normal historical ad-hoc search, ever run of your search will produce a set of results which will be passed to your command which calls external IP and that's it. But remember that if you have a refresh set on your dashboard, every time the dashboard refreshes, the search will get re-run however so your API calls will be performed again. That's simply how it works 🙂

[ITWhisperer]

Realtime searches tie up cpus and should be used sparingly. Does your dashboard need a realtime search? Can you split off the "realtime" part of the dashboard to another dashboard? Can you use short refresh cycles to produce "near realtime" results which you can then tune to an appropriate value? How near to realtime does the dashboard need to be? Do you have someone watching the dashboard every second of every minute of every hour of every day? How quickly do they need to respond to something they see in the dashboard?

[lnn2204]

Thank you, I'll think about that