Why does my real time search job keeps getting killed?

[robertlynch2020]

Hi

I have a real time search over the past 5 minutes, however it works for 30 seconds an then it dies.
any ideas?

I have this search at the top of my HOME page users log in and see data flowing into the system from there hosts.

Thanks in advance
Robert 🙂

[robertlynch2020 · Accepted solution]

Hi

So in the end i found the issue.
At the top of my dashboard i had the following refresh="30". When the first refresh comes into play, it killed the real time search i was running. But not all of them, i was running 2, only one stopped.

1st was killed - index=mlc_live | stats count as Events by _time host | timechart span=5s count(Events) as Events by host.
I was running it into a column chart.

2nd did not die (not sure why) index=mlc_live and i am was pursing it out to Events visualization

So i am assuming this is note the way it is suppost to be, but removing the refresh makes it go away.

cheers to all for there suggestions

Rob

[woodcock]

WOW! That is brilliant. Thank you for sharing (this might come up for me later). Be ware that you can set refresh on each panel instead of globally at the top.

[chrisyounger]

Does it happen even if you leave the tab in the foreground?

These days browsers like Chrome aggressively throttle backgrounded tabs. This can cause the real-time search to think you have closed the window and it will auto-cancel the search. I was able to just replicate this with a lot of tabs open and backgrounding the tab for 30 seconds. There are workarounds you can do to Chrome to prevent this behaviour.

[woodcock]

When you figure out what is doing that, post back here and let us know. I would like to deploy whatever it is to every Splunk Search Head that I administer!!!!

[skoelpin]

I like your way of thinking.. It's a feature not a bug!

[skoelpin]

Have you confirmed that your hardware can handle the realtime search? What does the internal logs say when it gets killed?