Would like to know what causes this issue , please see screenshot attached.
There's an event "42" showing and time range is showing , but the table is not showing.
SplunkEnterpriseSecuritySuite = version :5.3.0
Check your kvstore status. mongod.log generally has messages that can give you more information if it is a kvstore startup issue. Usually if you do a recursive chmod you can give too much access to the relevant key file. That's usually what gets me. The splunk.key file for mongo needs to be owner read-only 'chmod 400 ./splunk.key'
Did you just upgrade or make UI changes?
Sometimes there can be compatibility issues with custom view settings.
My guess is that you are hitting a bug:
SPL-189084 " /services/search/jobs returns empty results"
SOLNESS-22669 "Incident Review Filters causing results table to break"
Your best option here is file a support case and reference this number so we can do some research.
Ummm.. could you please check what you get if you run
index=notable from the search interface ? That will help you make sure that those notables are actually populated and not empty events.
index=notable is working then try this to ensure that events from incident review are there :
Also play around with the time picker to see if you can see older events on both searches and on the incident review page.
@DavidHourani - search is returning results, and when i change the time picker = time range shows the event count per day - but the table is not showing anything.
Also the pagination is showing. tried to change page still not showing.
@jadengoho, have you done any upgrades for ES recently ? Or have you changed any permissions for your user recently ?
There is a known issue for ES 5.3.0 where mis-configured roles might lead to the incident review page not loading :
Issue : SOLNESS-21783
@jadengoho, that's weird ... this "sometime the table shows but most of the time it's not showing" is most of the times due to cache... What browser are you using ? And could you try to change it ?
@DavidHourani - we are using Internet Explorer version 11.09. We can't use other browser [IT setup that way].
@MuS hahahha now i get it 42 is the real deal
i do audits on ES but nothing really .
@jadengoho, then in that case next time you face the issue please try hitting the
And make sure you didn't set up splunk on deep thought. This could be why you're getting 42. @MuS can confirm.
This ^^^ or permission issues or ... anything else that could cause an error in ES.
Did you check all the internal logs of Splunk to see if you get errors when opening the 'Incident Review'?
Any other error in any other log files?
As @DavidHourani has asked, did you recently upgraded and did you restart Splunk after that?
Have to tried to
_bump the Splunk instance?
I could add so many things to this list, but without more details we will never be able to help.
PS: It looks like you did not get my previous joke about 42 😉
Did you check all the internal logs of Splunk to see if you get errors when opening the 'Incident Review'? Yes i investigated it , think all Error and Warn are really not related to the issue like
did you recently upgraded and did you restart Splunk after that? im not the one who upgrade it 1yr ago, but i saw in the process that it has a restart.
Have to tried to _bump the Splunk instance? Not yet , will this once the issue occur again
It looks like you did not get my previous joke about 42 😉 - HAHAHA still didn't get it.
And you have some search errors there; on ES that could indicate a problem - just saying ...