I would say about 1/3 of the time and it falls into the following categories:
o Rarely, we need to prove that Splunk itself is wrong so we go into another system that has a different view on the same data and if it is different (one system is wrong) we need to dig deep in both systems to figure out why.
o We have not (yet) put the other data that we need to see inside Splunk.
o We need to enable debugging to enhance logging in some system (need more detail for stuff already in Splunk).
o We have pointed the finger at a system and we need to login to that system to check the configurations that control it (confirmation/resolution stage).
o We think we understand the problem and the resolution but we need to reproduce it in a test system and then test the fix (policy requirements in confirmation/resolution stage).
... View more