Issue is that none of the dashboards (except Overview) is populating with data. I have reviewed all of the help articles I could find as well as the official Troubleshooting page. What I have done:
NOTE: we are using Splunk cloud
pan_logspopulates events and uses the eventtype="pan".
I do not know what the problem could be. I did not originally setup the app and I don't think it ever worked properly. Since the 'Overview' dashboard does work, I originally thought it was a data model problem but those seem to be working fine.
Any help is greatly appreciated.
I will add that I am currently waiting on Splunk support to change the props.conf file on the indexers so that the time zones properly match. The issue is that our firewall logs are in UTC and Splunk is seeing them as EDT. This makes it so the only way to search for current logs is to force a search 4 hours into the future (we are UTC -4).
Would this be the issue as to why our dashboards are not populating? I figured that they would still populate, but simply have data that is behind by 4 hours.
Changes to the props.conf file did fix our time issue but the dashboards still are not populating.
I have also tried to open some of the searches to see the SPL, but clicking on the magnifying glass does nothing. I get no error and no additional window with the search.
However, when I inspect the search, I get an error stating "Unknown sid." I have let the screen run for as long as 20 minutes and still nothing populates (I throw in a source IP which shows up in the "Threat" logs and should show up in the dashboard). I have already tried rebuilding the data models multiple times and those appear to be working correctly (we have the Endpoint Logs, Firewall Logs, and WildFire and all are at 100%).
Most of the dashboards work fine now but having an issue with URL filtering (setting Palo Alto inputs.conf to TZ=UTC resolved most of the issues). Going to ask an additional question since much has changed since I originally asked this one.