I installed the Splunk for Palo Alto Networks app. I am getting data and my index and source types are correct. When I do searches, all the PA fields are getting extracted.
However, I only the Overview dashboard works; it displays real-time information.
The other dashboards and sub-dashboards under Traffic, Threat, Content and System all say "Search is waiting for input..." and the drop downs all say "Search produced no results."
We are using a cluster so the app in installed on the heavy forwarder that receives the logs and a search head that can search all of our indexers.
EDIT: Just realized that the heavy forwarder is still running v6.0.3. Maybe that's the issue. Upgrading tonight to find out.
Didn't get to upgrade the forwarder but I don't see why that would cause an issue anyway.
If I use Pivot or go to PAN App search and enter (with back quotes around the search)
I get the following error from all of my indexers.
[indexserver] The search for datamodel 'panlogs' failed to parse, cannot get indexes to search
Yet there are 300gb in /opt/splunk/var/lib/splunk/panlogs/datamodelsummary on all of my indexers.
I don't think this app works if your indexers are clustered.
I installed the app on my search head pool, heavy forwarders and indexers. As I stated above, on the search heads, I only get data in the Overview Dashboard.
Using the app on the indexers, I get data on all of the dashboards but it's fairly useless because I only get the data that's on that singe indexer in the cluster.
I opened a case with splunk. The built in data models work but they aren't accelerated.
When I turn off acceleration in the PAN App, I don't get the errors from my indexers. Of course the pivots will take forever and the dashboards relay on acceleration so that's useless but at least I can now assume that the problem is data model acceleration.
Trouble shooting with Splunk showed that I can go to the PAN App Search and enter "| datamodel pan_logs" and get results back.
I also enabled acceleration on the built in apps and they worked.
Support says the problem is in the App.
Looking at the search.log on the indexer shows, that the macro can not be found on the indexer:
06-30-2014 14:49:08.773 ERROR TsidxStats - Error in 'SearchParser': Could not find macro 'panindex' that takes 0 arguments. Expecting stanza name 'panindex'.
06-30-2014 14:49:08.773 INFO TsidxStats - Could not obtain a valid set of indexes to search
I fixed the problem with modifying the data model root object constraint from "
pan_index" to "index=pan_logs".
Thanks my2ndhead! That fixed it. It looks like the macro is not working so explicitly setting the root constraint to index=pan_logs "fixes" that.
If you're having this problem, here are the steps to fix it.
pan_index" to index=pan_logs and save.
All of the dashboards are working now.
This change is no longer needed in version 4.1.2 and higher. These versions of the Palo Alto Networks app contain the change already.
Thanks for this, but let me add, if you have a search head and multiple indexers, make the change on your search head, re-deploy the updated app to your indexers so they all receive the updated data model.
Thanks dfronck - your solution helped!