Palo Alto Networks Empty Dashboards

[scott778]

I've read through the documentation, followed all the steps but still cannot get dashboards to populate in Splunk for the Palo Alto App.

Versions -
Splunk - 7.0.1
Palo Alto Networks - 6.0.1
Palo Alto Networks Add-On - 6.0.1

Inputs.conf -

[udp://XXX]
index = pan_logs
sourcetype = pan:log
no_appending_timestamp = true


Data Model Acelleration is at 100% - There was an error preventing the data models from functioning 100%, related to system not having proper NTFS access.

eventtype=pan - returns results
eventtype=pan_config - no results
According to Documentation we should check the timestamp for upd which we've done, We are forwarding straight from the Palo Alto Firewalls with default format. We only have one splunk server running all roles.

I've found this in the _internal index -

CalcFieldProcessor - Invalid eval expression for 'EVAL-url_length' in stanza [pan:threat]: The expression is malformed. Expected LIKE.

I've no other ideas why data isn't being tagged properly. Any and all help is appreciated.

[btorresgil]

Try upgrading to add-on 6.0.2 which fixes some lookup issues.

Then try the dashboard troubleshooting guide here:
https://splunk.paloaltonetworks.com/troubleshoot.html#dashboards-not-working

[shirishkamat84]

Can you specify which dashboards are not working. There are 2 main things i would be looking:

1. Make sure you Update App Metadata from Content Pack, follow the steps mentioned here https://splunk.paloaltonetworks.com/lookups.html
2. Check all the macros used by the dashboard are populating the required results, this may cause issues post upgrade if you are not using the default sourcetypes and eventtypes.

[micahkemp]

Are your events being properly divvied up to the other sourcetypes (pan:threat, pan:traffic, etc)? What are the results of:

| tstats values(sourcetype) WHERE index=pan_logs

[scott778]

Results over last 30 days. We've been doing a bunch of commits recently so I know I should be seeing some pan_config

pan:system
pan:traffic

[micahkemp]

Which sourcetype is set in inputs.conf?

The Palo Alto TA expects events to come in initially as pan:log, and then it adjusts the sourcetype to be more specific per event. I'm just hoping to validate that's what's actually happening.

[scott778]

This is my inputs.conf -

[udp://XXX]
index = pan_logs
sourcetype = pan:log
no_appending_timestamp = true

It's definitely adjust some stuff because I get events tagged with pan:traffic which is not defined in any inputs.conf.

sourcetype pan:traffic

[micahkemp]

Can you find any events that should have been tagged as pan:config? This search might help:

index=pan_logs | rex "^[^,]+,[^,]+,[^,]+,(?<palo_config>CONFIG)," | search palo_config=*

[scott778]

No results are returned. I believe this is related to the error when parsing (i think) the props.config for sourcetypes? This error in _internal concerns me. Maybe splunk is choking on pan:threat and so stops processing other Evals?

CalcFieldProcessor - Invalid eval expression for 'EVAL-url_length' in stanza [pan:threat]: The expression is malformed. Expected LIKE.

Here's a search over the last 30 days of sourctypes -

Search - index=pan_logs | dedup sourcetype | table sourcetype

sourcetype
pan:traffic
pan:system

[micahkemp]

My seach above doesn't rely on any of the Palo Alto TA, but the regex I included was one from that TA. If you aren't getting results from that search it tells me you probably don't have any CONFIG lines in your events.

Can you search:

index=pan_logs | rex "(?<config>CONFIG)" | search config=*

This will tell you if you have any events at all with the literal string CONFIG. If you don't, those events aren't making it to Splunk, so the Palo Alto TA has nothing to work with.

[scott778]

index=pan_logs | rex "(?CONFIG)" | search config=*

No results returned over last 30 days.

[btorresgil]

Looks like you're using a index you created. Can you verify your user is set to search this index by default in the user settings?

[scott778]

Confirmed, my account is a member of the role 'admin'

Role admin has "all non-internal indexes' in default search

I confirm this by searching on eventtype=pan in the search app and results are returned.