How to get Palo Alto app transforms working?

[rawillacxiomcom]

Need someone to point out where I have made my mistake.

Env structure:
Network devices are sending data via syslog to a UF.
UF is sending on to a set of indexers.
App has been deployed to a search head and to the indexers.

I've verified that props.conf and transforms.conf are on my indexers at: /opt/splunk/etc/apps/SplunkforPaloAltoNetworks/default
The app is enabled on both the indexers and search head. I've cycled splunkd on the search head and on the indexers, but my logs are not being source typed correctly or at all. Where have I made my mistake?

[rawillacxiomcom]

Just to test, I appended the contents of both the props.conf and transforms.conf to the same files located here: /opt/splunk/etc/system/local. I also copied all of the lookup csv files to the lookup dir under /system. Cycled splunk on my indexers. I'm still getting the same results, the data is going to the correct index with an incorrect sourcetype.
Do I need to push this all the way our to the forwarder?

[joshd]

If they are Universal Forwarder's then no, this wont help as they cant do any parsing. If they are heavy-weight forwarders then yes the app would need to exist there as it would not work being on the indexers. Please see my comment below though.

[dmartinez_splun]

Hey mate, I have a very similar issue... https://answers.splunk.com/users/342662/dmartinez-splunk.html and the customer is using a Heavy Forwarder. Am I forced to install the all in the HF? Or is there a way for me to do it at the Indexer?

[joshd · Accepted solution]

I believe the Palo Alto app expects the data to originally come in with a sourcetype of 'pan_log' ... so if your syslog data is coming in as sourcetype=syslog and not sourcetype=pan_log then the transforms would not be taking affect.

You can do one of two things:
1. Create specific inputs.conf definition(s) to sourcetype it with pan_log for those files where the Palo Alto devices are having their events written.
2. Copy the props.conf in the Palo Alto app from the default directory to the local directory then open it up and change the line that says: [pan_log] ...to: [syslog]

[rawillacxiomcom]

Based on my limited understanding of Splunk apps I would have to create multiple entries in the inputs.conf since the dashboards are expecting different sourcetypes based on the incoming data from the network devices. I don't think that is how the app is supposed to work, but if that is what I have to do then I will do it.

[joshd]

Yes, the app is expecting the data to come in with the sourcetype=pan_log as I mentioned above. So for both the source type renaming and field extractions to work, the data would need to come in with this sourcetype. Otherwise as I mentioned in #2, you can edit the app itself to change the sourcetype it's expecting.

Currently what does your inputs.conf look like?

[rawillacxiomcom]

That was it there was a typo in the inputs.conf.

[rawillacxiomcom]

In this case it was source vs sourcetype.