Can someone help me parse this XML into Splunk on the backend? Also should I add this to the props.config in my Deployment Server or my Search Head Deployer?

[rshoun]

Here is what I have tried and it is not working:

Edit the local/inputs.conf file and add this:

[monitor:///directory/*.xml]
sourcetype = panorama:api:templates
index =  qt_palo
crcSalt = Belton Palo-Templates
alwaysOpenFile = 1
disabled = false

[monitor:///directory/*.xml]
sourcetype = panorama:api:templates
index =  qt_palo
crcSalt = Tulsa Palo-Templates
alwaysOpenFile = 1
disabled = false

[monitor:///directory/*.xml]
sourcetype = panorama:api:devicegroups
index =  qt_palo
crcSalt = Belton Palo-Templates
alwaysOpenFile = 1
disabled = false

[monitor:///directory/*.xml]
sourcetype = panorama:api:devicegroups
index =  qt_palo
crcSalt =  Belton Palo-Device Groups
alwaysOpenFile = 1
disabled = false

Create a local/props.conf file and add this:

[panorama:api:templates]
EXTRACT-Status = (?i)<response status>(?P<Status>[^<]+)
EXTRACT-Name = (?i)<entry name>(?P<Name>[^<]+)
EXTRACT-TemplateStack = (?i)<template-stack>(?P<TemplateStack>[^<]+)
EXTRACT-EntryName = (?i)<entry name>(?P<EntryName>[^<]+)
EXTRACT-Serial = (?i)<serial>(?P<Serial>[^<]+)
EXTRACT-Connected = (?i)<connected>(?P<Connected>[^<]+)
EXTRACT-UnsupportedVersion = (?i)<unsupported-version>(?P<UnsupportedVersion>[^<]+)
EXTRACT-LastCommitAllStateTPL = (?i)<last-commit-all-state-tpl>(?P<LastCommitAllStateTPL>[^<]+)
EXTRACT-LastValidateAllStateTPL= (?i)<last-validate-all-state-tpl>(?P<LastValidateAllStateTPL>[^<]+)


EXTRACT-Connected = (?i)<XMLtagInRawEvent1>(?P<YourXMLExtractionName1>[^<]+)
EXTRACT-Connected = (?i)<XMLtagInRawEvent1>(?P<YourXMLExtractionName1>[^<]+)
EXTRACT-Connected = (?i)<XMLtagInRawEvent1>(?P<YourXMLExtractionName1>[^<]+)
EXTRACT-Connected = (?i)<XMLtagInRawEvent1>(?P<YourXMLExtractionName1>[^<]+)
EXTRACT-Connected = (?i)<XMLtagInRawEvent1>(?P<YourXMLExtractionName1>[^<]+)
EXTRACT-Connected = (?i)<XMLtagInRawEvent1>(?P<YourXMLExtractionName1>[^<]+)
EXTRACT-Connected = (?i)<XMLtagInRawEvent1>(?P<YourXMLExtractionName1>[^<]+)

[skoelpin]

One thing to note here. Your deployment server will make changes to your forwarders while the deployer will make changes to your search head members in the cluster. Neither of these are responsible for parsing the data, you will need to make changes to your indexer(s) (aka search peers).. If your in an index cluster then you will make changes on the Master Node (aka the cluster master). These changes then will take affect on each search peer

To do this, you need to edit props.conf in $SPLUNK_HOME/etc/master-apps/_cluster/local/props.conf on master and push cluster-bundle with command 'splunk apply cluster-bundle'. Peers will restart and props.conf, in $SPLUNK_HOME/etc/slave-apps/_cluster/local/props.conf, will be layered when splunkd start.

[rshoun]

I just thought of something, will I need to add all of my XML (all the code above: ie: "[monitor:", etc. - all my XML extracted fields) to the props.conf Master Node Indexer as well? Or I just need to add the KV_MODE =1 to my Master indexer correct? I should not need to add everything else here or am I wrong? (Please let me know if my question does not make sense to you.)

[skoelpin]

It should auto-extract most of the fields without having to explicitly extract them, but yes you just need to apply KV_MODE = 1 to your master node and it will apply those config settings to all of your search peers. If the fields aren't auto-extracted then you can apply these extractions to $SPLUNK_HOME/etc/master-apps/_cluster/local/props.conf and push it out to all your search peers. If you pushed this out to only 1 indexer then only 1 indexer will have these fields available which will give bad data output

[rshoun]

Oh okay! That makes more sense! Geeze, I am really glad you are here! Thank you so much!

[rshoun]

Wow, you have been an outstanding resource! I will do this! Thank you sooo much for adding this extra content!

[skoelpin · Accepted solution]

Are you having issues parsing the XML file? If so you need to include KV_MODE =1 in your props.conf on the indexer(s)

[rshoun]

I was wondering if that was the problem! Let me try the KV_MODE =1 and then I will see if that corrects the issue. Thanks!

[skoelpin]

This will defiantly fix your issue. Make sure to restart splunkd after making changes to props.conf

[rshoun]

Dang it. I knew a restart was going to be necessary. Should I restart all places where my props.conf and inputs.config are located as well or just the place where the index KV_MODE =1 will be applied?

[skoelpin]

Yeah change management is tough to get around.. You should restart splunkd on every indexer you made changes to (You shouldn't have to make changes to inputs.conf on the indexer)

How many search peers are in your index cluster? I would strongly recommend testing this in a dev environment and verify it's working before making changes in production. You could download Splunk on your local machine, apply your props.conf changes and restart splunkd to verify the correct changes applied.

[rshoun]

We have 7 search peers in our index cluster, if you count the index cluster master. The dev environment idea is actually a really good idea. I will see if I can get my team on board with this. I absolutely agree with the dev environment idea but not everyone will always understand the logical need for this. In the meantime, your idea of doing of local Splunk download is actually a really good idea! Thank you for your help!