All Apps and Add-ons

Cortex Data Lake - missing field in logs breaks parsing

TitanAE2020
Loves-to-Learn

I am currently using the Splunk TA for Palo Alto data.  And I'm ingesting data from Cortex Data Lake to a new Azure Syslog server.

But there is a large problem with the data we're ingesting.  The data being sent is literally missing a single field.  Below is a reference of what we should be ingesting:

Configuration Syslog Field Order (paloaltonetworks.com)

If you look at the example from this link, you will see this log:

 

Oct 13 20:56:15 gke-standard-cluster-2-pool-1-6ea9f13a-fnid 394 <142>1 2020-10-13T20:56:15.519Z stream-logfwd20-156653024-10121421-eq28-harness-16kn logforwarder - panwlogs - 1,​2020-10-13T20:56:03.000000Z,​007051000113358,​CONFIG,​config,​,​2020-10-13T20:56:00.000000Z,​xxx.xx.x.xx,​,​rename,​admin,​,​submitted,​/config/shared/log-settings/globalprotect/match-list/entry[@name='rs-globalprotect'],​150,​-9223372036854775808,​0,​0,​0,​0,​,​PA-VM,​,​,​,​2020-10-13T20:56:00.284000Z

But what I'm receiving is:

Oct 13 20:56:15 gke-standard-cluster-2-pool-1-6ea9f13a-fnid 394 <142>1 2020-10-13T20:56:15.519Z stream-logfwd20-156653024-10121421-eq28-harness-16kn logforwarder - panwlogs - ​2020-10-13T20:56:03.000000Z,​007051000113358,​CONFIG,​config,​,​2020-10-13T20:56:00.000000Z,​xxx.xx.x.xx,​,​rename,​admin,​,​submitted,​/config/shared/log-settings/globalprotect/match-list/entry[@name='rs-globalprotect'],​150,​-9223372036854775808,​0,​0,​0,​0,​,​PA-VM,​,​,​,​2020-10-13T20:56:00.284000Z

In the log I'm receiving, I'm missing a comma (,) before the 2020 in this line:

harness-16kn logforwarder - panwlogs - ​2020-10-13T20:56:03.000000Z
 
I should be receiving data that looks like this:
 
harness-16kn logforwarder - panwlogs - 1,​2020-10-13T20:56:03.000000Z
 
I'm at a loss where this data is generated.  If this is data generated by a syslog server, the unix server hosting syslog, or if it's on the Palo Alto Cortex Data Lake side of things. The logs are passed through our firewall and towards the syslog server.
 
I believe this field is either 'log_source_id' or 'log_type.value'.  But outside of that, I'm at a loss as to where this value is generated.  Any help is appreciated.
 
- TitanAE

 

 

Labels (1)
0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Unlock What’s Next: The Splunk Cloud Platform at .conf25

In just a few days, Boston will be buzzing as the Splunk team and thousands of community members come together ...