Palo Alto Add-on CIM Network_Traffic, can we map only end events?

[MonkeyK]

Palo Alto traffic logs include start and end events. Sometimes multiple start events.
Since all traffic logs get the tags "network" and "communicate", they all get mapped to the Network_Traffic datamodel This is problematic for a couple of reasons:

1. The number of events is inaccurate (at least 2x the actual number of events, but sometimes more)
2. This is made worse if we had an additional source that logged Netflow (with only one event per connection)
3. Traffic summaries are inaccurate:
4. If I try to find allowed traffic, I will find some traffic that was actually blocked. this is because the start event may be allowed when the end event is blocked.
5. If I try to summarize on durations or bytes, I get values at each start and end added into my totals

It seems to me that these could be corrected by only mapping the PAN end event to the Network Traffic datamodel.

I guess that I am looking for feedback on the best way to do this. And what the drawbacks to doing so might be.
If it matters, I am using Splunk Enterprise Security, but mostly do my own correlations of Network Traffic in the search app.

My thoughts on how to do it:
-add the "end" tag to Network_Traffic
--if I do this then any other source with events to map to Network Traffic would also need to add the "end" tag. Seems unintuitive
-remove "communicate" from the PAN traffic "start" events
--not really sure what this might do
-add an additional tag to the PAN traffic "end" events ("traffic"?)
--add this to the Network_Traffic defn
--modify all sources that map to Network Traffic to include the "traffic" tag. I guess this is more intuitive than the first option.

Any other ideas or thoughts?

[panguy]

I assume you are sending logs at the start and end of the session. This is configured in PAN-OS as part of a security policy. I recommend sending at session end. This will reduce the amount of duplicates. You will still get all the logs from the start to the end of the session.

[MonkeyK]

Panguy,

I think that you are saying that I can get less start/end records if I remove "log at session start" from the policy. This is a good thing and I will bring it up with my firewall administration team.

I do not think that it will address the problem with the datamodel though. The datamodel will still include 2 Network_Traffic events for every Pan logged session. Is there a best way to incorporate only the end events in the datamodel?

[lakshman239]

You could filter the events with only 'end' events and send it to indexers, so the datamodel maps it correctly. would that help in your case?

[MonkeyK]

That might be an option. I'll bring it up with my Splunk Admin

[lakshman239]

Did that help resolve the issue?

[MonkeyK]

Panguy's suggestion addressed my need. Turns out that it is PA best practice to not log traffic start events. I was under the impression that my MSSP required that we log them, but when challenged, they said that they did not.

Firewall admins have disabled log traffic start and we have about a 40% reduction in Firewall log ingestion! I am also still pursuing having my Splunk admin filter out the logs to handle the situations where the Firewall admins may need to turn on start logging for troubleshooting.

[lakshman239]

is the issue resolved now?

[MonkeyK]

Or will removing "log at session start" result in only "end" events getting logged?