Getting Data In
Highlighted

Is it possible to assign different timestamps based on log line contents within the same sourcetype?

Path Finder

I am sending "pan:traffic" logs from our Palo Alto 3050 firewall to Splunk. I want the "time" fields to be the same value as the "starttime" field when the log line contains "start" and use the time stamp that follows "PA-3050 1," when the log line contains "end"

Is it possible to do different time stamps for events in the same source-type "pan:traffic" ? I've bolded the time stamps I want to use for each in the samples below:

Sample start log line:
Feb 14 09:07:25 PA-3050 1,2017/02/14 09:07:25,001701007055,TRAFFIC,start,0,2017/02/14 09:07:25,10.0.0.1,77.000.000.88,0.0.0.0,0.0.0.0,User-to-Ext-Allow-DLP,domain\user,,web-browsing,vsys1,trust,untrust,ethernet1/2,ethernet1/1,LF-LOG-ALLOW,2017/02/14 09:07:25,55537,1,58861,80,0,0,0x0,tcp,allow,740,678,62,4,2017/02/14 09:07:24,0,any,0,442739216,0x0,10.0.0.0-10.255.255.255,US,0,3,1,n/a,0,0,0,0,,PA-3050,from-policy

Sample end log line:
Feb 14 10:21:25 PA-3050 1,2017/02/14 10:21:24,001701007055,TRAFFIC,end,0,2017/02/14 10:21:24,10.1.1.2,8.8.8.8,0.0.0.0,0.0.0.0,IntDC-to-ExtDNS,,,dns,vsys1,trust,untrust,ethernet1/2,ethernet1/1,LF-LOG-ALLOW,2017/02/14 10:21:24,256848,1,50770,53,0,0,0x19,udp,allow,842,85,757,2,2017/02/14 10:20:52,30,any,0,443053418,0x0,10.0.0.0-10.255.255.255,US,0,1,1,aged-out,0,0,0,0,,PA-3050,from-policy

0 Karma
Highlighted

Re: Is it possible to assign different timestamps based on log line contents within the same sourcetype?

Legend

Hi daishih,
no: timestamp is unique for a sourcetype.

Maybe you could a try but you have more license consuption:

  • index all logs with the first sourcetype (so the first timestamp),
  • extract the logs that you want with the second sourcetype (so the second timestamp) writing them in a file
  • reindex the second one using the second sourcetype
  • use a filer in your searches to exclude the extracted events from the first search

I understand that is a pork-around, but I don't see anything else.

Bye.
Giuseppe

0 Karma
Highlighted

Re: Is it possible to assign different timestamps based on log line contents within the same sourcetype?

SplunkTrust
SplunkTrust

Give this a shot, should happen at index time (put on forwarders & indexers)

Props.conf:

[pan:traffic]
TRANSFORMS-dateStartTimeTransform=dateStartTimeTransform
TRANSFORMS-dateEndTimeTransform=dateEndTimeTransform

Transforms.conf:

[dateStartTimeTransform]
SOURCE_KEY = _raw
REGEX = (?<=start).*(\d{4}\/\d{2}\/\d{2}\s\d{2}:\d{2}:\d{2})
DEST_KEY = _time

[dateEndTimeTransform]
SOURCE_KEY = _raw
REGEX = (\d{4}\/\d{2}\/\d{2}\s\d{2}:\d{2}:\d{2})(?=.*end)
DEST_KEY = _time

View solution in original post

Highlighted

Re: Is it possible to assign different timestamps based on log line contents within the same sourcetype?

Path Finder

This works perfectly, thank you so much! It had never occurred to me to do a transform like that.

0 Karma
Highlighted

Re: Is it possible to assign different timestamps based on log line contents within the same sourcetype?

SplunkTrust
SplunkTrust

Im actually surprised it works because the transforms.conf documentation says _time is time in epoch... but hey... if it works, it works!

0 Karma