topic Re: Timestmap extraction for multiple events in single sourcetype in Getting Data In

Timestmap extraction for multiple events in single sourcetype

darshm — Tue, 09 Jul 2024 14:25:42 GMT

Hi folks,
I have a use case where I am having different types of events in a single sourcetype. I want to apply different timestamp extractions for both the events.
I am using TIME_PREFIX and MAX_TIMESTAMP_LOOKAHEAD to extract the timestamp from event #1. However, the same rules won't be useful for event #2.

Is there a way to extract the timestamp values from both the events in a single sourcetype?

Event #1 Timestamp should be extracted as (Oct 9 23:57:37.887)
Oct 10 05:27:48 192.168.100.1 593155: *Oct 9 23:57:37.887: blah blah blah

Event #2 Timestamp should be extracted as (Feb 13 11:27:46)
Feb 13 11:27:46 100.80.8.22 %abc-INFO-000: blah blah blah

TIME_PREFIX = \s[^\s]+\s\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}\s[^\s]+:\s|\s[^\s]+\s\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}\s MAX_TIMESTAMP_LOOKAHEAD = 30

Re: Timestmap extraction for multiple events in single sourcetype

ITWhisperer — Tue, 09 Jul 2024 13:08:21 GMT

The short answer is that the different log formats should be in different sourcetypes.

Re: Timestmap extraction for multiple events in single sourcetype

gcusello — Tue, 09 Jul 2024 13:15:33 GMT

Hi @darshm ,

if you're sure that in your events there's only one date and time, you could leave Splunk to choose the timestamp, but, my hint is the same of @ITWhisperer : different formats should have different sourcetypes, eventually with a similar name (e.g. for fortinet there are fortigate_events, fortigate_logs, fortigate_utm, etc...).

Ciao.

Giuseppe

Re: Timestmap extraction for multiple events in single sourcetype

darshm — Wed, 10 Jul 2024 09:50:25 GMT

Hi @gcusello @ITWhisperer , Thanks for your response.

Regarding the solution which you are recommending, I agree to your point. Ideally, the sourcetypes should be different for different types of events.

However, in our case, we are having parent-child relationship of the sourcetypes. We are bifurcating the child sourcetype from the parent sourcetype.

We observed that the TIME_PREFIX extractions were not getting applied if we defined them in the child stanza. Seems like Splunk first performs the timestamp extractions from parent and then the renaming of the sourcetype happens.

So, we are trying to figure out a way in which we can handle multiple event format in the parent sourcetype stanza itself.