Getting Data In

Is it possible to index a different timestamp based on a field value in the same log?

matthewe25
Engager

I'm a new Splunk user that needs to perform some analysis on a set of logs with the following format:

Status, Starttime, Endtime, JobID, ReturnCode

START,2021-03-15 10:56:15, ,123,

END,2021-03-15 10:56:15,2021-03-15 10:56:27,123,0

...

For a single job, there are separate START and END logs which can be paired together by matching the start time and job ID using the 'transactions' command.

When indexing these logs in Splunk, we've been parsing the Starttime field (2nd comma delimited value) as our timestamp but this means that all log entries with the 'END' status have a _time field corresponding to their start time rather than their end time. This leads to some extra work when filtering entries by time or calculating duration of jobs since we have to parse the Endtime field at search time.

Is there any way to add some sort of condition which allows us to index a different timestamp field depending on the 'Status' field of an individual log? (i.e. if Status=START, take _time=Starttime but if Status=END, take _time=Endtime)

Labels (3)
0 Karma
1 Solution

manjunathmeti
SplunkTrust
SplunkTrust

hi @matthewe25,

You can check with the below configurations in props.conf. Remove attribute TIMESTAMP_FIELDS if you are using it.

[sourcetype]
TIME_PREFIX = (START|END,\d+-\d+-\d+\s\d+:\d+:\d+),
TIME_FORMAT = %Y-%d-%m %H:%M:%S

 

If this reply helps you, an upvote/like would be appreciated.

View solution in original post

manjunathmeti
SplunkTrust
SplunkTrust

hi @matthewe25,

You can check with the below configurations in props.conf. Remove attribute TIMESTAMP_FIELDS if you are using it.

[sourcetype]
TIME_PREFIX = (START|END,\d+-\d+-\d+\s\d+:\d+:\d+),
TIME_FORMAT = %Y-%d-%m %H:%M:%S

 

If this reply helps you, an upvote/like would be appreciated.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...