Getting Data In

Is it possible to index a different timestamp based on a field value in the same log?

matthewe25
Engager

I'm a new Splunk user that needs to perform some analysis on a set of logs with the following format:

Status, Starttime, Endtime, JobID, ReturnCode

START,2021-03-15 10:56:15, ,123,

END,2021-03-15 10:56:15,2021-03-15 10:56:27,123,0

...

For a single job, there are separate START and END logs which can be paired together by matching the start time and job ID using the 'transactions' command.

When indexing these logs in Splunk, we've been parsing the Starttime field (2nd comma delimited value) as our timestamp but this means that all log entries with the 'END' status have a _time field corresponding to their start time rather than their end time. This leads to some extra work when filtering entries by time or calculating duration of jobs since we have to parse the Endtime field at search time.

Is there any way to add some sort of condition which allows us to index a different timestamp field depending on the 'Status' field of an individual log? (i.e. if Status=START, take _time=Starttime but if Status=END, take _time=Endtime)

Labels (3)
0 Karma
1 Solution

manjunathmeti
Champion

hi @matthewe25,

You can check with the below configurations in props.conf. Remove attribute TIMESTAMP_FIELDS if you are using it.

[sourcetype]
TIME_PREFIX = (START|END,\d+-\d+-\d+\s\d+:\d+:\d+),
TIME_FORMAT = %Y-%d-%m %H:%M:%S

 

If this reply helps you, an upvote/like would be appreciated.

View solution in original post

manjunathmeti
Champion

hi @matthewe25,

You can check with the below configurations in props.conf. Remove attribute TIMESTAMP_FIELDS if you are using it.

[sourcetype]
TIME_PREFIX = (START|END,\d+-\d+-\d+\s\d+:\d+:\d+),
TIME_FORMAT = %Y-%d-%m %H:%M:%S

 

If this reply helps you, an upvote/like would be appreciated.

Get Updates on the Splunk Community!

Enhance Your Splunk App Development: New Tools & Support

UCC FrameworkAdd-on Builder has been around for quite some time. It helps build Splunk apps faster, but it ...

Prove Your Splunk Prowess at .conf25—No Prereqs Required!

Your Next Big Security Credential: No Prerequisites Needed We know you’ve got the skills, and now, earning the ...

Splunk Observability Cloud's AI Assistant in Action Series: Observability as Code

This is the sixth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...