Splunk Search

How to add DATE to a timestamp without a date

season88481
Contributor

We have some log files with name like this: logs_2020-06-30.logs. A sample events looks like this:

 

2020-07-01 12:01:55.123 something something
   something
   something
   something
12:01:55.124 something something
12:01:55.125 something something

2020-07-01 12:02:57.234 something2 something2
   something2
   something2
   something2
12:02:57.235 something2 something2
12:02:57.236 something2 something2

 

 

We are breaking the events like this:

 

2020-07-01 12:01:55.123 something something
   something
   something
   something
12:01:55.124 something something
12:01:55.125 something something

 

 

As we can see, some of the events have a timestamp without a date.  Currently Splunk is using the date from the filename.  In this case, 2020-06-30. 

 

We don't want to use the DATE provided in the filename, as very often the log files are created in the previous date.

And we  cannot use DATETIME_CONFIG=CURRENT either, as this is not quite accurate. 

Is there a way to use the current date for these events? 

 

Many thanks.

S

0 Karma
1 Solution

season88481
Contributor

Hi everyone,

 

I end up using INGEST_EVAL in my Heavy Forwarder to append the current date into the timestamp.

Here is my sample props.conf and transforms.conf

#props.conf
[mysourcetype]
TRANSFORMS-getdate = get-date
#transforms.conf
INGEST_EVAL = time_only=strftime(_time,"%H:%M:%S.%3N"),current_date=strftime(time(),"%Y-%m-%d"),date_time=current_date."T".time_only,_time=strptime(date_time, "%Y-%m-%dT%H:%M:%S.%3N")

 

No need to add fields.conf at SHs. As _time is a meta field.

Hope this can help others.

 

Cheers,

S

View solution in original post

Tags (3)

season88481
Contributor

Hi everyone,

 

I end up using INGEST_EVAL in my Heavy Forwarder to append the current date into the timestamp.

Here is my sample props.conf and transforms.conf

#props.conf
[mysourcetype]
TRANSFORMS-getdate = get-date
#transforms.conf
INGEST_EVAL = time_only=strftime(_time,"%H:%M:%S.%3N"),current_date=strftime(time(),"%Y-%m-%d"),date_time=current_date."T".time_only,_time=strptime(date_time, "%Y-%m-%dT%H:%M:%S.%3N")

 

No need to add fields.conf at SHs. As _time is a meta field.

Hope this can help others.

 

Cheers,

S

Tags (3)
Get Updates on the Splunk Community!

Developer Spotlight with William Searle

The Splunk Guy: A Developer’s Path from Web to Cloud William is a Splunk Professional Services Consultant with ...

Major Splunk Upgrade – Prepare your Environment for Splunk 10 Now!

Attention App Developers: Test Your Apps with the Splunk 10.0 Beta and Ensure Compatibility Before the ...

Stay Connected: Your Guide to June Tech Talks, Office Hours, and Webinars!

What are Community Office Hours?Community Office Hours is an interactive 60-minute Zoom series where ...