We have some log files with name like this: logs_2020-06-30.logs. A sample events looks like this:
2020-07-01 12:01:55.123 something something
something
something
something
12:01:55.124 something something
12:01:55.125 something something
2020-07-01 12:02:57.234 something2 something2
something2
something2
something2
12:02:57.235 something2 something2
12:02:57.236 something2 something2
We are breaking the events like this:
2020-07-01 12:01:55.123 something something
something
something
something
12:01:55.124 something something
12:01:55.125 something something
As we can see, some of the events have a timestamp without a date. Currently Splunk is using the date from the filename. In this case, 2020-06-30.
We don't want to use the DATE provided in the filename, as very often the log files are created in the previous date.
And we cannot use DATETIME_CONFIG=CURRENT either, as this is not quite accurate.
Is there a way to use the current date for these events?
Many thanks.
S
Hi everyone,
I end up using INGEST_EVAL in my Heavy Forwarder to append the current date into the timestamp.
Here is my sample props.conf and transforms.conf
#props.conf
[mysourcetype]
TRANSFORMS-getdate = get-date
#transforms.conf
INGEST_EVAL = time_only=strftime(_time,"%H:%M:%S.%3N"),current_date=strftime(time(),"%Y-%m-%d"),date_time=current_date."T".time_only,_time=strptime(date_time, "%Y-%m-%dT%H:%M:%S.%3N")
No need to add fields.conf at SHs. As _time is a meta field.
Hope this can help others.
Cheers,
S
Hi everyone,
I end up using INGEST_EVAL in my Heavy Forwarder to append the current date into the timestamp.
Here is my sample props.conf and transforms.conf
#props.conf
[mysourcetype]
TRANSFORMS-getdate = get-date
#transforms.conf
INGEST_EVAL = time_only=strftime(_time,"%H:%M:%S.%3N"),current_date=strftime(time(),"%Y-%m-%d"),date_time=current_date."T".time_only,_time=strptime(date_time, "%Y-%m-%dT%H:%M:%S.%3N")
No need to add fields.conf at SHs. As _time is a meta field.
Hope this can help others.
Cheers,
S