Getting Data In

How to customise value of _time from event data at index-time field extraction?

imahadevia_splu
Splunk Employee
Splunk Employee

I am trying to extract following data, and I want the date which is in EVENT tab as default TIME field which is extracted by _time.

Sample data:

2012-02-03 20:11:56 SampleClass3 [INFO] everything normal for id 530537821
2012-02-03 20:11:56 SampleClass3 [TRACE] verbose detail for id 1718828806
2012-02-03 20:11:56 SampleClass8 [DEBUG] detail for id 2083681507

Current Output:

alt text

I have tried using different time formats in my prpos.conf but it didn't work for me. My current props.conf is as follows :

[source::/root/sample.log]
TRANSFORMS-extracted_data = extract-log-type extract-log-date
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 20
SHOULD_LINEMERGE = FALSE

There has been a lot of Q&As about _time but I have not found any definitive answers. Any help is appreciated!

Thank You

1 Solution

gpatel_splunk
Splunk Employee
Splunk Employee

Just change your props.conf stanza to

[source::/root/sample.log]
TRANSFORMS-extracted_data = extract-log-type extract-log-date
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 20
SHOULD_LINEMERGE = FALSE
MAX_DAYS_AGO = 10951
TIME_PREFIX = ^ 

When you add TIME_PREFIX = ^ to your props.conf will make splunk to try looking for Timestamp from the first character of any new event. and by adding MAX_DAYS_AGO to props.conf will specifies the maximum number of days in the past, from the current date, that an extracted date can be valid. By default, the value of MAX_DAYS_AGO is 2000 days i.e. 5.479452 Years

View solution in original post

gpatel_splunk
Splunk Employee
Splunk Employee

Just change your props.conf stanza to

[source::/root/sample.log]
TRANSFORMS-extracted_data = extract-log-type extract-log-date
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 20
SHOULD_LINEMERGE = FALSE
MAX_DAYS_AGO = 10951
TIME_PREFIX = ^ 

When you add TIME_PREFIX = ^ to your props.conf will make splunk to try looking for Timestamp from the first character of any new event. and by adding MAX_DAYS_AGO to props.conf will specifies the maximum number of days in the past, from the current date, that an extracted date can be valid. By default, the value of MAX_DAYS_AGO is 2000 days i.e. 5.479452 Years

thomasroulet
Path Finder

Hello,

because the date 2012-02-03 20:11:56
is too far in the past (more than 7 years) you have to add a parameter in your props.conf

MAX_DAYS_AGO = 3650

you have to adjust the value of the parameter.
Default: 2000 (5.48 years).

edit:
you may have to edit the frozenTimePeriodInSecs parameter in your indexes.conf
The default value is 188697600 (6 years).

Get Updates on the Splunk Community!

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...

Combine Multiline Logs into a Single Event with SOCK: a Step-by-Step Guide for ...

Combine multiline logs into a single event with SOCK - a step-by-step guide for newbies Olga Malita The ...

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...