Getting Data In

How to customise value of _time from event data at index-time field extraction?

imahadevia_splu
Splunk Employee
Splunk Employee

I am trying to extract following data, and I want the date which is in EVENT tab as default TIME field which is extracted by _time.

Sample data:

2012-02-03 20:11:56 SampleClass3 [INFO] everything normal for id 530537821
2012-02-03 20:11:56 SampleClass3 [TRACE] verbose detail for id 1718828806
2012-02-03 20:11:56 SampleClass8 [DEBUG] detail for id 2083681507

Current Output:

alt text

I have tried using different time formats in my prpos.conf but it didn't work for me. My current props.conf is as follows :

[source::/root/sample.log]
TRANSFORMS-extracted_data = extract-log-type extract-log-date
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 20
SHOULD_LINEMERGE = FALSE

There has been a lot of Q&As about _time but I have not found any definitive answers. Any help is appreciated!

Thank You

1 Solution

gpatel_splunk
Splunk Employee
Splunk Employee

Just change your props.conf stanza to

[source::/root/sample.log]
TRANSFORMS-extracted_data = extract-log-type extract-log-date
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 20
SHOULD_LINEMERGE = FALSE
MAX_DAYS_AGO = 10951
TIME_PREFIX = ^ 

When you add TIME_PREFIX = ^ to your props.conf will make splunk to try looking for Timestamp from the first character of any new event. and by adding MAX_DAYS_AGO to props.conf will specifies the maximum number of days in the past, from the current date, that an extracted date can be valid. By default, the value of MAX_DAYS_AGO is 2000 days i.e. 5.479452 Years

View solution in original post

gpatel_splunk
Splunk Employee
Splunk Employee

Just change your props.conf stanza to

[source::/root/sample.log]
TRANSFORMS-extracted_data = extract-log-type extract-log-date
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 20
SHOULD_LINEMERGE = FALSE
MAX_DAYS_AGO = 10951
TIME_PREFIX = ^ 

When you add TIME_PREFIX = ^ to your props.conf will make splunk to try looking for Timestamp from the first character of any new event. and by adding MAX_DAYS_AGO to props.conf will specifies the maximum number of days in the past, from the current date, that an extracted date can be valid. By default, the value of MAX_DAYS_AGO is 2000 days i.e. 5.479452 Years

thomasroulet
Path Finder

Hello,

because the date 2012-02-03 20:11:56
is too far in the past (more than 7 years) you have to add a parameter in your props.conf

MAX_DAYS_AGO = 3650

you have to adjust the value of the parameter.
Default: 2000 (5.48 years).

edit:
you may have to edit the frozenTimePeriodInSecs parameter in your indexes.conf
The default value is 188697600 (6 years).

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...