Splunk Enterprise

Own timestamp assignment not working

New Member

Hi,

I want to use timechart to analyse historical data files. The (*.log) files are just generated and each looks like this:

date = 1999-05-15
data1 = x
data2 = y

Now I want that "_time" contains my own "date" field. I found the article http://docs.splunk.com/Documentation/Splunk/6.5.3/Data/ConfigurePositionalTimestampExtraction
but I have a problem to bring this to work.

My environment: Forwarder (Windows 10), Indexer based on CentOS.
I cannot find any active props.conf.

I don't know if I need to place a props.conf in
/opt/splunk/etc/system/local (indexer) or
C:\Program Files\SplunkUniversalForwarder\etc\system\local (Forwarder)

I tried both and created a new props.conf with the following:

[host::FORWARDERHOSTNAME]
TIME_PREFIX = date =
TIME_FORMAT = %Y-%m-%d

And restarted all services. But the .log files I'm indexing will still be marked with a (_time) timestamp of import time (today) and not my historical date time (1999).

Some ideas?
Many thanks.

Tags (1)
0 Karma
1 Solution

Motivator

How about trying this stanza to be put in your props.conf:

DATETIME_CONFIG = 
MAX_DAYS_AGO = 7300
NO_BINARY_CHECK = true
TIME_FORMAT = %Y-%m-%d
TIME_PREFIX = date =
category = Custom
pulldown_type = true

I am attaching the jpg where I used some sample data locally and used additional MAX_DAYS_AGO setting to make the dates in your sample data to parse correctly into _time.

MAX_DAYS_AGO = 7300 is actually 20 years worth of days ignoring leap years (20*365). So if you think your data might have dates prior to that, choose this value accordingly. Let me know if this helped.

alt text

View solution in original post

0 Karma

Motivator

How about trying this stanza to be put in your props.conf:

DATETIME_CONFIG = 
MAX_DAYS_AGO = 7300
NO_BINARY_CHECK = true
TIME_FORMAT = %Y-%m-%d
TIME_PREFIX = date =
category = Custom
pulldown_type = true

I am attaching the jpg where I used some sample data locally and used additional MAX_DAYS_AGO setting to make the dates in your sample data to parse correctly into _time.

MAX_DAYS_AGO = 7300 is actually 20 years worth of days ignoring leap years (20*365). So if you think your data might have dates prior to that, choose this value accordingly. Let me know if this helped.

alt text

View solution in original post

0 Karma

New Member

Hi gokadroid,

it was not clear for me, that it's possible to configure that in the gui. I created a sourcetype and configured all above. After that I deleted my index and recreated a new one (Think it is possible to keep the old index!? - But that was a faster solution for me.). Now all Events are indexed with the correct date.
Also I can see the props.conf file now on the indexer (/opt/splunk/etc/apps/local) too.
WOW! That's exactly what I need. Now it is possible to use the timepicker with date range. First I tried it with reformatting the _time value ( | eval epoch = strptime(date, "%Y-%m-%d") | eval _time = epoch |) but that is not working with my time picker.
MAX_DAYS_AGO is also very usefull for me.
I thank you so much for that.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!