topic Re: getting timestamps in Getting Data In

getting timestamps

JeffTanYH — Wed, 09 May 2012 07:00:52 GMT

I would like to index my logs,however,I'm new to SPLUNK and I do not know how to break my logs up using timestamps. My data looks something like this:

hello1:/dev/console:Mon Nov  6 13:21:49 2010
hello2:/dev/console:Mon Nov  6 13:22:10 2010
hello3:/dev/console:Mon Nov  6 13:22:33 2010
hello4:/dev/console:Mon Nov  6 13:26:14 2010
hello5:/dev/console:Mon Nov  6 13:26:27 2010

Can someone assist me with a prop.conf to break these into different events with the appropiate timestamps? Each line is a different event.

Re: getting timestamps

MarioM — Wed, 09 May 2012 08:02:03 GMT

Normally splunk should break automatically the events and recognize the timestamps.It did on my splunk with your data...

But if you want to force it you could set the following in your props.conf:

[your_data_sourcetype]
SHOULD_LINEMERGE=false
TIME_FORMAT=%b %d %H:%M:%S %Y
TIME_PREFIX=\:\w+\s+

Re: getting timestamps

JeffTanYH — Wed, 09 May 2012 09:13:45 GMT

Hey,there seems to be a problem.
I copied the props.conf that you come up with,however,the timestamp still shows the date in which I uploaded the logs,instead of the dates that are stated in each event.

I had to remove the TIME_PREFIX line before the time could match,but the date still remained as the date in which I uploaded my logs into SPLUNK.

Re: getting timestamps

JeffTanYH — Wed, 09 May 2012 09:17:11 GMT

And one more thing,what does this mean?:

Could not use strptime to parse timestamp from "(null)"

and

Could not use regex to parse timestamp from "(null)"

Re: getting timestamps

MarioM — Wed, 09 May 2012 10:14:54 GMT

the conf is based on the sample data you pasted then if it doesnot work it means the data is different.

You need to adjust it(or paste a better extract of your raw data) and this will only apply to new data.

Have you tried without any configuration or with data preview?

Re: getting timestamps

kristian_kolb — Wed, 09 May 2012 10:51:13 GMT

TIME_PREFIX is wrong. Try;

TIME_PREFIX = /dev/console:

Re: getting timestamps

JeffTanYH — Mon, 28 Sep 2020 11:47:55 GMT

It doesnt seem to be working. When I used Mario's TIME_FORMAT and kristian's TIME_PREFIX. The time in the timestamp is right,however,the date is still incorrect. It looks something like this:

1 04/05/2012 13:21:49.000 hello1:/dev/console:Mon Nov 6 13:21:49 2006

2 04/05/2012 13:22:10.000 hell02:/dev/console:Mon Nov 6 13:22:10 2006

3 04/05/2012 13:22:33.000 hello3:/dev/console:Mon Nov 6 13:22:33 2006

4 04/05/2012 13:26:14.000 hello4:/dev/console:Mon Nov 6 13:26:14 2006

5 04/05/2012 13:26:27.000 hello5:/dev/console:Mon Nov 6 13:26:27 2006

As you can see,the time is correct,however,the date doesnt in the timestamp doesnt seem to match those in the event.

Re: getting timestamps

JeffTanYH — Thu, 10 May 2012 07:11:02 GMT

Is there any way in which I can tell SPLUNK that the logs I am uploading is for the year of 2011 instead of the current year (2012)? Whenever I upload any logs into SPLUNK,it automatically timestamps my logs,however,it is timestamps it for the year of 2012 (instead of 2011). Some of the timestamps shows the event occuring in June or July 2012,which hasn't even past! Please help me!

Re: getting timestamps

kristian_kolb — Thu, 10 May 2012 07:33:57 GMT

Splunk will not accept timestamps that are too far off from the actual date/time (i.e. the time on the indexer).

There are a few props.conf settings that can be adjusted to allow for indexing of old (or future) events, called MAX_DAYS_HENCE and MAX_DAYS_AGO.

See http://docs.splunk.com/Documentation/Splunk/4.3.1/admin/Propsconf

UPDATE:

Sorry, but I interpreted the log as having timestamps with years in them.

hello5:/dev/console:Mon Nov 6 13:26:27 2006

As for adding a 'false' year to the events, I have not heard of how to that could be achieved.
This section of the docs might give you further guidance;

http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps

Hope this helps,

Kristian

Re: getting timestamps

JeffTanYH — Thu, 10 May 2012 08:28:06 GMT

My logs do not state the year that the logs were produced,it only states the day and the month (i.e Wed Mar 4) that it was created. I presume SPLUNK automatically "thinks" these logs were created this year,but it is actually created in 2011. Is there any way in which I can tell SPLUNK that these logs were create in 2011,instead of letting it presumably take it that they were created this year?

Re: getting timestamps

kristian_kolb — Thu, 10 May 2012 08:58:12 GMT

see update above. /k