Getting Data In

getting timestamps

JeffTanYH
Engager

I would like to index my logs,however,I'm new to SPLUNK and I do not know how to break my logs up using timestamps. My data looks something like this:

hello1:/dev/console:Mon Nov  6 13:21:49 2010
hello2:/dev/console:Mon Nov  6 13:22:10 2010
hello3:/dev/console:Mon Nov  6 13:22:33 2010
hello4:/dev/console:Mon Nov  6 13:26:14 2010
hello5:/dev/console:Mon Nov  6 13:26:27 2010

Can someone assist me with a prop.conf to break these into different events with the appropiate timestamps? Each line is a different event.

Tags (1)
0 Karma

kristian_kolb
Ultra Champion

Splunk will not accept timestamps that are too far off from the actual date/time (i.e. the time on the indexer).

There are a few props.conf settings that can be adjusted to allow for indexing of old (or future) events, called MAX_DAYS_HENCE and MAX_DAYS_AGO.

See http://docs.splunk.com/Documentation/Splunk/4.3.1/admin/Propsconf


UPDATE:

Sorry, but I interpreted the log as having timestamps with years in them.

hello5:/dev/console:Mon Nov 6 13:26:27 2006

As for adding a 'false' year to the events, I have not heard of how to that could be achieved.
This section of the docs might give you further guidance;

http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps

/k

Hope this helps,

Kristian

kristian_kolb
Ultra Champion

see update above. /k

0 Karma

JeffTanYH
Engager

My logs do not state the year that the logs were produced,it only states the day and the month (i.e Wed Mar 4) that it was created. I presume SPLUNK automatically "thinks" these logs were created this year,but it is actually created in 2011. Is there any way in which I can tell SPLUNK that these logs were create in 2011,instead of letting it presumably take it that they were created this year?

0 Karma

JeffTanYH
Engager

It doesnt seem to be working. When I used Mario's TIME_FORMAT and kristian's TIME_PREFIX. The time in the timestamp is right,however,the date is still incorrect. It looks something like this:

1 04/05/2012 13:21:49.000 hello1:/dev/console:Mon Nov 6 13:21:49 2006

2 04/05/2012 13:22:10.000 hell02:/dev/console:Mon Nov 6 13:22:10 2006

3 04/05/2012 13:22:33.000 hello3:/dev/console:Mon Nov 6 13:22:33 2006

4 04/05/2012 13:26:14.000 hello4:/dev/console:Mon Nov 6 13:26:14 2006

5 04/05/2012 13:26:27.000 hello5:/dev/console:Mon Nov 6 13:26:27 2006

As you can see,the time is correct,however,the date doesnt in the timestamp doesnt seem to match those in the event.

0 Karma

JeffTanYH
Engager

Is there any way in which I can tell SPLUNK that the logs I am uploading is for the year of 2011 instead of the current year (2012)? Whenever I upload any logs into SPLUNK,it automatically timestamps my logs,however,it is timestamps it for the year of 2012 (instead of 2011). Some of the timestamps shows the event occuring in June or July 2012,which hasn't even past! Please help me!

0 Karma

MarioM
Motivator

Normally splunk should break automatically the events and recognize the timestamps.It did on my splunk with your data...

But if you want to force it you could set the following in your props.conf:

[your_data_sourcetype]
SHOULD_LINEMERGE=false
TIME_FORMAT=%b %d %H:%M:%S %Y
TIME_PREFIX=\:\w+\s+
0 Karma

kristian_kolb
Ultra Champion

TIME_PREFIX is wrong. Try;

TIME_PREFIX = /dev/console:

/K

0 Karma

MarioM
Motivator

the conf is based on the sample data you pasted then if it doesnot work it means the data is different.

You need to adjust it(or paste a better extract of your raw data) and this will only apply to new data.

Have you tried without any configuration or with data preview?

0 Karma

JeffTanYH
Engager

And one more thing,what does this mean?:

  • Could not use strptime to parse timestamp from "(null)"

and

  • Could not use regex to parse timestamp from "(null)"
0 Karma

JeffTanYH
Engager

Hey,there seems to be a problem.
I copied the props.conf that you come up with,however,the timestamp still shows the date in which I uploaded the logs,instead of the dates that are stated in each event.

I had to remove the TIME_PREFIX line before the time could match,but the date still remained as the date in which I uploaded my logs into SPLUNK.

0 Karma
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...