topic Re: Timestamp from file with no year in Getting Data In

Timestamp from file with no year

billysmusic — Mon, 28 Sep 2020 10:21:45 GMT

I have a time-stamp in format Wed Jan 25 16:36:02 EST. I can't get Splunk to match it.
I tried modifying the props.conf:
[host::rok*]
TIME_PREFIX = dst
TIME_FORMAT = %a %b %d %H:%M:%S %Z

But it doesn't recognize the pattern. Am I missing something?
Full-event line:
dst Thu Jan 26 07:45:12 EST 10.10.1.2:vmwsspapp02_prd_data01 rok:vmwsspapp02_prd_data01 Start

Thanks!

Re: Timestamp from file with no year

sbrant_tt — Thu, 26 Jan 2012 01:02:05 GMT

You appear to be missing the year match in your conf file. The pattern to match your time-stamp should be:

%a %b %d %Y %H:%M:%S %Z

Re: Timestamp from file with no year

billysmusic — Thu, 26 Jan 2012 01:08:11 GMT

Oops, I accidentally put the year in source format. It is actually not there. I updated my question with the correct format. Thank you for responding though.

Re: Timestamp from file with no year

gkanapathy — Thu, 26 Jan 2012 05:04:23 GMT

why do you have TIME_PREFIX = dst? What does the actual event line look like? Does it actually contain that string immediately before the timestamp?

Re: Timestamp from file with no year

hexx — Thu, 26 Jan 2012 05:41:57 GMT

I agree with @gkanapathy, we cannot really recommend a configuration without a sample event to base it on.

Re: Timestamp from file with no year

billysmusic — Mon, 28 Sep 2020 10:21:52 GMT

Sure, here is a full line of an event:
dst Thu Jan 26 07:45:12 EST 10.10.1.2:vmwsspapp02_prd_data01 rok:vmwsspapp02_prd_data01 Start

Re: Timestamp from file with no year

hexx — Mon, 28 Sep 2020 10:21:55 GMT

Thank you for providing a sample event.

Splunk should be able to interpret the time stamp on its own, but I would strongly recommend that you use TIME_PREFIX and MAX_TIMESTAMP_LOOKAHEAD to scope the time stamp extraction to the location in your events where it can be found :

[host::rok*]
TIME_PREFIX = ^dst
MAX_TIMESTAMP_LOOKAHEAD = 24

This is very important because you do not want Splunk to pick up a string that may look like a year somewhere else in the event, which may result in a wrong time stamp.

TIME_FORMAT is optional here, but you can specify it if desired to speed up the time stamp extraction process.

Make sure to refer to props.conf.spec for a full description of these configuration keys.

UPDATE: Since you seem to have line-breaking issues, I would suggest that you add the following configuration keys to explicitly declare how your source file should be split into events:

LINE_BREAKER = ([\r\n]+)dst\s+
SHOULD_LINEMERGE = false

This is assuming that all of your events begin with the string "dst ", and that no line that is not an event begins with that string.

Re: Timestamp from file with no year

billysmusic — Thu, 26 Jan 2012 18:00:35 GMT

Thanks for the suggestion. I just tried both of those and left out the TIME_FORMAT with no luck. It still sees the entire log as one event. I've also messed around with every setting I can think of in the "Data Preview" section when adding the source but cannot get it to recognize each individual line.

Re: Timestamp from file with no year

hexx — Thu, 26 Jan 2012 18:02:28 GMT

That means that you have an issue with line-breaking, which has to be addressed with different parameters. I'll update my answer.

Re: Timestamp from file with no year

walterleunghk — Mon, 28 Sep 2020 15:19:35 GMT

If the log is not too old, i think below should work.

MAX_DAYS_AGO = 100

TIME_FORMAT = %a %b %d %H:%M:%S %Z

TIME_PREFIX = ^dst\s+