Verifying what Splunk is using for timestamps? Trying to parse a UNIX Epoch Timestamp in an odd place

[Ricapar]

I have some log files from an internal application that look like this:

hostname|Linux|RHEL6|20Sep2012:1348163022|00:10:13(0days-0hours-10minutes-13secs)|1234|something|else|goes|here but it doesn't matter

Fields are in the order of...

hostname|os|os version|DDMonYYYY:TIMESTAMP|time something took to run|pid|etc..|etc..

I need TIMESTAMP.

I have this in my props.conf:

[custom_app_logfile]
MAX_TIMESTAMP_LOOKAHEAD = 10
NO_BINARY_CHECK = 1
REPORT-myfields = transform_mycustom_log
SHOULD_LINEMERGE = false
TIME_FORMAT = %s
TIME_PREFIX = \|[0-9]+[A-Za-z]+[0-9]+:
pulldown_type = 1

But it doesn't seem to be working consistently... I have certain entries where the timestamp is correct. But there are ones from back in 2005 that show up as today's date, and many where it's just getting the file's timestamp (also incorrect).

Questions are:

1. Is there a way to see exactly what Splunk is deciding to use for it's timestamp? It feels like I'm flying blind editing props.conf and hoping it changes something.
2. Is there a better approach to pulling that timestamp out? It's a standard UNIX Epoch Timestamp.

[dwaddle · Accepted solution]

If I understand what you're saying, the timestamp you're trying to extract is the epoch value of 1348163022 in the above event. If I've missed that assumption, the rest of this is all pretty much rubbish.

Your TIME_FORMAT and TIME_PREFIX look like they should work. But, I would probably make a change to make TIME_PREFIX a bit more specific, like:

TIME_PREFIX = \|[0-9]{1,2}[A-Z][a-z]{2}[0-9]{4}:

Just to try to make it less likely to match falsely.

[dwaddle]

You are right about MAX_TIMESTAMP_LOOKAHEAD. I was quite wrong. I edited my original response. However, a comment on the 948431652 timestamp - you may be running into MAX_DAYS_AGO, as that is over 2000 days ago.

[Ricapar]

I'm curious though - why isn't 948431652 being picked up? According to a quick lookup... that should be equal to: "01 / 20 / 00 @ 11:14:12pm EST"

That aside, is there a way to tell Splunk to drop those events where it can't find a proper timestamp?

[Ricapar]

dwaddle,

You're correct on the epoch value.

For MAX_TIMESTAMP_LOOKAHEAD .. I was under the assumption that that's the amount it looks ahead after it finds the match in TIME_PREFIX (in which case, it'd be the next 10 characters).

However, I think I found my real issue...

This is matching correctly:

hostname|SunOS|5.10|21Mar2009:1237582943|06:04:52(00days-00hours-00minutes-01secs)

... but this isn't ...

hostname|SunOS|5.10|18Mar2009:948431652|07:51:21(00days-00hours-01minutes-41secs)

The timestamp there is still a valid timestamp (although it is wrong)

(continued on next post)

[jbsplunk]

I think the problem here is with the TIME_PREFIX, try using this one, I tested it and it matches, whereas the one you have posted didn't:

TIME_FORMAT = \d{2}\w{3}\d{4}:

[bmacias84]

I would do use data preview which can be found in Manager » Data inputs » Files & directories » Data preview. Enter a location containing the data sample. On the Advanced Mode tab enter your config from your props.conf. Splunk will highlight the events.

You could also do a search ...| table _time,_raw |head 20 . _time should match your time from your raw event.