topic Re: Could not use strptime to parse timestamp in Splunk Search

Could not use strptime to parse timestamp

asarolkar — Tue, 05 Mar 2013 02:01:46 GMT

I have researched this error previously (and found a lot of helpful material).
I am stuck with a slightly complicated variation of this commonly known problem.

http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition

I need to extract the second timestamp from a certain log file.
The log file has different kinds of sub-log-types merged into one giant log file.

Which means, I need to extract the second timestamp (that presents itself at a varying number of characters distance from the FIRST useless time stamps)

Mar  4 10:05:02 america-p01 access_combined: 10.142.1.109 - - [04/Mar/2013:02:05:03 -0800] "GET /healthCheck/status " 200 13 "-" "-"

Mar  4 10:05:10 america-p01 syslog: 2013-03-04 02:05:11,771 INFO  [http-0.0.0.0-8080-3] -TpaiL5RBCo4-CH-Fjo9rw__ ERI IdsPatientLogger - Logging the CREATE of Account: 464c-9f5c-074ab072ee58 by User: ERI

Mar  4 10:06:27 america-p01 auditlog: AuditEntry[event=LoginRequest,ip=,date=2013-03-04T02:06:28.057-08:00,user=olivia,status=Success,description=]

My props.conf looks like this

  NO_BINARY_CHECK=1
    SHOULD_LINEMERGE=false
    TIME_FORMAT=%d/%b/%Y:%H:%M:%S %Z
    TIME_PREFIX=america-

What I expect is for Splunk to recognize the following as correct timestamps and use these SECOND timestamps instead

i) For access_combined -> [04/Mar/2013:02:05:03 -0800]
ii) For syslog -> 2013-03-04 02:05:11,771
iii) For auditlog -> 2013-03-04T02:06:28.057-08:00

My configuration errors out with the following error for all three types of sub-logs:

-> Could not use strp to parse time stamp ....

Is it because my configuration is not correct ?
Is there no such thing as one regex for all three types of timestamps ( what I tried to setup in TIME_FORMAT) ?
I dont see the point of adding a MAX _ TIMESTAMP _ LOOKAHEAD here - would that be helpful ?

Re: Could not use strptime to parse timestamp

lguinn2 — Tue, 05 Mar 2013 03:57:51 GMT

I suggest that you leave out the TIME_FORMAT and just have

NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_PREFIX=america-

Splunk is very good at figuring out the time format automatically, and can easily adjust to the fact that there are variations. You also don't need the MAX_TIMESTAMP_LOOKAHEAD, and you probably shouldn't use it if you can't predict the number of characters after america- to the timestamp.

Re: Could not use strptime to parse timestamp

asarolkar — Tue, 05 Mar 2013 04:01:01 GMT

Hi there,

I tried that and it did not work unfortunately.

Splunk keeps thinking that the first timestamp is the correct timestamp.

Do you think a TIME_FORMAT regex like %d/%b/%Y:%H:%M:%S %Z would be helpful here ?

Re: Could not use strptime to parse timestamp

lguinn2 — Tue, 05 Mar 2013 14:27:02 GMT

No, I don't think that the TIME_FORMAT will help you.

Try

TIME_PREFIX=america-.*?:

I think that may work better.