Splunk Search

Could not use strptime to parse timestamp

asarolkar
Builder

I have researched this error previously (and found a lot of helpful material).
I am stuck with a slightly complicated variation of this commonly known problem.

http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition

I need to extract the second timestamp from a certain log file.
The log file has different kinds of sub-log-types merged into one giant log file.

Which means, I need to extract the second timestamp (that presents itself at a varying number of characters distance from the FIRST useless time stamps)

Mar  4 10:05:02 america-p01 access_combined: 10.142.1.109 - - [04/Mar/2013:02:05:03 -0800] "GET /healthCheck/status " 200 13 "-" "-"

Mar  4 10:05:10 america-p01 syslog: 2013-03-04 02:05:11,771 INFO  [http-0.0.0.0-8080-3] -TpaiL5RBCo4-CH-Fjo9rw__ ERI IdsPatientLogger - Logging the CREATE of Account: 464c-9f5c-074ab072ee58 by User: ERI

Mar  4 10:06:27 america-p01 auditlog: AuditEntry[event=LoginRequest,ip=,date=2013-03-04T02:06:28.057-08:00,user=olivia,status=Success,description=]



My props.conf looks like this

  NO_BINARY_CHECK=1
    SHOULD_LINEMERGE=false
    TIME_FORMAT=%d/%b/%Y:%H:%M:%S %Z
    TIME_PREFIX=america-

What I expect is for Splunk to recognize the following as correct timestamps and use these SECOND timestamps instead

i) For access_combined -> [04/Mar/2013:02:05:03 -0800]
ii) For syslog -> 2013-03-04 02:05:11,771
iii) For auditlog -> 2013-03-04T02:06:28.057-08:00

My configuration errors out with the following error for all three types of sub-logs:

-> Could not use strp to parse time stamp ....



Is it because my configuration is not correct ?
Is there no such thing as one regex for all three types of timestamps ( what I tried to setup in TIME_FORMAT) ?
I dont see the point of adding a MAX _ TIMESTAMP _ LOOKAHEAD here - would that be helpful ?

0 Karma
1 Solution

lguinn2
Legend

I suggest that you leave out the TIME_FORMAT and just have

NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_PREFIX=america-

Splunk is very good at figuring out the time format automatically, and can easily adjust to the fact that there are variations. You also don't need the MAX_TIMESTAMP_LOOKAHEAD, and you probably shouldn't use it if you can't predict the number of characters after america- to the timestamp.

View solution in original post

lguinn2
Legend

I suggest that you leave out the TIME_FORMAT and just have

NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_PREFIX=america-

Splunk is very good at figuring out the time format automatically, and can easily adjust to the fact that there are variations. You also don't need the MAX_TIMESTAMP_LOOKAHEAD, and you probably shouldn't use it if you can't predict the number of characters after america- to the timestamp.

lguinn2
Legend

No, I don't think that the TIME_FORMAT will help you.

Try

TIME_PREFIX=america-.*?:

I think that may work better.

asarolkar
Builder

Hi there,

I tried that and it did not work unfortunately.

Splunk keeps thinking that the first timestamp is the correct timestamp.

Do you think a TIME_FORMAT regex like %d/%b/%Y:%H:%M:%S %Z would be helpful here ?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...