I have researched this error previously (and found a lot of helpful material).
I am stuck with a slightly complicated variation of this commonly known problem.
http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition
I need to extract the second timestamp from a certain log file.
The log file has different kinds of sub-log-types merged into one giant log file.
Which means, I need to extract the second timestamp (that presents itself at a varying number of characters distance from the FIRST useless time stamps)
Mar 4 10:05:02 america-p01 access_combined: 10.142.1.109 - - [04/Mar/2013:02:05:03 -0800] "GET /healthCheck/status " 200 13 "-" "-"
Mar 4 10:05:10 america-p01 syslog: 2013-03-04 02:05:11,771 INFO [http-0.0.0.0-8080-3] -TpaiL5RBCo4-CH-Fjo9rw__ ERI IdsPatientLogger - Logging the CREATE of Account: 464c-9f5c-074ab072ee58 by User: ERI
Mar 4 10:06:27 america-p01 auditlog: AuditEntry[event=LoginRequest,ip=,date=2013-03-04T02:06:28.057-08:00,user=olivia,status=Success,description=]
My props.conf looks like this
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_FORMAT=%d/%b/%Y:%H:%M:%S %Z
TIME_PREFIX=america-
What I expect is for Splunk to recognize the following as correct timestamps and use these SECOND timestamps instead
i) For access_combined -> [04/Mar/2013:02:05:03 -0800]
ii) For syslog -> 2013-03-04 02:05:11,771
iii) For auditlog -> 2013-03-04T02:06:28.057-08:00
My configuration errors out with the following error for all three types of sub-logs:
-> Could not use strp to parse time stamp ....
Is it because my configuration is not correct ?
Is there no such thing as one regex for all three types of timestamps ( what I tried to setup in TIME_FORMAT) ?
I dont see the point of adding a MAX _ TIMESTAMP _ LOOKAHEAD here - would that be helpful ?
I suggest that you leave out the TIME_FORMAT
and just have
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_PREFIX=america-
Splunk is very good at figuring out the time format automatically, and can easily adjust to the fact that there are variations. You also don't need the MAX_TIMESTAMP_LOOKAHEAD
, and you probably shouldn't use it if you can't predict the number of characters after america-
to the timestamp.
I suggest that you leave out the TIME_FORMAT
and just have
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_PREFIX=america-
Splunk is very good at figuring out the time format automatically, and can easily adjust to the fact that there are variations. You also don't need the MAX_TIMESTAMP_LOOKAHEAD
, and you probably shouldn't use it if you can't predict the number of characters after america-
to the timestamp.
No, I don't think that the TIME_FORMAT
will help you.
Try
TIME_PREFIX=america-.*?:
I think that may work better.
Hi there,
I tried that and it did not work unfortunately.
Splunk keeps thinking that the first timestamp is the correct timestamp.
Do you think a TIME_FORMAT regex like %d/%b/%Y:%H:%M:%S %Z would be helpful here ?