topic Extract timestamp from the logs in Getting Data In

Extract timestamp from the logs

isha_rastogi — Tue, 29 Sep 2020 19:13:55 GMT

I've logs where events are not starting with time. Log format is
10.100.28.108 - - 2018-04-25--02-31-14 "PUT /mifs/c/i/abc/abc.html?c=1073768600 HTTP/1.1" 200 20 "-" "abc/1.0" 5252
I was trying below but getting error: couldnot use strptime to parse
[ test ]
SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true
TIME_FORMAT=%Y-%m-%d--%H%-M-%S
TIME_PREFIX=\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}\s-\s-
TZ=America/New_York

Re: Extract timestamp from the logs

p_gurav — Wed, 25 Apr 2018 10:49:45 GMT

Can you try to edit your time_prefix:

TIME_PREFIX=\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\s-\s-

Re: Extract timestamp from the logs

somesoni2 — Wed, 25 Apr 2018 14:27:02 GMT

Give this a try

[ test ]
SHOULD_LINEMERGE=false
LINE_BREAKER = ([\r\n]+)(?=\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}(\s+\S+){2}\s)
NO_BINARY_CHECK=true
TIME_FORMAT=%Y-%m-%d--%H%-M-%S
TIME_PREFIX=^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}(\s+\S+){2}\s
MAX_TIMESTAMP_LOOKAHEAD = 20
TZ=America/New_York

Re: Extract timestamp from the logs

TISKAR — Wed, 25 Apr 2018 15:09:53 GMT

Can you try this please:

[ __auto__learned__ ]
SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true
TIME_FORMAT=%Y-%m-%d--%H-%M-%S
TIME_PREFIX=(\d+\.){3}.\d+(\s-){2}\s