topic Re: How should I extract the time stamp when it appears in two different formats and locations? in Getting Data In

How should I extract the time stamp when it appears in two different formats and locations?

ddrillic — Tue, 18 Jun 2019 17:16:39 GMT

We have data that comes in two different formats -

Jun 18 14:02:21 <host> DataCollector: [0x7f08f6ffd700] INFO  Metrics null - {"snapshot":[{"Syslog":{"totalBytesReceived":{"count":80535209337320,"timestamp":"20190618T140221.616466"},...

Jun 18 14:02:19 <host> DataCollector: [0x7f4e0b2c1700] INFO  RevisionManager null....

I did the following which works fine for the first case, but not the second, obviously ; -)

[syslog<case>]
TRANSFORMS-host_override = host_override
LINE_BREAKER=([\r\n]+)\S+\s\d+\s\d{2}:\d{2}:\d{2}
TIME_PREFIX=\"timestamp\":\"
TIME_FORMAT=%Y%m%dT%H%M%S.%6N
MAX_TIMESTAMP_LOOKAHEAD=50
TZ = UTC
TRUNCATE=10000
SHOULD_LINEMERGE=false
disabled=false

How can I handle the second case of the log? Here there isn't any other choice besides the time stamp at the beginning of the line.

Re: How should I extract the time stamp when it appears in two different formats and locations?

DavidHourani — Tue, 18 Jun 2019 17:45:46 GMT

Hi @ddrillic,

Ouch...how did you get into that hole ?

How about routing each into a different sourcetype and applying the right time format there ?

If you try to apply a match on this format : Jun 18 14:02:19 even if it's conditional it will match for both so no way out that.

Cheers,
David

Re: How should I extract the time stamp when it appears in two different formats and locations?

ddrillic — Tue, 18 Jun 2019 17:48:04 GMT

I know - it's a cute one ; -)