Need help with TIME_PREFIX

[reswob4]

Given a representative sample of my logs:

Jan 25 14:19:20 1.1.1.1 64: Jan 25 22:19:19.281: %LINK-3-UPDOWN: xxxxxxxxxxxxxxxxxxxxxxxxxx
Jan 25 14:19:15 1.1.1.1 74: Jan 25 22:19:15.282: %LINK-3-UPDOWN: xxxxxxxxxxxxxxxxxxxxxxxxxx
Jan 25 14:18:56 1.1.1.1 79: Jan 25 22:18:56.285: %LINK-3-UPDOWN: xxxxxxxxxxxxxxxxxxxxxxxxxx
Jan 25 14:18:25 1.1.1.1 66: Jan 25 22:18:25.284: %LINK-3-UPDOWN: xxxxxxxxxxxxxxxxxxxxxxxxxx
Jan 25 14:18:15 1.1.1.1 62: Jan 25 22:18:15.274: %LINK-3-UPDOWN: xxxxxxxxxxxxxxxxxxxxxxxxxx
Jan 25 14:17:22 1.1.1.1 34: Jan 25 22:17:22.287: %LINK-3-UPDOWN: xxxxxxxxxxxxxxxxxxxxxxxxxx

These logs are being written to a file on my Heavy Forwarder and the HF is monitoring the file and sending to the indexers. Currently I'm testing a new source on a new HF. The same configuration is being used on another HF and I've just copied the props.conf from the previous HF to the new one. But I'm having some weird behavior when trying to extract the time field.

I want to use the second timestamp as the time. Here is my props.conf:

TIME_PREFIX = \w{3}\s+\d{1,2}\s+\d{2}:\d{2}:\d{2}\s+\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\s\d+:\s+
MAX_TIMESTAMP_LOOKAHEAD = 15
TIME_FORMAT = %b\s+%d\s+%H:%M:%S.%3N

Splunk shows the time for all events as _time = "Jan 25 2018 5:19:19 PM"

Which is the time of the top event and the timestamp of the file it is reading from. Which makes me thing that Splunk could not parse the timestamp.

I've tried the following TIME_PREFIX modifications:

TIME_PREFIX = ^(?:[^\n]* ) {5}
TIME_PREFIX = \d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\s\d+:\s+

and I've tried TIME_PREFIX = ^to try to get the first timestamp.

Same issue. Did I make a config mistake, syntax?

I saw other people had found an error that says "Could not use strptime to parse timestamp from xxxxx", but I'm not sure where splunk would write that (splunkd.log?)....

Thanks for any suggestions

[reswob4 · Accepted solution]

Fixed. Not the most elegant, but it works. Took the sourcetype Splunk created in the above step, found the built in Cisco/syslog config, copied it to the created sourcetype, restarted and it works.

for now.

It's ugly but it works.

EDIT: To clarify, that is the sourcetype Splunk created as I describe in my answer above.

[Anam]

Hey reswob4

If you found the solution yourself, make sure you accept the answer.

Thanks

[reswob4]

That didn't work either.

So I copied some logs in to a new file, then ran through the wizard to import those logs into splunk and let splunk create the props.conf.

Here's what Splunk created:

DATETIME_CONFIG = 
NO_BINARY_CHECK = true
category = Custom
pulldown_type = true

That worked.

----- whatever.....

Now the problem is that props.conf is not tied to the syslog autoparsing that splunk does....

problem for another day... I'll update next week...

[mayurr98]

hey try this

TIME_FORMAT = %b %d %H:%M:%S.%3N
TIME_PREFIX = \s\d{1,3}:\s

let me know if this helps!

[reswob4]

Again, this SHOULD work, but doesn't and returns the same result. Is there anywhere in the splunk logs to see where it fails?

[mayurr98]

have you changed your MAX_TIMESTAMP_LOOKAHEAD = 15 to MAX_TIMESTAMP_LOOKAHEAD = 30 ?
Also try putting the same on indexer as well!

look for warning on search head

index=_internal "Failed to parse timestamp"