How to break events based on timestamp at index-time?

[patouellet]

Hello,

Having a hard time parsing a file the way I need it too. Got a file with events spilling over multiple lines. There is no disnernable event breaking regex I could use to break out events. What seems to make more sense is to recognize multiple lines as a single event based on timestamp. That's what I want to do. I read that I should use the TIME_FORMAT and some other thing to tell Splunk to do that at Index time. But I'm stuck - for some reason I'm getting parsing errors.

Given this sample log line:
[3/14/18 4:00:08:816 EDT] 00000033 SystemErr     R  at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:278)

Using this TIME_FORMAT value:
%m/%d/%y %H:%M:%S:%3N %Z

Why am I getting "Could not use strptime to parse timestamp from "[3/14/18 4:00:08:816 EDT]"

Thank you.

[patouellet]

Thanks for the reply. Looks better now. Timestamps being properly recognised. But it's not breaking events like I want it. It's doing one event per line, but I want all lines with exact same timestamp to be one event. I'm playing in the Add Data interface and can't figure it out. Here's the props.conf settings I have.

I tried BREAK_ONLY_BEFORE=^[ like you suggested but it was still not doing what I want it to do.

[ <SOURCETYPE NAME> ]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
CHARSET=UTF-8
MAX_TIMESTAMP_LOOKAHEAD=60
disabled=false
TIME_FORMAT=%m/%d/%y %H:%M:%S:%3N %Z
TIME_PREFIX=\[

Data example: I would like line 45, 46 and 47 to be 1 event. They all have the same timestamp. Line 48 would be the start of a new event.

45      3/14/18
12:12:41.610 AM 
[3/14/18 0:12:41:610 EDT] 00000039 SystemErr     R  at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
46      3/14/18
12:12:41.610 AM 
[3/14/18 0:12:41:610 EDT] 00000039 SystemErr     R  at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
47      3/14/18
12:12:41.610 AM 
[3/14/18 0:12:41:610 EDT] 00000039 SystemErr     R  at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1662)
48      3/14/18
1:00:02.465 AM  
[3/14/18 1:00:02:465 EDT] 0000006b SystemErr     R com.ibm.wcc.service.intf.ProcessingException

[surekhasplunk]

Hi @patouellet

Did you get your issue resolved as i am also facing the same issue of all events being indexed at he same time i.e. the file creation time of the log file which has the data and not the event time.

Let me know if its working for you.

Thanks

[patouellet]

Hi, I see that was one year ago. I honestly don't remember. But I was able to find the settings I'm currently using. Here they are. Hopefully it helps you out.

[your_sourcetype]
DATETIME_CONFIG =
MAX_TIMESTAMP_LOOKAHEAD = 60
NO_BINARY_CHECK = true
TIME_FORMAT = %m/%d/%y %H:%M:%S:%3N %Z
TIME_PREFIX = \[
category = Application
disabled = false
pulldown_type = true

[yannK]

You should read those manual first :
http://docs.splunk.com/Documentation/Splunk/7.0.2/Data/Configureeventlinebreaking
http://docs.splunk.com/Documentation/Splunk/7.0.2/Data/Configuretimestamprecognition

then you can upload a sample to your search-head and use the wizard to create and test a sourcetype.
then when working, export the sourcetype props.conf config and deploy it to your parsing layer (indexers, and heavy forwarders)

I suspect that the sourcetype will look like :

[custom_sourcetype]
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=^\[
TIME_FORMAT=%m/%d/%y %H:%M:%S:%3N %Z
TIME_PREFIX=\[
MAX_TIMESTAMP_LOOKAHEAD=60

[niketn]

@yannK [Splunk], there is an extra percent after %m it should be TIME_FORMAT=%m/%d/%y %H:%M:%S:%3N %Z

[yannK]

oh yes, it's a typo, let me edit it