Getting Data In

Timestamp Extraction

royimad
Builder

I have a log with multiple lines that contains several timestamps. When monitoring the logs splunk is split them into several events , I just need to split on the first timestamp of the begining of the lines with a specific format without considered my log as several events:

I have used TIME_FORMAT and LINE_BREAKER but this isn't help- maybe TIME_PREFIX will help but don't know how to figure it out, MY TIME is only the begin of a newline.

props.conf
----------

[log4j]
NO_BINARY_CHECK = 1
TIME_FORMAT = %a %b %e %Y %k:%M:%S,%3 %Z
LINE_BREAKER = \w{3}\s\w{3}\s\d{2}\s\d{4}\s\d{2}:\d{2}:\d{2}

MYLOG:
-------

<EVENT1>
Thu Feb 28 2013 08:58:39,024 EST ERROR .persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQL
bind => [SH90054, 2013-02-27 10:20:16.425, 2013-02-27 10:20:16.425, null, null, F, MD80828, null]
Query: InsertObjectQuery()
at org...
<EVENT2>

Thu Feb 28 2013 08:60:39,024 EST WARN XYZ

0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

To grab the first time stamp:
MAX_TIMESTAMP_LOOKAHEAD will tell Splunk how far to look before identifying the timestamp.
In other situations, you might use this to take the 2nd timestamp by setting MAX_TIMESTAMP_LOOKAHEAD = 40.

If you would prefer to use LINE_BREAKER, you'll want to review this:
http://docs.splunk.com/Documentation/Splunk/5.0.2/Data/Indexmulti-lineevents

LINE_BREAKER requires a capturing group that defines the newline and carriage returns. It's a little difficult to work with because you need to be sure that you account for the accurate pattern.

You might want to try something like this:

[log4j]
BREAK_ONLY_BEFORE=\w{3}\s\w{3}\s\d{2}\s\d{4}\s\d{2}:\d{2}:\d{2}
MAX_TIMESTAMP_LOOKAHEAD=10
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=true

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
Get Updates on the Splunk Community!

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...

Cloud Monitoring Console - Unlocking Greater Visibility in SVC Usage Reporting

For Splunk Cloud customers, understanding and optimizing Splunk Virtual Compute (SVC) usage and resource ...

Automatic Discovery Part 3: Practical Use Cases

If you’ve enabled Automatic Discovery in your install of the Splunk Distribution of the OpenTelemetry ...