Getting Data In

Timestamp Extraction

royimad
Builder

I have a log with multiple lines that contains several timestamps. When monitoring the logs splunk is split them into several events , I just need to split on the first timestamp of the begining of the lines with a specific format without considered my log as several events:

I have used TIME_FORMAT and LINE_BREAKER but this isn't help- maybe TIME_PREFIX will help but don't know how to figure it out, MY TIME is only the begin of a newline.

props.conf
----------

[log4j]
NO_BINARY_CHECK = 1
TIME_FORMAT = %a %b %e %Y %k:%M:%S,%3 %Z
LINE_BREAKER = \w{3}\s\w{3}\s\d{2}\s\d{4}\s\d{2}:\d{2}:\d{2}

MYLOG:
-------

<EVENT1>
Thu Feb 28 2013 08:58:39,024 EST ERROR .persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQL
bind => [SH90054, 2013-02-27 10:20:16.425, 2013-02-27 10:20:16.425, null, null, F, MD80828, null]
Query: InsertObjectQuery()
at org...
<EVENT2>

Thu Feb 28 2013 08:60:39,024 EST WARN XYZ

0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

To grab the first time stamp:
MAX_TIMESTAMP_LOOKAHEAD will tell Splunk how far to look before identifying the timestamp.
In other situations, you might use this to take the 2nd timestamp by setting MAX_TIMESTAMP_LOOKAHEAD = 40.

If you would prefer to use LINE_BREAKER, you'll want to review this:
http://docs.splunk.com/Documentation/Splunk/5.0.2/Data/Indexmulti-lineevents

LINE_BREAKER requires a capturing group that defines the newline and carriage returns. It's a little difficult to work with because you need to be sure that you account for the accurate pattern.

You might want to try something like this:

[log4j]
BREAK_ONLY_BEFORE=\w{3}\s\w{3}\s\d{2}\s\d{4}\s\d{2}:\d{2}:\d{2}
MAX_TIMESTAMP_LOOKAHEAD=10
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=true

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Deep Dive: Accelerate threat investigation with Splunk’s AI Assistant in Security

AI is one of the biggest topics in the market today, and for security teams, its value goes far beyond the ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Detection Engineering Office Hours: Real-World Troubleshooting & Q&A

[REGISTER HERE] This thread is for the Community Office Hours session on Detection Engineering Office Hours: ...