Getting Data In

How to break multi-line event logfile so event starts when date found at beginning of a line?

a212830
Champion

Hi,

I have a multi-line event logfile that I'm having issues with. I want to say that an event starts when it finds date at the beginning of a line. The output looks like this (the number of lines between dates can vary widely - this is just a sample.

09 Jan 2014 02:02:13,477 [TaskEngineWorker-pool-1] [::] INFO tasks.AnalyticsCachePrimerETL - AnalyticsCachePrimerETL: Task started
09 Jan 2014 02:02:15,436 [TaskEngineWorker-pool-1] [::] INFO tasks.AnalyticsCachePrimerETL - AnalyticsCachePrimerETL: Task completed in 1958 milliseconds
09 Jan 2014 02:02:42,087 [TaskEngineWorker-pool-1] [::] INFO impl.VantageAsyncRestCall -
ParentResourceId=;
ResourceDisplayName=;
ResourceId=2020-78323;
ResourceType=USERCONTAINER;
ActionType=EDIT;
ApplicationId=1000000;
EventTime=1389250824323;
EventType=DISCUSSION;
Modality=COLLAB;
FirstName=;
LastName=;
Managed=true;
MimeType=html;
09 Jan 2014 02:25:40,424 [TaskEngineWorker-pool-1] [::] ERROR emailwhitelist.WhitelistEmailManagerImpl - UAT EMAIL WHITELIST PLUGIN INSTALLED. ONLY ALLOWING EMAIL TO A RESTRICTED SET OF USERS!!!!
09 Jan 2014 02:25:40,424 [TaskEngineWorker-pool-1] [::] ERROR emailwhitelist.WhitelistEmailManagerImpl - UAT EMAIL WHITELIST PLUGIN INSTALLED. ONLY ALLOWING EMAIL TO A RESTRICTED SET OF USERS!!!!

Can someone help me?

Tags (1)
0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

It's hard to tell where the actual line breaks are in your file... (whether we are seeing real ones or a wrap here)
But basically you want something like this:

BREAK_ONLY_BEFORE=\d{2}\s+[JFMASOND][aepuco][nbrylgptvc]\s+\d{4}\s\d{2}:\d{2}:\d{2},\d{3}
MAX_TIMESTAMP_LOOKAHEAD=30
SHOULD_LINEMERGE=true

I have literally "spelled out" the structure of the timestamp. Being so specific is probably overkill...
However, to line break on a multiline event file. You specify something that will allow Splunk to see where to break. Tell it where the timestamp is and then tell it to merge the lines in between the "markers".

Also the way I've done it... it doesn't care about or use any actual carriage returns. However if you have them, and you can count on them showing up in a consistent place, you can use them too...
If you bring a sample into the Previewer, you will see how these things are applied.

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!

lguinn2
Legend

What is in props.conf right now? What sourcetype is assigned to this data?

0 Karma
Get Updates on the Splunk Community!

Index This | What is broken 80% of the time by February?

December 2025 Edition   Hayyy Splunk Education Enthusiasts and the Eternally Curious!    We’re back with this ...

Unlock Faster Time-to-Value on Edge and Ingest Processor with New SPL2 Pipeline ...

Hello Splunk Community,   We're thrilled to share an exciting update that will help you manage your data more ...

Splunk MCP & Agentic AI: Machine Data Without Limits

Discover how the Splunk Model Context Protocol (MCP) Server can revolutionize the way your organization uses ...