Splunk Split automatically 2 events when it detect two different dates in a log

[royimad]

I have a log generated from log4j:
an event of this log could be described by the following:

Fri Jan 04 2013 13:05:34,114 EST ERROR wavemark.webapp.interceptors.WmExceptionInterceptor - WaveMarkException occurred
wavemark.common.exceptions.WaveMarkException: Error while calling method [getReportData] in delegate [ReportSessionDelegate]
at wavemark.webapp.delegates.ReportSessionDelegate.getReportData(ReportSessionDelegate.java:52)
...multipleline
RepName, LastCabinetID, ExpectedDeliveryDate From MFR_MissingItem_RP, DB, N/A, dany.ostamdtru, null, 2013-03-06 10:58:30.974, 170, MEDTRONIC]
Query: InsertObjectQuery(wavemark.core.entities.PerformanceTiming@1d46a898)
at org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:699)

The above event contain 2 date one in the header top of the event and one on a line ( the second dates could exist or not ) depending on the stacktrace.

The Question why Splunk index this a 2 separated event and how to prevent from this to happen. I only need Splunk to consider what begin with a date as a single event.

[sowings · Accepted solution]

The default behavior is to consider a new timestamp to be a new event. You can do a couple of things:

• Change your TIME_FORMAT to be explicit about which time stamps are the "real" ones for an event
• Configure MAX_TIMESTAMP_LOOKAHEAD to only go "so far" into the event looking for a time stamp

If these don't work, you might have to resort to the following:

• Turn off BREAK_ONLY_BEFORE_DATE
• Consider turning off SHOULD_LINEMERGE, coupling it with an explicit regex in your LINE_BREAKER.

[royimad]

do you think i need a LINE_BREAKER in my case?
\w{3}\s\w{3}\s\d{2}\s\d{4}\s\d{2}:\d{2}:\d{2}

[sowings]

Yes, you're right that it's props.conf. The stanza name will depend upon on what you have defined for the sourcetype of the data in inputs.conf. Let's say it's called "mylog4j". The stanza would then look like:

See strftime.net for hints on the format characters in TIME_FORMAT.


[mylog4j]
TIME_FORMAT = %a %b %d %Y %H:%M:%S,%3N %z
MAX_TIMESTAMP_LOOKAHEAD = 30


[royimad]

In the props.conf ?

[royimad]

Thanks for the Hint, but where should i edit this parameters? Can you give me an example?