How to break Weblogic JVM events using regex with multiple timestamps & exceptions in single event

[p_gurav]

Hi All,

I have the following JVM logs:

May 8, 2016 1:26:26 AM IST Warning Socket BEA-000449 Closing socket as no data read from it on x.x.x.x:x,x during the configured idle timeout of 5 secs
01:45:05.078 [Listener-null] ERROR c.c.s.ListenerHandler - Error occurred while delivering response message for listener rilBPMService.createServiceOrderInterface.createOrder.
java.lang.NullPointerException
        at com.conceptwave.serviceprovider.MsgQueueListener$MsgQueueHandler.doCreateOutConnection(MsgQueueListener.java:250)
        at com.conceptwave.serviceprovider.ListenerHandler.createOutConnection(ListenerHandler.java:517)
        at com.conceptwave.serviceprovider.ListenerHandler.run(ListenerHandler.java:281)
        at com.conceptwave.serviceprovider.CwfListener.run(CwfListener.java:320)
02:00:12.712 [Listener-null] ERROR c.c.s.ListenerHandler - Error occurred while delivering response message for listener rilBPMService.createServiceOrderInterface.createOrder.
java.lang.NullPointerException
        at com.conceptwave.serviceprovider.MsgQueueListener$MsgQueueHandler.doCreateOutConnection(MsgQueueListener.java:250)
        at com.conceptwave.serviceprovider.ListenerHandler.createOutConnection(ListenerHandler.java:517)
        at com.conceptwave.serviceprovider.ListenerHandler.run(ListenerHandler.java:281)
        at com.conceptwave.serviceprovider.CwfListener.run(CwfListener.java:320)
02:00:14.008 [Listener-null] ERROR c.c.s.ListenerHandler - Error occurred while delivering response message for listener rilBPMService.serviceOrderNotificationInterface.serviceOrderNotification.
java.lang.NullPointerException
        at com.conceptwave.serviceprovider.MsgQueueListener$MsgQueueHandler.doCreateOutConnection(MsgQueueListener.java:250)
        at com.conceptwave.serviceprovider.ListenerHandler.createOutConnection(ListenerHandler.java:517)
        at com.conceptwave.serviceprovider.ListenerHandler.run(ListenerHandler.java:281)
        at com.conceptwave.serviceprovider.CwfListener.run(CwfListener.java:320)
02:00:46.377 [Listener-null] ERROR c.c.s.ListenerHandler - Error occurred while delivering response message for listener rilBPMService.createServiceOrderInterface.createOrder.
java.lang.NullPointerException
        at com.conceptwave.serviceprovider.MsgQueueListener$MsgQueueHandler.doCreateOutConnection(MsgQueueListener.java:250)
        at com.conceptwave.serviceprovider.ListenerHandler.createOutConnection(ListenerHandler.java:517)
        at com.conceptwave.serviceprovider.ListenerHandler.run(ListenerHandler.java:281)
        at com.conceptwave.serviceprovider.CwfListener.run(CwfListener.java:320)

I want Splunk to consider above event as one event. But as it has multiple timestamps without date.

I tried following properties in props.conf:

BREAK_ONLY_BEFORE = \<\w\w\w\s\d{2},\s\d{4}\s\d+:\d{2}:\d{2}\>
SHOULD_LINEMERGE = true
MAX_EVENTS= 10000
TIME_FORMAT=%b %d, %Y %H:%M:%S %p
TIME_PREFIX=^

In this case Splunk is breaking event whenever it encounters time (i.e. at each exception), creating multiple events for each exception instead of single event.

If I don't use the BREAK_ONLY_BEFORE in props, then Splunk is still breaking the event whenever it encounters time, but not indexing the exception part. I end up with missing data.

Could anybody will help me with this?
Thank You in advance

[michael_sleep]

This would work as a props configuration. Just take whatever you need. The regex may be different if only because I can't tell if your timestamp line starts with a space or not, so I've included both. This one will assume there is no space:

SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=^(\w{3}\s\d{1,2}\,\s\d{4}\s\d{1,2}:\d{2}:\d{2}\s\w{2}\s\w{3})
MAX_TIMESTAMP_LOOKAHEAD=27

If there is a space:

^(\s\w{3}\s\d{1,2}\,\s\d{4}\s\d{1,2}:\d{2}:\d{2}\s\w{2}\s\w{3})

[campbellj1977]

It looks like you are defining your time prefix to start of new line, but no line breaks mentioned. Try this;

SHOULD_LINEMERGE = false
LINE_BREAKER =  \w{3}\s\w{1,2}\,\s\d{4}\s\d{1,2}\:\d{1,2}\:\d{1,2}\s\w{2}
TIME_FORMAT=%b %d, %Y %H:%M:%S %p
TIME_PREFIX=^
LOOK_AHEAD = 45

I believe this is best practice all props.

Thanks,
Joshua

[woodcock]

Try this:

TIME_PREFIX = ^
TIME_FORMAT = %b %d, %Y %H:%M:%S %p
BREAK_ONLY_BEFORE_DATE = true
SHOULD_LINEMERGE = true