Getting Data In

Linebreaking problem on a small amount of events

mightaswelby
Explorer

Having an issue with a linebreak. Seems most events are breaking properly, but a small number are not.

I think this may be due to the ',' following the capture, which is inconsistent in the logs.

Any suggestions to break on this for the remaining events not working?

Existing props.conf
[  ]
SHOULD_LINEMERGE=true
LINE_BREAKER=StandartDeviation=NaN
NO_BINARY_CHECK=true
CHARSET=UTF-8
disabled=false

Raw Data: (**bolded** instances not breaking properly)

2019-06-05 09:35:33,617 - AOB:com.fnf.xes.PRF:IntRateInqRq=ServiceName=IntRateInqRq, ChannelName=AOB, SPName=com.fnf.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=0, sqrSuccessTotalTime=0, Id=IntRateInqRq, TotalCount=1, TotalTime=84, TotalMinTime=84, TotalMaxTime=84, TotalAvgTime=84.0, **StandartDeviation=NaN,** null:com.fnis.xes.PRF:com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO=ServiceName=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, ChannelName=null, SPName=com.fnis.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=9, sqrSuccessTotalTime=0, Id=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, TotalCount=2, TotalTime=20, TotalMinTime=9, TotalMaxTime=11, TotalAvgTime=10.0, StandartDeviation=NaN

2019-06-05 09:54:23,352 - null:null:IFXServiceBase=EndPointName=IFXServiceBase, Protocol=null, ReqMsgTotalLength=0, ReqMinLength=0, ReqMaxLength=0, ResMsgTotalLength=0, ResMinLength=0, ResMaxLength=0, ServiceName=IFXServiceBase, ChannelName=null, SPName=null, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=68, sqrSuccessTotalTime=0, Id=IFXServiceBase, TotalCount=3, TotalTime=217, TotalMinTime=68, TotalMaxTime=81, TotalAvgTime=72.0, StandartDeviation=NaN

2019-06-05 09:48:51,991 - AOB:com.fnf.xes.PRF:IntRateInqRq=ServiceName=IntRateInqRq, ChannelName=AOB, SPName=com.fnf.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=0, sqrSuccessTotalTime=0, Id=IntRateInqRq, TotalCount=1, TotalTime=63, TotalMinTime=63, TotalMaxTime=63, TotalAvgTime=63.0, **StandartDeviation=NaN,** null:com.fnis.xes.PRF:com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO=ServiceName=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, ChannelName=null, SPName=com.fnis.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=35, sqrSuccessTotalTime=0, Id=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, TotalCount=2, TotalTime=68, TotalMinTime=33, TotalMaxTime=35, TotalAvgTime=34.0, StandartDeviation=NaN

2019-06-05 09:48:51,991 - null:null:SOAP_IFX20=EndPointName=SOAP_IFX20, Protocol=null, ReqMsgTotalLength=0, ReqMinLength=0, ReqMaxLength=0, ResMsgTotalLength=0, ResMinLength=0, ResMaxLength=0, ServiceName=SOAP_IFX20, ChannelName=null, SPName=null, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=38, sqrSuccessTotalTime=0, Id=SOAP_IFX20, TotalCount=2, TotalTime=74, TotalMinTime=36, TotalMaxTime=38, TotalAvgTime=37.0, **StandartDeviation=NaN,** null:null:IFXServiceBase=EndPointName=IFXServiceBase, Protocol=null, ReqMsgTotalLength=0, ReqMinLength=0, ReqMaxLength=0, ResMsgTotalLength=0, ResMinLength=0, ResMaxLength=0, ServiceName=IFXServiceBase, ChannelName=null, SPName=null, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=0, sqrSuccessTotalTime=0, Id=IFXServiceBase, TotalCount=1, TotalTime=67, TotalMinTime=67, TotalMaxTime=67, TotalAvgTime=67.0, StandartDeviation=NaN

2019-06-05 09:45:33,617 - AOB:com.fnf.xes.PRF:IntRateInqRq=ServiceName=IntRateInqRq, ChannelName=AOB, SPName=com.fnf.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=81, sqrSuccessTotalTime=0, Id=IntRateInqRq, TotalCount=2, TotalTime=141, TotalMinTime=60, TotalMaxTime=81, TotalAvgTime=70.0, StandartDeviation=NaN

2019-06-05 09:45:33,617 - null:null:IFXServiceBase=EndPointName=IFXServiceBase, Protocol=null, ReqMsgTotalLength=0, ReqMinLength=0, ReqMaxLength=0, ResMsgTotalLength=0, ResMinLength=0, ResMaxLength=0, ServiceName=IFXServiceBase, ChannelName=null, SPName=null, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=86, sqrSuccessTotalTime=0, Id=IFXServiceBase, TotalCount=2, TotalTime=150, TotalMinTime=64, TotalMaxTime=86, TotalAvgTime=75.0, StandartDeviation=NaN

2019-06-05 09:44:23,352 - null:com.fnis.xes.PRF:com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO=ServiceName=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, ChannelName=null, SPName=com.fnis.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=10, sqrSuccessTotalTime=0, Id=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, TotalCount=2, TotalTime=20, TotalMinTime=10, TotalMaxTime=10, TotalAvgTime=10.0, StandartDeviation=NaN

2019-06-05 09:44:23,352 - null:null:SOAP_IFX20=EndPointName=SOAP_IFX20, Protocol=null, ReqMsgTotalLength=0, ReqMinLength=0, ReqMaxLength=0, ResMsgTotalLength=0, ResMinLength=0, ResMaxLength=0, ServiceName=SOAP_IFX20, ChannelName=null, SPName=null, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=12, sqrSuccessTotalTime=0, Id=SOAP_IFX20, TotalCount=2, TotalTime=24, TotalMinTime=12, TotalMaxTime=12, TotalAvgTime=12.0, StandartDeviation=NaN

2019-06-05 09:38:51,991 - AOB:com.fnf.xes.PRF:IntRateInqRq=ServiceName=IntRateInqRq, ChannelName=AOB, SPName=com.fnf.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=72, sqrSuccessTotalTime=0, Id=IntRateInqRq, TotalCount=2, TotalTime=142, TotalMinTime=70, TotalMaxTime=72, TotalAvgTime=71.0, StandartDeviation=NaN
0 Karma
1 Solution

oscar84x
Contributor

I deleted my original answer since I misunderstood your question. You DO want to break the events up further at those points.
Use the following as your regex: StandartDeviation=NaN(, )|StandartDeviation=NaN()
This will break up every event after every "StandartDeviation=NaN" and get rid of the comma and the space at the beginning of the new event. Seems like there's no time stamp for that second event though.

View solution in original post

0 Karma

oscar84x
Contributor

I deleted my original answer since I misunderstood your question. You DO want to break the events up further at those points.
Use the following as your regex: StandartDeviation=NaN(, )|StandartDeviation=NaN()
This will break up every event after every "StandartDeviation=NaN" and get rid of the comma and the space at the beginning of the new event. Seems like there's no time stamp for that second event though.

0 Karma

mightaswelby
Explorer

worked perfect, thank you!

oscar84x
Contributor

Excellent! Thank you.

0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...