Getting Data In

Linebreaking problem on a small amount of events

mightaswelby
Explorer

Having an issue with a linebreak. Seems most events are breaking properly, but a small number are not.

I think this may be due to the ',' following the capture, which is inconsistent in the logs.

Any suggestions to break on this for the remaining events not working?

Existing props.conf
[  ]
SHOULD_LINEMERGE=true
LINE_BREAKER=StandartDeviation=NaN
NO_BINARY_CHECK=true
CHARSET=UTF-8
disabled=false

Raw Data: (**bolded** instances not breaking properly)

2019-06-05 09:35:33,617 - AOB:com.fnf.xes.PRF:IntRateInqRq=ServiceName=IntRateInqRq, ChannelName=AOB, SPName=com.fnf.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=0, sqrSuccessTotalTime=0, Id=IntRateInqRq, TotalCount=1, TotalTime=84, TotalMinTime=84, TotalMaxTime=84, TotalAvgTime=84.0, **StandartDeviation=NaN,** null:com.fnis.xes.PRF:com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO=ServiceName=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, ChannelName=null, SPName=com.fnis.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=9, sqrSuccessTotalTime=0, Id=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, TotalCount=2, TotalTime=20, TotalMinTime=9, TotalMaxTime=11, TotalAvgTime=10.0, StandartDeviation=NaN

2019-06-05 09:54:23,352 - null:null:IFXServiceBase=EndPointName=IFXServiceBase, Protocol=null, ReqMsgTotalLength=0, ReqMinLength=0, ReqMaxLength=0, ResMsgTotalLength=0, ResMinLength=0, ResMaxLength=0, ServiceName=IFXServiceBase, ChannelName=null, SPName=null, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=68, sqrSuccessTotalTime=0, Id=IFXServiceBase, TotalCount=3, TotalTime=217, TotalMinTime=68, TotalMaxTime=81, TotalAvgTime=72.0, StandartDeviation=NaN

2019-06-05 09:48:51,991 - AOB:com.fnf.xes.PRF:IntRateInqRq=ServiceName=IntRateInqRq, ChannelName=AOB, SPName=com.fnf.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=0, sqrSuccessTotalTime=0, Id=IntRateInqRq, TotalCount=1, TotalTime=63, TotalMinTime=63, TotalMaxTime=63, TotalAvgTime=63.0, **StandartDeviation=NaN,** null:com.fnis.xes.PRF:com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO=ServiceName=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, ChannelName=null, SPName=com.fnis.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=35, sqrSuccessTotalTime=0, Id=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, TotalCount=2, TotalTime=68, TotalMinTime=33, TotalMaxTime=35, TotalAvgTime=34.0, StandartDeviation=NaN

2019-06-05 09:48:51,991 - null:null:SOAP_IFX20=EndPointName=SOAP_IFX20, Protocol=null, ReqMsgTotalLength=0, ReqMinLength=0, ReqMaxLength=0, ResMsgTotalLength=0, ResMinLength=0, ResMaxLength=0, ServiceName=SOAP_IFX20, ChannelName=null, SPName=null, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=38, sqrSuccessTotalTime=0, Id=SOAP_IFX20, TotalCount=2, TotalTime=74, TotalMinTime=36, TotalMaxTime=38, TotalAvgTime=37.0, **StandartDeviation=NaN,** null:null:IFXServiceBase=EndPointName=IFXServiceBase, Protocol=null, ReqMsgTotalLength=0, ReqMinLength=0, ReqMaxLength=0, ResMsgTotalLength=0, ResMinLength=0, ResMaxLength=0, ServiceName=IFXServiceBase, ChannelName=null, SPName=null, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=0, sqrSuccessTotalTime=0, Id=IFXServiceBase, TotalCount=1, TotalTime=67, TotalMinTime=67, TotalMaxTime=67, TotalAvgTime=67.0, StandartDeviation=NaN

2019-06-05 09:45:33,617 - AOB:com.fnf.xes.PRF:IntRateInqRq=ServiceName=IntRateInqRq, ChannelName=AOB, SPName=com.fnf.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=81, sqrSuccessTotalTime=0, Id=IntRateInqRq, TotalCount=2, TotalTime=141, TotalMinTime=60, TotalMaxTime=81, TotalAvgTime=70.0, StandartDeviation=NaN

2019-06-05 09:45:33,617 - null:null:IFXServiceBase=EndPointName=IFXServiceBase, Protocol=null, ReqMsgTotalLength=0, ReqMinLength=0, ReqMaxLength=0, ResMsgTotalLength=0, ResMinLength=0, ResMaxLength=0, ServiceName=IFXServiceBase, ChannelName=null, SPName=null, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=86, sqrSuccessTotalTime=0, Id=IFXServiceBase, TotalCount=2, TotalTime=150, TotalMinTime=64, TotalMaxTime=86, TotalAvgTime=75.0, StandartDeviation=NaN

2019-06-05 09:44:23,352 - null:com.fnis.xes.PRF:com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO=ServiceName=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, ChannelName=null, SPName=com.fnis.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=10, sqrSuccessTotalTime=0, Id=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, TotalCount=2, TotalTime=20, TotalMinTime=10, TotalMaxTime=10, TotalAvgTime=10.0, StandartDeviation=NaN

2019-06-05 09:44:23,352 - null:null:SOAP_IFX20=EndPointName=SOAP_IFX20, Protocol=null, ReqMsgTotalLength=0, ReqMinLength=0, ReqMaxLength=0, ResMsgTotalLength=0, ResMinLength=0, ResMaxLength=0, ServiceName=SOAP_IFX20, ChannelName=null, SPName=null, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=12, sqrSuccessTotalTime=0, Id=SOAP_IFX20, TotalCount=2, TotalTime=24, TotalMinTime=12, TotalMaxTime=12, TotalAvgTime=12.0, StandartDeviation=NaN

2019-06-05 09:38:51,991 - AOB:com.fnf.xes.PRF:IntRateInqRq=ServiceName=IntRateInqRq, ChannelName=AOB, SPName=com.fnf.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=72, sqrSuccessTotalTime=0, Id=IntRateInqRq, TotalCount=2, TotalTime=142, TotalMinTime=70, TotalMaxTime=72, TotalAvgTime=71.0, StandartDeviation=NaN
0 Karma
1 Solution

oscar84x
Contributor

I deleted my original answer since I misunderstood your question. You DO want to break the events up further at those points.
Use the following as your regex: StandartDeviation=NaN(, )|StandartDeviation=NaN()
This will break up every event after every "StandartDeviation=NaN" and get rid of the comma and the space at the beginning of the new event. Seems like there's no time stamp for that second event though.

View solution in original post

0 Karma

oscar84x
Contributor

I deleted my original answer since I misunderstood your question. You DO want to break the events up further at those points.
Use the following as your regex: StandartDeviation=NaN(, )|StandartDeviation=NaN()
This will break up every event after every "StandartDeviation=NaN" and get rid of the comma and the space at the beginning of the new event. Seems like there's no time stamp for that second event though.

0 Karma

mightaswelby
Explorer

worked perfect, thank you!

oscar84x
Contributor

Excellent! Thank you.

0 Karma
Get Updates on the Splunk Community!

Join Us for Splunk University and Get Your Bootcamp Game On!

If you know, you know! Splunk University is the vibe this summer so register today for bootcamps galore ...

.conf24 | Learning Tracks for Security, Observability, Platform, and Developers!

.conf24 is taking place at The Venetian in Las Vegas from June 11 - 14. Continue reading to learn about the ...

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...