Getting Data In

Linebreaking problem on a small amount of events

mightaswelby
Explorer

Having an issue with a linebreak. Seems most events are breaking properly, but a small number are not.

I think this may be due to the ',' following the capture, which is inconsistent in the logs.

Any suggestions to break on this for the remaining events not working?

Existing props.conf
[  ]
SHOULD_LINEMERGE=true
LINE_BREAKER=StandartDeviation=NaN
NO_BINARY_CHECK=true
CHARSET=UTF-8
disabled=false

Raw Data: (**bolded** instances not breaking properly)

2019-06-05 09:35:33,617 - AOB:com.fnf.xes.PRF:IntRateInqRq=ServiceName=IntRateInqRq, ChannelName=AOB, SPName=com.fnf.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=0, sqrSuccessTotalTime=0, Id=IntRateInqRq, TotalCount=1, TotalTime=84, TotalMinTime=84, TotalMaxTime=84, TotalAvgTime=84.0, **StandartDeviation=NaN,** null:com.fnis.xes.PRF:com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO=ServiceName=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, ChannelName=null, SPName=com.fnis.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=9, sqrSuccessTotalTime=0, Id=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, TotalCount=2, TotalTime=20, TotalMinTime=9, TotalMaxTime=11, TotalAvgTime=10.0, StandartDeviation=NaN

2019-06-05 09:54:23,352 - null:null:IFXServiceBase=EndPointName=IFXServiceBase, Protocol=null, ReqMsgTotalLength=0, ReqMinLength=0, ReqMaxLength=0, ResMsgTotalLength=0, ResMinLength=0, ResMaxLength=0, ServiceName=IFXServiceBase, ChannelName=null, SPName=null, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=68, sqrSuccessTotalTime=0, Id=IFXServiceBase, TotalCount=3, TotalTime=217, TotalMinTime=68, TotalMaxTime=81, TotalAvgTime=72.0, StandartDeviation=NaN

2019-06-05 09:48:51,991 - AOB:com.fnf.xes.PRF:IntRateInqRq=ServiceName=IntRateInqRq, ChannelName=AOB, SPName=com.fnf.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=0, sqrSuccessTotalTime=0, Id=IntRateInqRq, TotalCount=1, TotalTime=63, TotalMinTime=63, TotalMaxTime=63, TotalAvgTime=63.0, **StandartDeviation=NaN,** null:com.fnis.xes.PRF:com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO=ServiceName=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, ChannelName=null, SPName=com.fnis.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=35, sqrSuccessTotalTime=0, Id=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, TotalCount=2, TotalTime=68, TotalMinTime=33, TotalMaxTime=35, TotalAvgTime=34.0, StandartDeviation=NaN

2019-06-05 09:48:51,991 - null:null:SOAP_IFX20=EndPointName=SOAP_IFX20, Protocol=null, ReqMsgTotalLength=0, ReqMinLength=0, ReqMaxLength=0, ResMsgTotalLength=0, ResMinLength=0, ResMaxLength=0, ServiceName=SOAP_IFX20, ChannelName=null, SPName=null, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=38, sqrSuccessTotalTime=0, Id=SOAP_IFX20, TotalCount=2, TotalTime=74, TotalMinTime=36, TotalMaxTime=38, TotalAvgTime=37.0, **StandartDeviation=NaN,** null:null:IFXServiceBase=EndPointName=IFXServiceBase, Protocol=null, ReqMsgTotalLength=0, ReqMinLength=0, ReqMaxLength=0, ResMsgTotalLength=0, ResMinLength=0, ResMaxLength=0, ServiceName=IFXServiceBase, ChannelName=null, SPName=null, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=0, sqrSuccessTotalTime=0, Id=IFXServiceBase, TotalCount=1, TotalTime=67, TotalMinTime=67, TotalMaxTime=67, TotalAvgTime=67.0, StandartDeviation=NaN

2019-06-05 09:45:33,617 - AOB:com.fnf.xes.PRF:IntRateInqRq=ServiceName=IntRateInqRq, ChannelName=AOB, SPName=com.fnf.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=81, sqrSuccessTotalTime=0, Id=IntRateInqRq, TotalCount=2, TotalTime=141, TotalMinTime=60, TotalMaxTime=81, TotalAvgTime=70.0, StandartDeviation=NaN

2019-06-05 09:45:33,617 - null:null:IFXServiceBase=EndPointName=IFXServiceBase, Protocol=null, ReqMsgTotalLength=0, ReqMinLength=0, ReqMaxLength=0, ResMsgTotalLength=0, ResMinLength=0, ResMaxLength=0, ServiceName=IFXServiceBase, ChannelName=null, SPName=null, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=86, sqrSuccessTotalTime=0, Id=IFXServiceBase, TotalCount=2, TotalTime=150, TotalMinTime=64, TotalMaxTime=86, TotalAvgTime=75.0, StandartDeviation=NaN

2019-06-05 09:44:23,352 - null:com.fnis.xes.PRF:com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO=ServiceName=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, ChannelName=null, SPName=com.fnis.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=10, sqrSuccessTotalTime=0, Id=com.fnis.ifx.xbo.v1_1.commondata.CommonDataXQO, TotalCount=2, TotalTime=20, TotalMinTime=10, TotalMaxTime=10, TotalAvgTime=10.0, StandartDeviation=NaN

2019-06-05 09:44:23,352 - null:null:SOAP_IFX20=EndPointName=SOAP_IFX20, Protocol=null, ReqMsgTotalLength=0, ReqMinLength=0, ReqMaxLength=0, ResMsgTotalLength=0, ResMinLength=0, ResMaxLength=0, ServiceName=SOAP_IFX20, ChannelName=null, SPName=null, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=12, sqrSuccessTotalTime=0, Id=SOAP_IFX20, TotalCount=2, TotalTime=24, TotalMinTime=12, TotalMaxTime=12, TotalAvgTime=12.0, StandartDeviation=NaN

2019-06-05 09:38:51,991 - AOB:com.fnf.xes.PRF:IntRateInqRq=ServiceName=IntRateInqRq, ChannelName=AOB, SPName=com.fnf.xes.PRF, successCount=0, successTotalTime=0, successMinTime=0, sucssessMaxTime=72, sqrSuccessTotalTime=0, Id=IntRateInqRq, TotalCount=2, TotalTime=142, TotalMinTime=70, TotalMaxTime=72, TotalAvgTime=71.0, StandartDeviation=NaN
0 Karma
1 Solution

oscar84x
Contributor

I deleted my original answer since I misunderstood your question. You DO want to break the events up further at those points.
Use the following as your regex: StandartDeviation=NaN(, )|StandartDeviation=NaN()
This will break up every event after every "StandartDeviation=NaN" and get rid of the comma and the space at the beginning of the new event. Seems like there's no time stamp for that second event though.

View solution in original post

0 Karma

oscar84x
Contributor

I deleted my original answer since I misunderstood your question. You DO want to break the events up further at those points.
Use the following as your regex: StandartDeviation=NaN(, )|StandartDeviation=NaN()
This will break up every event after every "StandartDeviation=NaN" and get rid of the comma and the space at the beginning of the new event. Seems like there's no time stamp for that second event though.

0 Karma

mightaswelby
Explorer

worked perfect, thank you!

oscar84x
Contributor

Excellent! Thank you.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Get Agentic with Splunk Lantern: Connect to Cisco Cloud Control, Transform ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...

July Community Events: Master ITSI 5.0 & Automate Splunk

Struggling with alert fatigue or feeling like you're spending more time on infrastructure maintenance than ...

New Release of Federated Search: Bringing Splunk Analytics to More of Your Data

Organizations today are generating more data than ever and storing it across cloud object stores, data lakes, ...