Event timestamp is getting truncated.

[SagarSplunk]

Hi All,

Need your help inputs on below issue.

We have applied Line breaking configuration its working fine on PREPROD but on PROD time stamp in the events are getting truncated intermittently.

Log File Entry (truncated text in bold):-
2018/02/19 04:47:55.09,GPP.TODP.ACK.PAYMNT.INSTRCTN.OUT,414d51205052444750503031202020205a87e19520f85d05,414d51204c41554b4e53494c202020205a7eab8f24c04bf6,pain.002.001.06,,,I02JF4754HOD0D1N,569d7e66fba74b5faebf2bd3eda595edK0,,W8281188204,

Event In Splunk after indexing:-

47:55.09,GPP.TODP.ACK.PAYMNT.INSTRCTN.OUT,414d51205052444750503031202020205a87e19520f85d05,414d51204c41554b4e53494c202020205a7eab8f24c04bf6,pain.002.001.06,,,I02JF4754HOD0D1N,569d7e66fba74b5faebf2bd3eda595edK0,,W8281188204,

[richgalloway]

Are the date and time on separate lines?
You say you have applied line breaking, but I see nothing in your props related to line breaking.
I also see nothing in the props relating to parsing timestamps. At a minimum, you should have TIME_PREFIX, TIME_FORMAT, and MAX_TIMESTAMP_LOOKAHEAD.

[SagarSplunk]

Hi.there is is no prefix event starts with the time stamps only e.g. 2018/02/19 04:47:55.09
The issue is not line breaking, The data itself is not present in events.

If Line breaking is the issue, truncated event data will get added to earlier event but that is not case here , data is not at all indexing.

[richgalloway]

If the timestamp is at the beginning of the line then set TIME_PREFIX = ^. You should also set TIME_FORMAT to something, probably %Y/%m/%d %H:%M:%S.%2N.
Have you verified the inputs.conf file in PROD references the correct sourcetype?

[niketn]

@SagarSplunk, have you checked the props.conf for both Pre-Prod and Prod are the same or not (provided data is the same in both the environments)?

[SagarSplunk]

@niketnilay Yes both environments are having same props.conf configuration.

[493669]

can you provide your props.conf

[SagarSplunk]

EXTRACT-isoClearSysRef = ^(?:[^,\n],){6}(?P\w+)
EXTRACT-mqCoreId = ^(?:[^,\n],){3}(?P[^,]+)
EXTRACT-timeStamp,queueName,MQMsgId,isoMsgDefId,TX_UID = ^(?P[^,]+),(?P[^,]+),(?P[^,]+)(?:[^,\n],){2}(?P[^,]+)[^,\n],(?P[^,]+)
EXTRACT-timeStamp,queueName,isoMsgDefId,isoMsgId = ^(?P[^,]+),(?P[^,]+)(?:[^,\n],){3}(?P[^,]+),,,(?P[^,]+)
EXTRACT-timeStamp,queueName,mqCoreId,isoMsgDefId,isoMsgId,isoOriginalMsgId,isoOriginalInstructionId = ^(?P[^,]+),(?P[^,]+),[a-f0-9]+,(?P[a-f0-9]+),(?P[^,]+),,,(?P[^,]+),(?P\w+)(?:[^,\n],){2}(?P[^,]+)
EXTRACT-timeStamp,queueName,mqMsgId,isoMsgDefId,isoClearSysRef,isoInstructionId = ^(?P[^,]+),(?P[^,]+),(?P[^,]+)(?:[^,\n],){2}(?P[^,]+),,(?P\w+)(?:[^,\n],){3}(?P[^,]+)
EXTRACT-timeStamp,queueName,mqMsgId,isoMsgDefId,isoInstructionId = ^(?P[^,]+),(?P[^,]+),(?P[^,]+)(?:[^,\n],){2}(?P[^,]+)(?:[^,\n],){6}(?P\w+)
EXTRACT-timeStamp,queueName,mqMsgId,isoMsgDefId,isoMsgId,isoInstructionId = ^(?P[^,]+),(?P[^,]+)[^,\n],(?P[^,]+)(?:[^,\n],){2}(?P[^,]+),,,(?P[^,]+),,(?P[^,]+)
EXTRACT-timeStamp,queueName,mqMsgId,mqCoreId,isoMsgDefId,isoClearSysRef,isoMsgId,isoOriginalInstructionId = ^(?P[^,]+),(?P[^,]+),(?P[^,]+),(?P[^,]+),(?P[^,]+),,,(?P[^,]+),(?P[^,]+)(?:[^,\n],){2}(?P\w+)
EXTRACT-timeStamp,queueName,mqMsgId,mqCoreId,isoMsgDefId,isoMsgId,isoInstructionId = ^(?P[^,]+),(?P[^,]+),(?P[^,]+),(?P[^,]+),(?P\w+.\d+.\d+.\d+)(?:[^,\n],){4}(?P\w+),,(?P[^,]+)EXTRACT-timeStamp,queueName,MsgID = ^(?P\d+/\d+/\d+\s+\d+:\d+:\d+.\d+),(?P[^,]+)[^,\n]*,(?P[^,]+)

[skoelpin]

Is this your search head props.conf? You should paste the indexer props

If this is your indexer props, then you need to set your base configs which tells Splunk how to identify the timestamp and linebreak

[FrankVl]

That only shows your EXTRACTS. You mention you also defined linebreaking config? What does that look like?