How to break Multiline events into single event

[keshavgupta]

We are ingesting network events into a log file. And it looks like below 

Network_Event=ThresholdViolation
Network_EventDesc=A Threshold Violation event has been cleared. (Profile Name: INTERFACE -VRT Interfaces, Rule Name: Network::: Bandwidth Utilization Out >85%
30 min out of 1 hr, Reason for clearing: Clear Threshold criteria has been satisfied for Event Rule.)
Network_ItemName=vpn-0-ipv4-if-ge0/3
Network_EvProp_AlarmClearRuleDetail=UtilizationOut < 75.0
Network_ItemParentId=123456
Network_EvProp_ThresholdProfileFolderId=12334
Network_EventOccurredOn=Thu Apr 10 22:55:00 PDT 2021
Netwok_ItemDesc=Viptela-Interface-AL1-123422
Network_EventState=CLOSED
Network_ItemName=vpn-0-ipv3-313
Network_EvProp_AlarmClearRuleDetail=UtilizationOut < 75.0
Network_EventSubType=Cleared

So when we monitor the above log and move to indexer and on splunk it will split above all lines into 2 event. Because of the timestamp Network_EventOccurredOn=Thu Apr 10 22:55:00 PDT 2021 coming inthe middle.

1st event: 

Network_Event=ThresholdViolation
Network_EventDesc=A Threshold Violation event has been cleared. (Profile Name: INTERFACE -VRT Interfaces, Rule Name: Network::: Bandwidth Utilization Out >85%
30 min out of 1 hr, Reason for clearing: Clear Threshold criteria has been satisfied for Event Rule.)
Network_ItemName=vpn-0-ipv4-if-ge0/3
Network_EvProp_AlarmClearRuleDetail=UtilizationOut < 75.0
Network_ItemParentId=123456
Network_EvProp_ThresholdProfileFolderId=12334

2nd event:

Network_EventOccurredOn=Thu Apr 10 22:55:00 PDT 2021
Netwok_ItemDesc=Viptela-Interface-AL1-123422
Network_EventState=CLOSED
Network_ItemName=vpn-0-ipv3-313
Network_EvProp_AlarmClearRuleDetail=UtilizationOut < 75.0
Network_EventSubType=Cleared

Tried adding below props.conf but still it didnt worked.

[SOURCETYPE]

SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)Network_EventType=ThresholdViolation

Please help me on this. I want above 2 event as single event.

[tscroggins]

@keshavgupta 

The following stanza should break the event on Network_Event= and extract the timestamp from Network_EventOccuredOn=:

[my_source_type]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(Network_Event=)
TIME_PREFIX = ^Network_EventOccurredOn=
TIME_FORMAT = %+
MAX_TIMESTAMP_LOOKAHEAD = 28

[keshavgupta]

Still same issue. when i use above stanze. I see 2 events. I want that as single event.

[tscroggins]

@keshavgupta 

This will provide less ambiguous timestamp extraction:

[my_source_type]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(Network_Event=)
TIME_PREFIX = [\r\n]+Network_EventOccurredOn=
TIME_FORMAT = %a %b %d %H:%M:%S %Z %Y
MAX_TIMESTAMP_LOOKAHEAD = 28

but line breaking works as expected:

Does the content in your environment vary slightly from the example you provided?

If your events start with Network_EventType=, for example, try:

LINE_BREAKER = ([\r\n]+)(Network_EventType=)