Splunk Search

How to break Multiline events into single event

keshavgupta
Engager

We are ingesting network events into a log file. And it looks like below 

Network_Event=ThresholdViolation
Network_EventDesc=A Threshold Violation event has been cleared. (Profile Name: INTERFACE -VRT Interfaces, Rule Name: Network::: Bandwidth Utilization Out >85%
30 min out of 1 hr, Reason for clearing: Clear Threshold criteria has been satisfied for Event Rule.)
Network_ItemName=vpn-0-ipv4-if-ge0/3
Network_EvProp_AlarmClearRuleDetail=UtilizationOut < 75.0
Network_ItemParentId=123456
Network_EvProp_ThresholdProfileFolderId=12334
Network_EventOccurredOn=Thu Apr 10 22:55:00 PDT 2021
Netwok_ItemDesc=Viptela-Interface-AL1-123422
Network_EventState=CLOSED
Network_ItemName=vpn-0-ipv3-313
Network_EvProp_AlarmClearRuleDetail=UtilizationOut < 75.0
Network_EventSubType=Cleared

So when we monitor the above log and move to indexer and on splunk it will split above all lines into 2 event. Because of the timestamp Network_EventOccurredOn=Thu Apr 10 22:55:00 PDT 2021 coming inthe middle.

1st event: 

Network_Event=ThresholdViolation
Network_EventDesc=A Threshold Violation event has been cleared. (Profile Name: INTERFACE -VRT Interfaces, Rule Name: Network::: Bandwidth Utilization Out >85%
30 min out of 1 hr, Reason for clearing: Clear Threshold criteria has been satisfied for Event Rule.)
Network_ItemName=vpn-0-ipv4-if-ge0/3
Network_EvProp_AlarmClearRuleDetail=UtilizationOut < 75.0
Network_ItemParentId=123456
Network_EvProp_ThresholdProfileFolderId=12334

2nd event:

Network_EventOccurredOn=Thu Apr 10 22:55:00 PDT 2021
Netwok_ItemDesc=Viptela-Interface-AL1-123422
Network_EventState=CLOSED
Network_ItemName=vpn-0-ipv3-313
Network_EvProp_AlarmClearRuleDetail=UtilizationOut < 75.0
Network_EventSubType=Cleared

Tried adding below props.conf but still it didnt worked.

[SOURCETYPE]

SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)Network_EventType=ThresholdViolation

 

Please help me on this. I want above 2 event as single event.

0 Karma

tscroggins
Influencer

@keshavgupta 

The following stanza should break the event on Network_Event= and extract the timestamp from Network_EventOccuredOn=:

[my_source_type]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(Network_Event=)
TIME_PREFIX = ^Network_EventOccurredOn=
TIME_FORMAT = %+
MAX_TIMESTAMP_LOOKAHEAD = 28

0 Karma

keshavgupta
Engager

Still same issue. when i use above stanze. I see 2 events. I want that as single event.

0 Karma

tscroggins
Influencer

@keshavgupta 

This will provide less ambiguous timestamp extraction:

[my_source_type]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(Network_Event=)
TIME_PREFIX = [\r\n]+Network_EventOccurredOn=
TIME_FORMAT = %a %b %d %H:%M:%S %Z %Y
MAX_TIMESTAMP_LOOKAHEAD = 28

but line breaking works as expected:

tscroggins_0-1619877795965.png

Does the content in your environment vary slightly from the example you provided?

If your events start with Network_EventType=, for example, try:

LINE_BREAKER = ([\r\n]+)(Network_EventType=)

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...