Hi Team,
We have ingested the NSG flow logs from azure and few events are not breaking the events properly, could you please help me to write the LINE_BREAKER rule for below events.
Sample events:
"time":"2018-05-30T16:07:43.6682050Z","systemId":"","category":"NetworkSecurityGroupFlowEvent","12000,T,O,A"]}]}]}{"time":"2018-05-30T16:06:43.6499999Z","systemId":"","category":"NetworkSecurityGroupFlowEvent","resourceId":"/,T,O,A"]}]}]}{"time":"2018-05-30T16:06:43.6499999Z","systemId":"","category":"NetworkSecurityGroupFlowEvent","resourceId":"/,T,O,A"]}]}]}
LINE_BREAKER=?
@lksridhar, can you try with this?
LINE_BREAKER=(]}]}]})
TIME_PREFIX="time":
MAX_TIMESTAMP_LOOKAHEAD=29
hey guys, sorry for making my comment an answer. For some reason, it won't allow me to make comments, only answers.... ???
When I click on "comment" it drops down to the main answers section.
@niketnilay, no your 29 is correct, I wasn't looking at timezone. Was more focused on the line break actually.
Thanks nikentilay and dpetracca for your input. i tried nikentilay command it is working for me.
LINE_BREAKER=(]}]}]})
TIME_PREFIX="time":
MAX_TIMESTAMP_LOOKAHEAD=29
Thanks for helping to fix the issue.
I came to same conclusion as @niketn
I tested it out and here is the props.conf that works:
[test_line_break]
DATETIME_CONFIG =
LINE_BREAKER = (]}]}]})
MAX_TIMESTAMP_LOOKAHEAD = 27
NO_BINARY_CHECK = true
TIME_PREFIX = time
category = Custom
disabled = false
pulldown_type = true
@dpetracca I had MAX_TIMESTAMP_LOOKAHEAD
as 29 to extract even the timezone. Are you planning to apply TIMESTAMP_FORMAT %6N
and stop Timestamp extraction?
Are you missing a curly bracket in the line 1??
@lksridhar, can you try with this?
LINE_BREAKER=(]}]}]})
TIME_PREFIX="time":
MAX_TIMESTAMP_LOOKAHEAD=29
@lksridhar, great to hear that your issue is resolved. I have converted my comment to answer. Please accept to mark this question as answered!