Mixed single-line and multi-line events in heavy forwarder problem

floko — Tue, 29 Sep 2020 16:40:48 GMT

I have a heavy forwarder (Splunk Enterprise 7.0) that needs to parse a very nasty log file. I am interested in only a few of the events, so the forwarder needs to discard most of them. Most of the content are one-line messages about Comfort Noise.
At some time, a large multi-line event may appear that signals the creation of an RTSP stream. This is what I am interested in.
This is how it looks like in the log file:

09.11.2017 11:49:32,350; INFO; T0329; [sometext]; : [01 RTPReceiverTETRA::Receive(CN)]
09.11.2017 11:49:32,350; INFO; T0329; [sometext]; : Receive Comfort Noise -> Provide Silence (1600 Samples, 43246 Sequence Nr)
09.11.2017 11:49:32,350; INFO; T0042; [sometext]; : [01 RTPReceiverTETRA::Receive(CN)]
09.11.2017 11:49:32,350; INFO; T0042; [sometext]; : Receive Comfort Noise -> Provide Silence (1600 Samples, 42925 Sequence Nr)
09.11.2017 11:49:32,350; INFO; T0431; [sometext]; : [01 RTPReceiverTETRA::Receive(CN)]
09.11.2017 11:49:32,350; INFO; T0431; [sometext]; : Receive Comfort Noise -> Provide Silence (1600 Samples, 43399 Sequence Nr)
18.10.2017 08:08:56,032; INFO; T0030; [sometext]; : RTSPRcv <192.168.25.181|37093>
18.10.2017 08:08:56,032; INFO; T0030; [sometext]; :    ANNOUNCE rtsp://192.168.25.138:554/iprecorder/ RTSP/1.0
18.10.2017 08:08:56,032; INFO; T0030; [sometext]; :    CSeq: 3912
18.10.2017 08:08:56,032; INFO; T0030; [sometext]; :    User-Agent: RecService
18.10.2017 08:08:56,032; INFO; T0030; [sometext]; :    Timestamp: 260983967488 0
18.10.2017 08:08:56,032; INFO; T0030; [sometext]; :    Content-Type: application/sdp
18.10.2017 08:08:56,032; INFO; T0030; [sometext]; :    Content-Length: 291
18.10.2017 08:08:56,032; INFO; T0030; [sometext]; :    
18.10.2017 08:08:56,032; INFO; T0030; [sometext]; :    v=0
18.10.2017 08:08:56,032; INFO; T0030; [sometext]; :    o=unnamed 0 0 IN IP4 192.168.25.138
18.10.2017 08:08:56,032; INFO; T0030; [sometext]; :    s=conversation
18.10.2017 08:08:56,032; INFO; T0030; [sometext]; :    c=IN IP4 0.0.0.0
18.10.2017 08:08:56,032; INFO; T0030; [sometext]; :    t=0 0
18.10.2017 08:08:56,032; INFO; T0030; [sometext]; :    u=sip:9109000@192.168.25.180:5060
18.10.2017 08:08:56,032; INFO; T0030; [sometext]; :    m=audio 0 RTP/AVP 99 123
18.10.2017 08:08:56,033; INFO; T0030; [sometext]; :    a=rtpmap:99 x-ptt-TETRA/8000
18.10.2017 08:08:56,033; INFO; T0030; [sometext]; :    a=rtpmap:123 x-ptt-R2S/8000
18.10.2017 08:08:56,033; INFO; T0030; [sometext]; :    a=sendonly
18.10.2017 08:08:56,033; INFO; T0030; [sometext]; :    a=type:Radio-Rxonly
18.10.2017 08:08:56,033; INFO; T0030; [sometext]; :    a=R2S-KeepAlivePeriod:200
18.10.2017 08:08:56,033; INFO; T0030; [sometext]; :    a=R2S-KeepAliveMultiplier:10
18.10.2017 08:08:56,033; INFO; T0030; [sometext]; :    
18.10.2017 08:08:56,033; INFO; T0030; [sometext]; : [01 RTSPServer::OnRTSPRequestAnnounce()]
18.10.2017 08:08:56,033; INFO; T0030; [sometext]; : [02 RTSPController::TraceRTSPMessage()]
18.10.2017 08:08:56,033; INFO; T0030; [sometext]; : RTSPTxm <192.168.25.181|37093>
18.10.2017 08:08:56,033; INFO; T0030; [sometext]; :    RTSP/1.0 200 OK
18.10.2017 08:08:56,033; INFO; T0030; [sometext]; :    CSeq: 3912
18.10.2017 08:08:56,033; INFO; T0030; [sometext]; :    Timestamp: 260983967488 0.001
18.10.2017 08:08:56,033; INFO; T0030; [sometext]; :    Server: RECORDER.RTSP/3.30.012 libTPP/0.07.000
18.10.2017 08:08:56,033; INFO; T0030; [sometext]; :    Date: Wed, 18 Oct 2017 08:08:56 GMT

On the Forwarder, I have transforms.conf:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[set-rtsp-events]
REGEX = RTSPRcv|RTSPTxm|RTPChannel|rtsp:\/\/
DEST_KEY = queue
FORMAT = indexQueue

And the props.conf:

[rtsp]
TZ=UTC
SHOULD_LINEMERGRE=true
BREAK_ONLY_BEFORE_DATE=false
LINE_BREAKER=(; : {4}([\r\n]+)|(([\r\n])+\d+.\d+.\d+ \d+:\d+:\d+,\d+)
MUST_NOT_BREAK_AFTER=\]; : (?<RxTxDirection>RTSPRcv|RTSPTxm) <(?<SourceIP>\d+\.\d+\.\d+\.\d+)\|(?<SourcePort>\d+)>
MUST_BREAK_AFTER=\]; : {4}([\r\n]+)
TRANSFORMS-parsing=setnull,set-rtsp-events

Here I need some help.
The setnull parsing should take care to discard everything. Only the things discovered by the regexes will have the indexQueue.
As long as I don't use the BREAK-rules, it works fine, but I don't get the multi line events (only the first line of each).
When I try to fetch the multi-line event, it captures far too much, mostly several thousands of lines, because the end-criteria don't match.
Unfortunately, every line of the multi-line event begins with a timestamp, so I can't use the BREAK_ONLY_BEFORE_DATE.

Re: Mixed single-line and multi-line events in heavy forwarder problem

tmarlette — Thu, 09 Nov 2017 17:33:53 GMT

Try this setting in props.conf with the appropriate REGEX:

BREAK_ONLY_BEFORE = * When set, Splunk creates a new event only if it encounters a new line that matches the regular expression. * Defaults to empty.

check out this link:
https://docs.splunk.com/Documentation/Splunk/7.0.0/Admin/Propsconf

Re: Mixed single-line and multi-line events in heavy forwarder problem

floko — Tue, 29 Sep 2020 16:45:25 GMT

Thank you!
I thought I had tried everything, but obviously I overlooked this one. After all, it looks very similar to BREAK_ONLY_BEFORE_DATE 😉

topic Re: Mixed single-line and multi-line events in heavy forwarder problem in Getting Data In

Mixed single-line and multi-line events in heavy forwarder problem

Re: Mixed single-line and multi-line events in heavy forwarder problem

Re: Mixed single-line and multi-line events in heavy forwarder problem