Log snippet:
0416 12:45:59.50: classify_origination(newcall)
0416 12:45:59.50: colp: 1419371523
0416 12:45:59.50: process_trunk_parm: use_trunk_fields=false
0416 12:45:59.50: ++++ SBCCTL STATS DUMP START (interval = 60) +++
0416 12:45:59.50: STATE: ONLINE
0416 12:45:59.50: uas: rcvd=13271(2.2e+02 mps) sent=16593(2.8e+02 mps)
0416 12:45:59.50: uac: rcvd=15253(2.5e+02 mps) sent=11931(2e+02 mps)
0416 12:45:59.50: mpxy: rcvd=28322(4.7e+02 mps) sent=13869(2.3e+02 mps)
0416 12:45:59.50: other: rcvd=93899(1.6e+03 mps) sent=93899(1.6e+03 mps)
0416 12:45:59.50: spool: rcvd=6209(1e+02 mps) sent=6209(1e+02 mps)
0416 12:45:59.50: MESSAGES: rcvd=156954(2615.9 mps) sent=142501(2375 mps)
0416 12:45:59.50: Offered load: total=56846, uac=15253, uas=13271, mpxy=28322
0416 12:45:59.50: OVERFLOWS: uas=0 uac=0 mpxy=0 sbcspool=0 admctl=0
0416 12:45:59.50: QUEUES: mux:0:0:70, uas:0:0:0, uac:0:0:0, mpxy:0:0:69, sbcspool:0:0:0, admctl:0:0:3
0416 12:45:59.50: MSGS UAC/UAS: 6181:5215:129:82:3320:0:1354:1351:5508:5228
0416 12:45:59.50: MSGS MPXY: 4941:0:4949:14453:0:0
0416 12:45:59.50: MSGS SPOOL: 6209:6209
0416 12:45:59.50: CSPS (last 60 seconds): 103.02
0416 12:45:59.50: Call counts: active:2690 (init:43, trying:865, stable:1781, ending:1, teoc:0)
0416 12:45:59.50: Tunnel count: 2531
0416 12:45:59.50: sbcsipuas.1@b12sb01: CONNECTED ONLINE 2408/9000
0416 12:45:59.50: sbcsipuas.2@b12sb01: CONNECTED ONLINE 282/9000
0416 12:45:59.50: brawt.1@b12sb01: CLOSED UNKNOWN 0/9000
0416 12:45:59.50: sbch323uas.1@b12sb01: CONNECTED ONLINE 0/9000
0416 12:45:59.50: sbcsipuac.1@b12sb01: CONNECTED ONLINE 2544/9000
0416 12:45:59.50: sbcsipuac.2@b12sb01: CONNECTED ONLINE 96/9000
0416 12:45:59.50: sbcsipuac.3@b12sb01: CONNECTED ONLINE 0/9000
0416 12:45:59.50: sbcsipuac.4@b12sb01: CONNECTED ONLINE 0/9000
0416 12:45:59.50: sbcsipuac.5@b12sb01: CONNECTED ONLINE 0/9000
0416 12:45:59.50: sbcsipuac.6@b12sb01: CONNECTED ONLINE 0/9000
0416 12:45:59.50: sbcsipuac.7@b12sb01: CONNECTED ONLINE 0/9000
0416 12:45:59.50: sbcsipuac.8@b12sb01: CONNECTED ONLINE 0/9000
0416 12:45:59.50: sbch323uac.1@b12sb01: CONNECTED ONLINE 0/9000
0416 12:45:59.50: mpxy.1@b12sb01: CONNECTED ONLINE 332/1300
0416 12:45:59.50: mpxy.2@b12sb01: CONNECTED ONLINE 306/1300
0416 12:45:59.50: mpxy.3@b12sb01: CONNECTED ONLINE 289/1300
0416 12:45:59.50: mpxy.4@b12sb01: CONNECTED ONLINE 313/1300
0416 12:45:59.50: mpxy.5@b12sb01: CONNECTED ONLINE 329/1300
0416 12:45:59.50: mpxy.6@b12sb01: CONNECTED ONLINE 324/1300
0416 12:45:59.50: mpxy.7@b12sb01: CONNECTED ONLINE 327/1300
0416 12:45:59.50: mpxy.8@b12sb01: CONNECTED ONLINE 311/1300
0416 12:45:59.50: MPXY Tunnel: timeouts:0, orphans:2, min(ms):0 max(ms):23 count:4939 avg(ms):0.72262 HISTO:4939:0:0:0:0:0:0:0:0:0:0
0416 12:45:59.50: MPXY Packets (Ingress): total:5695213, toss:1988(0.034907%), out_of_seq:104(0.0018261%), lost:941(0.01652%)
0416 12:45:59.50: MPXY Packets (Egress): total:7041461, toss:220(0.0031244%), out_of_seq:74(0.0010509%), lost:3832(0.054391%)
0416 12:45:59.50: Memory Size: 1,205,137,408
0416 12:45:59.50: ++++ SBCCTL STATS DUMP END +++
0416 12:45:59.50: classify_origination(call)
0416 12:45:59.50: colp:
0416 12:45:59.50: memory:
0416 12:45:59.50: process_trunk_parm: use_trunk_fields=true
I am trying to extract all of the text between
0416 12:45:59.50: ++++ SBCCTL STATS DUMP START (interval = 60) +++
and
0416 12:45:59.50: ++++ SBCCTL STATS DUMP END +++
and include those lines and all the lines in between into a single event.
This is what I have tried with no luck
inputs:
[monitor:///opt/splunk/testdata/*.log]
sourcetype=test
index = app
disabled = false
props:
[test]
TIME_PREFIX = ^
TIME_FORMAT = %m-%d %H:%M:%S.%2N
MAX_TIMESTAMP_LOOKAHEAD = 32
TRUNCATE = 99999
SHOULD_LINEMERGE = true
TRANSFORMS-set = setnull, setnonull
Transforms:
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[setnonull]
REGEX = ^.*START.*(?ms).*\+{3}
DEST_KEY = queue
FORMAT = indexQueue
Testing the regex via an online tester selects the lines I am looking to extract. I am not sure of the issue. Any assistance would be greatly appeciated. Tks
Try this:
In props.conf:
[YourSourcetypeHere]
SHOULD_LINEMERGE = false
LINE_BREAKER = (?ms)(.*?)\d+\s+\d+:\d+:\d+\.\d+:\s+\+\+\+\+ SBCCTL STATS DUMP START.*?\d+\s+\d+:\d+:\d+\.\d+:\s+\+\+\+\+ SBCCTL STATS DUMP END \+\+\+
TIME_PREFIX = ^
TIME_FORMAT = %m-%d %H:%M:%S.%2N
MAX_TIMESTAMP_LOOKAHEAD = 32
TRANSFORMS-set = setnull, setnonull
In transforms.conf:
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[setnonull]
REGEX = \+\+\+\+ SBCCTL STATS DUMP START
DEST_KEY = queue
FORMAT = indexQueue
@woodcock Thank you, I was very hopeful in trying this out as I thought it would work, Unfortunately, when implementing these changes, I am not seeing any events via the search head.
Inputs
[monitor:///opt/splunk/testdata/sbc*.log]
sourcetype=test
index = app_sbc
disabled = false
Props
[test]
TIME_PREFIX = ^
TIME_FORMAT = %m-%d %H:%M:%S.%2N
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = false
LINE_BREAKER = (?ms)(.*?)\d+\s+\d+:\d+:\d+\.\d+:\s+\+\+\+\+ SBCCTL STATS DUMP START.*?\d+\s+\d+:\d+:\d+\.\d+:\s+\+\+\+\+ SBCCTL STATS DUMP END \+\+\+
TRANSFORMS-set = setnull, setnonull
Transforms
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[setnonull]
REGEX = \+\+\+\+ SBCCTL STATS DUMP START
DEST_KEY = queue
FORMAT = indexQueue
It definitely works; I tested it. Did you:
1) Deploy to the first full instance of Splunk that touches the data (HF or Indexer tier)?
2) Restart all splunk instances there?
3) Forward in new data (previous data will stay wrong forever)?
4) Test using `_index_earliest=-5m` to be absolutely sure that it is new data?
What do you want to do with lines before START and after END??
Discard them. I thought set all to null and then extract what I need.