I have a script that works fine.
When I do run it from cli like this, I get correct result:
/opt/splunk/bin/splunk cmd /opt/splunk/etc/apps/MikroTik/bin/mikrotik_upnp.sh
Flags: X - disabled, I - invalid, D - dynamic
0 D ;;; upnp 10.10.10.32: Teredo
chain=dstnat action=dst-nat to-addresses=10.10.10.32 to-ports=57050
protocol=udp dst-address=110.12.197.134 in-interface=ether1
dst-port=57050
1 D ;;; upnp 10.10.10.84: Skype UDP at 10.10.10.84:48153 (3904)
chain=dstnat action=dst-nat to-addresses=10.10.10.84 to-ports=48153
protocol=udp dst-address=110.12.197.134 in-interface=ether1
dst-port=48153
2 D ;;; upnp 10.10.10.84: Skype TCP at 10.10.10.84:48153 (3904)
chain=dstnat action=dst-nat to-addresses=10.10.10.84 to-ports=48153
protocol=tcp dst-address=110.12.197.134 in-interface=ether1
dst-port=48153
3 D ;;; upnp 10.10.10.128: Skype UDP at 10.10.10.128:43905 (3909)
chain=dstnat action=dst-nat to-addresses=10.10.10.128 to-ports=43905
protocol=udp dst-address=110.12.197.134 in-interface=ether1
dst-port=43905
4 D ;;; upnp 10.10.10.128: Skype TCP at 10.10.10.128:43905 (3909)
chain=dstnat action=dst-nat to-addresses=10.10.10.128 to-ports=43905
protocol=tcp dst-address=110.12.197.134 in-interface=ether1
dst-port=43905
5 D ;;; upnp 10.10.10.129: Skype UDP at 10.10.10.129:20139 (3910)
chain=dstnat action=dst-nat to-addresses=10.10.10.129 to-ports=20139
protocol=udp dst-address=110.12.197.134 in-interface=ether1
dst-port=20139
6 D ;;; upnp 10.10.10.129: Skype TCP at 10.10.10.129:20139 (3910)
chain=dstnat action=dst-nat to-addresses=10.10.10.129 to-ports=20139
protocol=tcp dst-address=110.12.197.134 in-interface=ether1
dst-port=20139
7 D ;;; upnp 10.10.10.125: 3074 UDP
chain=dstnat action=dst-nat to-addresses=10.10.10.125 to-ports=3074
protocol=udp dst-address=110.12.197.134 in-interface=ether1
dst-port=3074
8 D ;;; upnp 10.10.10.152: WhatsApp (1505943818) ()
chain=dstnat action=dst-nat to-addresses=10.10.10.152 to-ports=56265
protocol=udp dst-address=110.12.197.134 in-interface=ether1
dst-port=56265
9 D ;;; upnp 10.10.10.152: WhatsApp (1505944513) ()
chain=dstnat action=dst-nat to-addresses=10.10.10.152 to-ports=61271
protocol=udp dst-address=110.12.197.134 in-interface=ether1
dst-port=61271
10 D ;;; upnp 10.10.10.152: WhatsApp (1505945615) ()
chain=dstnat action=dst-nat to-addresses=10.10.10.152 to-ports=62934
protocol=udp dst-address=110.12.197.134 in-interface=ether1
dst-port=62934
11 D ;;; upnp 10.10.10.32: uTorrent (TCP)
chain=dstnat action=dst-nat to-addresses=10.10.10.32 to-ports=28816
protocol=tcp dst-address=110.12.197.134 in-interface=ether1
dst-port=28816
12 D ;;; upnp 10.10.10.32: uTorrent (UDP)
chain=dstnat action=dst-nat to-addresses=10.10.10.32 to-ports=28816
protocol=udp dst-address=110.12.197.134 in-interface=ether1
dst-port=28816
But in Splunk, I only get 9 events??? It stops at event 7, so 8,9,10,11,12 is missing and result is like this:
25/09/2017
11:21:52.000
7 D ;;; upnp 10.10.10.125: 3074 UDP
chain=dstnat action=dst-nat to-addresses=10.10.10.125 to-ports=3074
protocol=udp dst-address=110.12.197.134 in-interface=ether1
dst-port=3074
host = Varg source = /opt/splunk/etc/apps/MikroTik/bin/mikrotik_upnp.sh sourcetype = mikrotik2
25/09/2017
11:21:52.000
6 D ;;; upnp 10.10.10.129: Skype TCP at 10.10.10.129:20139 (3910)
chain=dstnat action=dst-nat to-addresses=10.10.10.129 to-ports=20139
protocol=tcp dst-address=110.12.197.134 in-interface=ether1
dst-port=20139
host = Varg source = /opt/splunk/etc/apps/MikroTik/bin/mikrotik_upnp.sh sourcetype = mikrotik2
25/09/2017
11:21:52.000
5 D ;;; upnp 10.10.10.129: Skype UDP at 10.10.10.129:20139 (3910)
chain=dstnat action=dst-nat to-addresses=10.10.10.129 to-ports=20139
protocol=udp dst-address=110.12.197.134 in-interface=ether1
dst-port=20139
host = Varg source = /opt/splunk/etc/apps/MikroTik/bin/mikrotik_upnp.sh sourcetype = mikrotik2
25/09/2017
11:21:52.000
4 D ;;; upnp 10.10.10.128: Skype TCP at 10.10.10.128:43905 (3909)
chain=dstnat action=dst-nat to-addresses=10.10.10.128 to-ports=43905
protocol=tcp dst-address=110.12.197.134 in-interface=ether1
dst-port=43905
host = Varg source = /opt/splunk/etc/apps/MikroTik/bin/mikrotik_upnp.sh sourcetype = mikrotik2
25/09/2017
11:21:52.000
3 D ;;; upnp 10.10.10.128: Skype UDP at 10.10.10.128:43905 (3909)
chain=dstnat action=dst-nat to-addresses=10.10.10.128 to-ports=43905
protocol=udp dst-address=110.12.197.134 in-interface=ether1
dst-port=43905
host = Varg source = /opt/splunk/etc/apps/MikroTik/bin/mikrotik_upnp.sh sourcetype = mikrotik2
25/09/2017
11:21:52.000
2 D ;;; upnp 10.10.10.84: Skype TCP at 10.10.10.84:48153 (3904)
chain=dstnat action=dst-nat to-addresses=10.10.10.84 to-ports=48153
protocol=tcp dst-address=110.12.197.134 in-interface=ether1
dst-port=48153
host = Varg source = /opt/splunk/etc/apps/MikroTik/bin/mikrotik_upnp.sh sourcetype = mikrotik2
25/09/2017
11:21:52.000
1 D ;;; upnp 10.10.10.84: Skype UDP at 10.10.10.84:48153 (3904)
chain=dstnat action=dst-nat to-addresses=10.10.10.84 to-ports=48153
protocol=udp dst-address=110.12.197.134 in-interface=ether1
dst-port=48153
host = Varg source = /opt/splunk/etc/apps/MikroTik/bin/mikrotik_upnp.sh sourcetype = mikrotik2
25/09/2017
11:21:52.000
0 D ;;; upnp 10.10.10.32: Teredo
chain=dstnat action=dst-nat to-addresses=10.10.10.32 to-ports=57050
protocol=udp dst-address=110.12.197.134 in-interface=ether1
dst-port=57050
host = Varg source = /opt/splunk/etc/apps/MikroTik/bin/mikrotik_upnp.sh sourcetype = mikrotik2
25/09/2017
11:21:52.000
Flags: X - disabled, I - invalid, D - dynamic
host = Varg source = /opt/splunk/etc/apps/MikroTik/bin/mikrotik_upnp.sh sourcetype = mikrotik2
I did try to split events by a blank line but did not get it to work),
inputs.conf
[script://$SPLUNK_HOME/etc/apps/MikroTik/bin/mikrotik_upnp.sh]
disabled = false
interval = 300
sourcetype = mikrotik2
props.conf
[mikrotik2]
DATETIME_CONFIG =
NO_BINARY_CHECK = true
category = Custom
pulldown_type = true
LINE_BREAKER =
BREAK_ONLY_BEFORE = \d+\s+D\s
disabled = false
Why does 5 events get lost?
Is it due to my BREAK_ONLY_BEFORE?
Are there a better way to du it (use LINE_BREAKER instead)?
Update.
For some reason I now get all events with one digit ID.
So I get 0 to 9, but not 10,11 or 12.
BREAK_ONLY_BEFORE do contain \d+, so it should take any number.
try this,
[ mikrotik2]
DATETIME_CONFIG=CURRENT
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=\d+\s+\S\s+
CHARSET=UTF-8