All Apps and Add-ons

Splunk Stream not showing netflow data

neo_traffic
New Member

I installed Splunk Stream per the instructions and I see data coming in when I run a search sourcetype=stream:netflow.

In the Stream App, I only see the local data, nothing from my netflow devices.

I am running it as a standalone server.

My configs are as follows:

/opt/splunk/etc/apps/Splunk_TA_stream/local/streamfwd.conf

[streamfwd]
logConfig = streamfwdlog.conf
port = 8889

netflowReceiver.0.ip = XXX.XXX.XXX.XXX (real IP hidden)
netflowReceiver.0.port = 9995
netflowReceiver.0.decoder = netflow
netflowReceiver.0.protocol = udp
netflowReceiver.0.decodingThreads = 4

/opt/splunk/etc/apps/Splunk_TA_stream/local/inputs.conf

[streamfwd://streamfwd]
splunk_stream_app_location = http://localhost:8000/en-us/custom/splunk_app_stream/
stream_forwarder_id =
disabled = 0

/opt/splunk/etc/apps/splunk_httpinput/local/inputs.conf

[http]
disabled = 0
port = 8088
dedicatedIoThreads = 8

[http://streamfwd]
disabled = 0
index=main
token = dcb7872a-9438-4e2e-a314-a20d2991df7b
indexes=_internal,main

netstat -l shows me:

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN
tcp 0 0 localhost:8065 0.0.0.0:* LISTEN
tcp 0 0 localhost:domain 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:ssh 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:omniorb 0.0.0.0:* LISTEN
tcp 0 0 localhost:8889 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:8089 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:8191 0.0.0.0:* LISTEN
tcp6 0 0 [::]:ssh [::]:* LISTEN
udp 0 0 neo-monitor:9995 0.0.0.0:*
udp 0 0 localhost:domain 0.0.0.0:*
raw6 0 0 [::]:ipv6-icmp [::]:* 7
Active UNIX domain sockets (only servers)
Proto RefCnt Flags Type State I-Node Path
unix 2 [ ACC ] SEQPACKET LISTENING 12213 /run/udev/control
unix 2 [ ACC ] STREAM LISTENING 307858 /run/user/0/systemd/private
unix 2 [ ACC ] STREAM LISTENING 307864 /run/user/0/gnupg/S.gpg-agent
unix 2 [ ACC ] STREAM LISTENING 307865 /run/user/0/gnupg/S.gpg-agent.ssh
unix 2 [ ACC ] STREAM LISTENING 307866 /run/user/0/gnupg/S.dirmngr
unix 2 [ ACC ] STREAM LISTENING 307867 /run/user/0/gnupg/S.gpg-agent.browser
unix 2 [ ACC ] STREAM LISTENING 307868 /run/user/0/gnupg/S.gpg-agent.extra
unix 2 [ ACC ] STREAM LISTENING 11797 /run/systemd/private
unix 2 [ ACC ] STREAM LISTENING 11804 /run/systemd/journal/stdout
unix 2 [ ACC ] STREAM LISTENING 11909 /run/lvm/lvmpolld.socket
unix 2 [ ACC ] STREAM LISTENING 11950 /run/lvm/lvmetad.socket
unix 2 [ ACC ] STREAM LISTENING 16749 /run/snapd.socket
unix 2 [ ACC ] STREAM LISTENING 16737 /var/snap/lxd/common/lxd/unix.socket
unix 2 [ ACC ] STREAM LISTENING 16770 @ISCSIADM_ABSTRACT_NAMESPACE
unix 2 [ ACC ] STREAM LISTENING 16742 /var/run/dbus/system_bus_socket
unix 2 [ ACC ] STREAM LISTENING 16751 /run/snapd-snap.socket
unix 2 [ ACC ] STREAM LISTENING 16766 /run/uuidd/request
unix 2 [ ACC ] STREAM LISTENING 16768 /run/acpid.socket

streamfwd.log shows me:

2019-04-04 15:01:47 INFO 140379561435840 stream.CaptureServer - Found DataDirectory: /opt/splunk/etc/apps/Splunk_TA_stream/data
2019-04-04 15:01:47 INFO 140379561435840 stream.CaptureServer - Found UIDirectory: /opt/splunk/etc/apps/Splunk_TA_stream/ui
2019-04-04 15:01:48 INFO 140379561435840 stream.CaptureServer - Default configuration directory: /opt/splunk/etc/apps/Splunk_TA_stream/default
2019-04-04 15:01:48 ERROR 140379561435840 stream.CaptureServer - Unable to ping server (c7a112ef-d447-423b-8231-e5a9206b86aa): Unable to establish connection to localhost: Connection refused
2019-04-04 15:01:48 INFO 140379561435840 stream.main - streamfwd has started successfully (version 7.1.2 build 157)
2019-04-04 15:01:48 INFO 140379561435840 stream.main - web interface listening on port 8889
2019-04-04 15:01:54 ERROR 140379406546688 stream.CaptureServer - Unable to ping server (c7a112ef-d447-423b-8231-e5a9206b86aa): Unable to establish connection to localhost: Connection refused
2019-04-04 15:01:59 ERROR 140379406546688 stream.CaptureServer - Unable to ping server (c7a112ef-d447-423b-8231-e5a9206b86aa): Unable to establish connection to localhost: Connection refused
2019-04-04 15:02:04 ERROR 140379423332096 stream.CaptureServer - Unable to ping server (c7a112ef-d447-423b-8231-e5a9206b86aa): Unable to establish connection to localhost: Connection refused
2019-04-04 15:02:09 ERROR 140379406546688 stream.CaptureServer - Unable to ping server (c7a112ef-d447-423b-8231-e5a9206b86aa): Unable to establish connection to localhost: Connection refused
2019-04-04 15:02:14 ERROR 140379406546688 stream.CaptureServer - Unable to ping server (c7a112ef-d447-423b-8231-e5a9206b86aa): Unable to establish connection to localhost: Connection refused
2019-04-04 15:02:19 ERROR 140379406546688 stream.CaptureServer - Unable to ping server (c7a112ef-d447-423b-8231-e5a9206b86aa): Unable to establish connection to localhost: Connection refused
2019-04-04 15:02:26 INFO 140379406546688 stream.CaptureServer - Netflow receiver configuration defined; disabling default automatic promiscuous mode packet capture on all available interfaces. Configure one or more streamfwdcapture parameters in streamfwd.conf to enable network packet capture.
2019-04-04 15:02:26 INFO 140379406546688 stream.SnifferReactor - No packet processors configured
2019-04-04 15:02:26 INFO 140379406546688 stream.CaptureServer - Starting data capture
2019-04-04 15:02:26 INFO 140379406546688 stream.SnifferReactor - Starting network capture: sniffer

I am running Ubuntu 18.04, Splunk 7.2.5.1, Splunk Stream 7.1.2

Any help would be appreciated.

Tags (2)
0 Karma

michaeljorgense
Path Finder

This looks like close something I am experiencing. My understanding is the streamfwd binary needs to phone home to the Splunk App for Stream as described here:

https://docs.splunk.com/Documentation/StreamApp/7.1.2/DeployStreamApp/DeploymentArchitecture#How_str...

This is where you configure streamfwd to talk to the Stream App:

https://docs.splunk.com/Documentation/StreamApp/7.1.2/DeployStreamApp/ConfigureStreamForwarder#Verif...

In your config this is set to:

splunk_stream_app_location = http://localhost:8000/en-us/custom/splunk_app_stream/

To me it looks like your logs are showing that streamfwd is getting a connection refused when connecting via http to localhost on tcp port 8000. It's getting a connection refused when attempting that.

Are you able to, on that same splunk server, access:

http://localhost:8000/en-us/custom/splunk_app_stream/ping

If you can't that might indicate your problem, i.e. a local firewall, DNS resolution of "localhost" etc might not be working for you?

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...