You can check the HEC endpoints streamfwd is using by navigating to http://streamfwd-host:8889. By default this endpoint is only accessible from the same host (ie http://localhost:8889), but can be changed by adding ipAddr=0.0.0.0 param to `streamfwd.conf. This will open an internal debugging/monitoring web interface. From there, click on the [Stats] link on the white splunk box and check the returned JSON file. HEC endpoints are presented as "indexers" array, e.g.: ... indexers": [ "https://hec-host1:8088", "https://hec-host2:8088", ... "https://hec-hostn:8088", ] } Check if this data matches the expected protocol/host list. PS: be careful not to leave it open after you're done with the troubleshooting (for security concerns, etc). ... View more

hello @hemendralodhi, Load balancer is needed to scale the event ingestion beyond what a single HEC endpoint can process. I don't have the exact perf numbers (yet), but the ball park is that a single instance of independent Stream Forwarder can collect and process/push to indexers up to ~several TB/day of netflow data. A single HEC-enabled indexer won't be able to handle such load, so that's why we recommend a load-balanced architecture. ... View more

HI @mathiask, Re: 1. Stream has REST API that can be used to update the black/white list. If that's something you'd consider, we can come up with some sample code (python). Re: 2. NetFlow is received via a UDP listening socket. Do you want to whitelist/blacklist certain IP addresses that can/cannot send Netflow to Stream, or want to filter Netflow events by IP addresses? For former, you probably want to configure it on a firewall or router; latter can be achieved by configuring a filter on netflow stream(s) in the Configure Stream UI. ... View more

@ZacEsa, To uninstall independent Stream Forwarder, you can do something like the following as root (you may need to change the chkconfig command to what your distro is using for init.d daemons management): service streamfwd stop chkconfig --del streamfwd rm -rf /opt/streamfwd/ rm -rf /etc/init.d/streamfwd That said, you should be able to run both independent Stream Forwarder and Stream Forwarder TA under Universal forwarder on the same machine in parallel, so I'm not sure the error message you're seeing is due to the co-existance of these packages. Have you run the ./set_permissions.sh script on the Stream Forwarder TA? ... View more

Hi @wweiland, Sorry to hear about your troubles with Stream.. What OS is your heavy forwarder running on? What's the Stream forwarder config there? Have you run ./set_permissions.sh script (assuming it's *nix)? Do you have anything suspicious in $SPLUNK_HOME/var/log/splunk/streamfwd.log file on the heavy forwarder? Do you have _internal index on heavy being forwarded from HFW to IDX? ... View more

@mathiask, While it may be worth troubleshooting HWF -> IDX forwarding performance, there's a intrinsic limit of how much data one can push through a modular input interface (ie. STM -> HWF). The error you're receiving suggests that you're either hitting this limit, or Stream modinput is throttled by HWF. Have you checked the effective maxKBps setting the heavy forwarder is running with (http://docs.splunk.com/Documentation/Splunk/latest/Admin/Limitsconf)? It defaults to some small value and while Stream ships with limits.conf that sets this parameter to 0 (unlimited) it may not be the one Splunk uses (if other apps contain conflicting values, etc.) If tuning the modinput and HWF forwarding doesn't solve the problem, you may want to consider deploying Stream Forwarder as a stand-alone agent (http://docs.splunk.com/Documentation/StreamApp/7.0.0/DeployStreamApp/InstallStreamForwarderonindependentmachine) that sends data to indexers directly via HTTP Event Collector API. This deployment avoids modinput limitations and can be scaled to send to multiple indexers with either a load balances such as Nginx (preferred) or specifying a list of HEC indexers in Distributed Forwarder Management UI inside Splunk Stream App. PS: I'll pass your feedback to the doc team ... View more

@jeff unfortunately, Stream currently doesn't dissect SSH. It's on our roadmap for 2017. ... View more