All Apps and Add-ons

Stream data in SplunkCloud but no forwarders in stream app on-premise

johnansett
Communicator

Hello!

We're using SplunkCloud but in a restricted environment, servers do not have direct access to internet/SplunkCloud. As such, I have a dedicated Stream App server on-premise and the UF's forward via a intermediate forwarder.

I have deployed the Splunk_TA_stream to some test UF's and I can see data in the stream index on SplunkCloud, although not from all servers.
I can only see the local stream app server as a forwarder - I cannot see the the forwarders on the Stream App on-premise, so I cannot validate functionality, configure them etc.

The TA has the inputs configured as follows:

[streamfwd://streamfwd]
splunk_stream_app_location = https://10.1.1.1:8000/en-us/custom/splunk_app_stream/
stream_forwarder_id =
index = stream
disabled = 0

No other configurations in the local dir and the Splunk_TA_Stream has not been changed since it was created by the install of the app. 10.1.1.1 is the Stream App server on-premise.

The _internal logs show no errors and I can see lots of data in metrics.
There should be no firewall between the UF's and the internal on-premise server, I ran the following:

[splunk@server1 local]$ curl -k https://10.1.1.1:8000/en-us/custom/splunk_app_stream/ping/

{"_key": "appsmeta", "_user": "nobody", "api_versions": {"ping": 1, "vocabularies": 1, "streamforwardergroups": 1, "indexers": 1, "httpinputs": 1, "users": 1, "captureipaddresses": 1, "streams": 1}, "id": "appsmeta", "dateLastUpdated": 1553228394376, "version": "7.1.2"}

Thanks for your help!!

0 Karma
1 Solution

johnansett
Communicator

The Stream App admin dashboards require access to the _internal events, so keep this in mind while architecting in a hybrid environment.

Configuration of streams is separate from this and as long as the inputs is configured correctly this can be mannaged via the app.

View solution in original post

0 Karma

johnansett
Communicator

The Stream App admin dashboards require access to the _internal events, so keep this in mind while architecting in a hybrid environment.

Configuration of streams is separate from this and as long as the inputs is configured correctly this can be mannaged via the app.

View solution in original post

0 Karma

asokanex
New Member

Hi,

I'm also facing the same problem, We have a splunk cloud environment and we have deployed stream app in HF and configured stream TA to point to HF in port 8000. But I can't see any matched forwarders in Distributed forwarder management in HF.

How can I send the streamfwd configurations to UF. Please advise.

0 Karma

joelscrt
Engager

Hello,

I'm facing the exact same issue, and I don't have the index specified in my inputs.conf file.

I' have an "Universal Forwarder" on a linux server with the Splunk_TA_stream app. When creating a new stream group in the app, it get pushed to the forwarder. ( I can see it when accessing "localhost:8889" ) So communication between my two machines is working. But on the "Stream App" I can only see events from the "Search Head Forwarder" where the stream app is installed on.

There are no errors in splunkd.log and streamfwd.log.

My external forwarder doesn't get matched as a forwarder in the "Distributed Forwarder Management" and doesnt' appear in "Stream Forwarder Status" dashboard.

Thanks for helping.

[edit]

I could solve my issue by setting the forwarder to send logs to my search head.

output.conf on the forwarder:

[tcpout]
defaultGroup = primary_indexers 

forwardedindex.2.whitelist = (_audit|_introspection|_internal)

[tcpout:primary_indexers]
server = SEARCH_HEAD_FQDN:9997

input.conf on search head:

[splunktcp://9997]
connection_host = ip
disabled=false

johnansett
Communicator

It's all about the internal logs - I'll architect to have the Splunk stream server peer with the cloud indexers - I can't forwrd to the stream server.
Thx for your input!

0 Karma

markhill1
Path Finder

You dont need the index stanza in the inputs.conf, have you tried removing it?
Index is set in the Stream app when you create a group and new stream collection in the Stream app.

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.