×

Join the Conversation

Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Ask a Question
petops147
Explorer
Member since: ‎04-23-2022
‎07-07-2022
Community Statistics
Posts 7
Solutions 0
Karma Given 0
Karma Received 1
Member Since ‎04-23-2022
View All

Re: peer nodes not receiving data from Universal f...

by in Getting Data In
‎05-07-2022 12:10 AM
‎05-07-2022 12:10 AM
Hello rich, Can you check this output to see if you can make any meaning out it: 05-07-2022 05:40:15.060 +0000 WARN TcpOutputFd [3311 TcpOutEloop] - Connect to 172.31.18.160:9997 failed. Connection refused 05-07-2022 05:40:15.060 +0000 ERROR TcpOutputFd [3311 TcpOutEloop] - Connection to host=172.31.18.160:9997 failed 05-07-2022 05:40:15.061 +0000 WARN TcpOutputFd [3311 TcpOutEloop] - Connect to 172.31.22.208:9997 failed. Connection refused 05-07-2022 05:40:15.061 +0000 ERROR TcpOutputFd [3311 TcpOutEloop] - Connection to host=172.31.22.208:9997 failed 05-07-2022 05:40:15.062 +0000 WARN TcpOutputFd [3311 TcpOutEloop] - Connect to 172.31.21.254:9997 failed. Connection refused 05-07-2022 05:40:15.062 +0000 ERROR TcpOutputFd [3311 TcpOutEloop] - Connection to host=172.31.21.254:9997 failed 05-07-2022 05:40:15.063 +0000 WARN TcpOutputFd [3311 TcpOutEloop] - Connect to 172.31.21.254:9997 failed. Connection refused 05-07-2022 05:40:15.063 +0000 ERROR TcpOutputFd [3311 TcpOutEloop] - Connection to host=172.31.21.254:9997 failed 05-07-2022 05:40:15.063 +0000 WARN TcpOutputFd [3311 TcpOutEloop] - Connect to 172.31.18.160:9997 failed. Connection refused 05-07-2022 05:40:15.063 +0000 ERROR TcpOutputFd [3311 TcpOutEloop] - Connection to host=172.31.18.160:9997 failed 05-07-2022 05:40:15.063 +0000 WARN TcpOutputFd [3311 TcpOutEloop] - Connect to 172.31.22.208:9997 failed. Connection refused 05-07-2022 05:40:15.064 +0000 ERROR TcpOutputFd [3311 TcpOutEloop] - Connection to host=172.31.22.208:9997 failed 05-07-2022 06:44:57.434 +0000 INFO loader [3846 MainThread] - SAML cert db registration with KVStore failed 05-07-2022 06:44:57.434 +0000 INFO loader [3846 MainThread] - Auth cert db registration with KVStore failed 05-07-2022 06:44:57.434 +0000 INFO loader [3846 MainThread] - JsonWebToken Manager registration with KVStore failed. ... View more

Re: peer nodes not receiving data from Universal f...

by in Getting Data In
‎05-06-2022 10:52 AM
‎05-06-2022 10:52 AM
index=_internal host=<your UF name/ip> Yes I can see both UFs. And can be seen as the defined names uf01 & uf02.  I can practically see all Splunk component when issue the command : index = _internal . ... View more

Re: peer nodes not receiving data from Universal f...

by in Getting Data In
‎05-06-2022 08:18 AM
‎05-06-2022 08:18 AM
Thank you for the response. Though I am new to splunk; however, here is how i configured the inputs.conf and outputs.conf in the Deployment-app subdirectory. the clients are phoning home /polling the DS and I can see the distributed Apps in the UF /etc/apps/ only that the  data I am monitoring are not getting into the  peer nodes or search peers. I have 3 Search Head Cluster members, 3 Peer nodes , 1 Deployer, 1 Cluster master/master node, 2 Universal forwarder, 1 DS in my AWS splunk environment. 1, I created a sub directory(fwd_receivers) in deployment-apps directory  :  deployment-apps>fwd_receivers>default>output.conf   2. I created another subdirectory(mon_input) in the deployment-apps directory for monitoring inputs: deployment-apps>mon_input>default>inputs.conf   * I am wondering if I am using the right path "default" instead of  "local"? splunk best practices is to only modify in the local directory and not in the default. I did my config based on some video tutorials I watched.   ... View more

Re: peer nodes not receiving data from Universal f...

by in Getting Data In
‎05-05-2022 09:56 AM
‎05-05-2022 09:56 AM
Thank you rich! However, I'm still stuck. I issue this command but no output messages; it was blank:   [splunkforwarder@uf01 splunk]$ [splunkforwarder@uf01 splunk]$ tail -f splunkd.log | egrep 'TcpOutputProc|TcpOutputFd' no messages came up. I have checked the security to see if any port is not allowed; everything seem alright Note: I am pretty new to splunk  ... View more

Why are peer nodes not receiving data from Univers...

by in Getting Data In
‎05-05-2022 04:25 AM
‎05-05-2022 04:25 AM
Hello, I recently setup a test environment(clustered deployment) on  AWS  to monitor and get data into the peer nodes. My environment include: cluster master(hosting license), 3 Indexers, 1 Deployer, 3 Search heads, 1 Deployment server and 2 Universal forwarders.  I configured the deployment server to push configuration to the forwarders, and all seem working fine; the forwarders are phoning home, their is sync between the DS and the UFs. But the peer nodes are not receiving the data. even though, I set up the listening port (9997). I did troubleshoot on the UF to see if they are pushing, excerpt of the output from the UFs: 05-05-2022 11:02:11.118 +0000 INFO HttpPubSubConnection [3276 HttpClientPollingThread_71E59550-AD46-4814-8460-DB66C1DD0BAD] - Running phone uri=/services/broker/phonehome/connection_172.31.28.182_8089_ip-172-31-28-182.ec2.internal_uf01_71E59550-AD46-4814-8460-DB66C1DD0BAD 05-05-2022 11:02:25.767 +0000 INFO TailReader [3323 tailreader0] - Batch input finished reading file='/opt/splunkforwarder/var/spool/splunk/tracker.log' 05-05-2022 11:02:31.076 +0000 INFO AutoLoadBalancedConnectionStrategy [3316 TcpOutEloop] - Connected to idx=172.31.21.254:9997, pset=0, reuse=0. using ACK. 05-05-2022 11:02:55.767 +0000 INFO TailReader [3323 tailreader0] - Batch input finished reading file='/opt/splunkforwarder/var/spool/splunk/tracker.log' 05-05-2022 11:03:01.006 +0000 INFO AutoLoadBalancedConnectionStrategy [3316 TcpOutEloop] - Connected to idx=172.31.22.208:9997, pset=0, reuse=0. using ACK. 05-05-2022 11:03:11.118 +0000 INFO HttpPubSubConnection [3276 HttpClientPollingThread_71E59550-AD46-4814-8460-DB66C1DD0BAD] - Running phone uri=/services/broker/phonehome/connection_172.31.28.182_8089_ip-172-31-28-182.ec2.internal_uf01_71E59550-AD46-4814-8460-DB66C1DD0BAD 05-05-2022 11:03:25.597 +0000 INFO TailReader [3323 tailreader0] - Batch input finished reading file='/opt/splunkforwarder/var/spool/splunk/tracker.log' 05-05-2022 11:03:30.890 +0000 INFO AutoLoadBalancedConnectionStrategy [3316 TcpOutEloop] - After randomization, current is first in the list. Swapping with last item 05-05-2022 11:03:30.891 +0000 INFO AutoLoadBalancedConnectionStrategy [3316 TcpOutEloop] - Connected to idx=172.31.21.254:9997, pset=0, reuse=1. 05-05-2022 11:03:55.596 +0000 INFO TailReader [3323 tailreader0] - Batch input finished reading file='/opt/splunkforwarder/var/spool/splunk/tracker.log' 05-05-2022 11:04:00.813 +0000 INFO AutoLoadBalancedConnectionStrategy [3316 TcpOutEloop] - Connected to idx=172.31.18.160:9997, pset=0, reuse=0. using ACK. 05-05-2022 11:04:11.124 +0000 INFO HttpPubSubConnection [3276 HttpClientPollingThread_71E59550-AD46-4814-8460-DB66C1DD0BAD] - Running phone uri=/services/broker/phonehome/connection_172.31.28.182_8089_ip-172-31-28-182.ec2.internal_uf01_71E59550-AD46-4814-8460-DB66C1DD0BAD 05-05-2022 11:04:25.596 +0000 INFO TailReader [3323 tailreader0] - Batch input finished reading file='/opt/splunkforwarder/var/spool/splunk/tracker.log' 05-05-2022 11:04:30.704 +0000 INFO AutoLoadBalancedConnectionStrategy [3316 TcpOutEloop] - Connected to idx=172.31.22.208:9997, pset=0, reuse=0. using ACK. 05-05-2022 11:04:55.596 +0000 INFO TailReader [3323 tailreader0] - Batch input finished reading file='/opt/splunkforwarder/var/spool/splunk/tracker.log' 05-05-2022 11:05:00.613 +0000 INFO AutoLoadBalancedConnectionStrategy [3316 TcpOutEloop] - Connected to idx=172.31.18.160:9997, pset=0, reuse=1. 05-05-2022 11:05:11.129 +0000 INFO HttpPubSubConnection [3276 HttpClientPollingThread_71E59550-AD46-4814-8460-DB66C1DD0BAD] - Running phone  Any idea on solution to this ?   ... View more
Contact Me
Online Status
Offline
Date Last Visited
‎07-07-2022 06:59 AM