Getting Data In
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Path Finder

where are the splunkd.logs located?

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Builder

SPLUNKHOME/var/log/splunk/splunkd.logs

On *nix home is usually /opt/splunkforwarder and on windows it would be under Program Files/splunkforwarder

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Path Finder

i did this command: index="_internal" 54.174.120.69 source="/opt/splunk/var/log/splunk/splunkd.log" and I get this error:

1/12/15
6:30:44.095 PM

01-12-2015 18:30:44.095 +0000 ERROR TcpInputProc - Error encountered for connection from src=54.174.120.69:60649. Local side shutting down
host = ip-192-168-16-190 source = /opt/splunk/var/log/splunk/splunkd.log sourcetype = splunkd
1/12/15
6:30:44.095 PM

01-12-2015 18:30:44.095 +0000 ERROR TcpInputProc - Error encountered for connection from src=54.174.120.69:60648. Local side shutting down
host = ip-192-168-16-190 source = /opt/splunk/var/log/splunk/splunkd.log sourcetype = splunkd
1/12/15
6:30:44.045 PM

01-12-2015 18:30:44.045 +0000 ERROR TcpInputProc - Error encountered for connection from src=54.174.120.69:60546. Local side shutting down
host = ip-192-168-16-190 source = /opt/splunk/var/log/splunk/splunkd.log sourcetype = splunkd
1/10/15
12:28:46.502 AM
01-10-2015 00:28:46.502 +0000 WARN TcpOutputProc - Cooked connection to ip=54.174.120.69:9997 timed out
host = ip-192-168-16-190 source = /opt/splunk/var/log/splunk/splunkd.log sourcetype = splunkd
1/10/15
12:28:16.500 AM
01-10-2015 00:28:16.500 +0000 WARN TcpOutputProc - Cooked connection to ip=54.174.120.69:9997 timed out
host = ip-192-168-16-190 source = /opt/splunk/var/log/splunk/splunkd.log sourcetype = splunkd

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Path Finder

and with the first command, index=internal xx.xx.xx.xx, i get:
1/12/15
7:43:52.260 PM

192.168.48.247 - admin [12/Jan/2015:19:43:52.260 +0000] "GET /en-US/api/shelper?snippet=true&snippetEmbedJS=false&namespace=search&search=search+index%3D%22
audi%22+54.174.120.69&useTypeahead=true&useAssistant=true&showCommandHelp=true&showCommandHistory=true&showFieldInfo=false&=1421091322744 HTTP/1.0" 200 641 "https://prd-p-c325dgfktbm7.cloud.splunk.com/en-US/app/search/search?q=search%20index%3D%22audit%22%2054.174.120.69&earliest=&latest=&display.page.search.tab=events&sid=1421091825.12991" "Mozilla/5.0 (Macintosh; Intel Mac OS X 1094) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36" - 54b423f8427f421431a250 20ms
host = ip-192-168-16-190 source = /opt/splunk/var/log/splunk/webaccess.log sourcetype = splunkwebaccess
1/12/15
7:43:46.729 PM
01-12-2015 19:43:46.729 +0000 INFO StatusMgr - destPort=9997, eventType=connect
close, group=tcpinconnections, sourceHost=54.174.120.69, sourceIp=54.174.120.69, sourcePort=33886, statusee=TcpInputProcessor
host = ip-192-168-16-190 source = /opt/splunk/var/log/splunk/metrics.log sourcetype = splunkd
1/12/15
7:43:46.707 PM
01-12-2015 19:43:46.707 +0000 INFO StatusMgr - destPort=9997, eventType=connect
done, group=tcpin_connections, sourceHost=54.174.120.69, sourceIp=54.174.120.69, sourcePort=33886, statusee=TcpInputProcessor
host = ip-192-168-16-190 source = /opt/splunk/var/log/splunk/metrics.log sourcetype = splunkd

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Path Finder

and it appears that I cant access my splunkd.logs from my forwarder

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Builder

Sorry missed this. Do not have administrative access to this system or are you just not finding the log?

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Builder

Okay a couple of things here. Is the 54.x.x.69 IP your universal forwarder? A couple of log entries indicate that something was trying to forward logs TO this IP which makes me think that this is your sandbox IP or there was some other misconfiguration.... Also, the local side shutting down errors might be missed heartbeats and could simply be when splunk is being restarted.

please execute the following on your forwarder to check connectivity:

splunk list forward-server and again checking the splunkd.log from the forwarder might help.

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Path Finder

54.x.x.69 is the IP where the universal forwarder was dled. After running the command I get:
Active forwards:
input-prd-p-c325dgfktbm7.cloud.splunk.com:9997 (ssl)
Configured but inactive forwards:
None

and Im trying to check the logs from the forwarder but I don't think any exists, but Ill try again

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Path Finder

no it exists, but I have to change permission

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Path Finder

01-09-2015 19:40:02.418 +0000 INFO ServerConfig - Will generate GUID, as none found on this server.
01-09-2015 19:40:02.418 +0000 INFO ServerConfig - My newly generated GUID is C0A9901E-8B38-4435-8677-2DA23C1595EA
01-09-2015 19:40:02.419 +0000 INFO ServerConfig - My server name is "ip-172-31-35-141".
01-09-2015 19:40:02.419 +0000 INFO ServerConfig - Found no site defined in server.conf
01-09-2015 19:40:02.419 +0000 INFO ServerConfig - My hostname is "ip-172-31-35-141".
01-09-2015 19:40:02.452 +0000 INFO ServerConfig - Setting HTTP server compression state=on
01-09-2015 19:40:02.452 +0000 INFO ServerConfig - Setting HTTP client compression state=0 (false)
01-09-2015 19:40:02.452 +0000 INFO ServerConfig - Default output queue for file-based input: parsingQueue.
01-09-2015 19:40:02.481 +0000 INFO LicenseMgr - Initing LicenseMgr
01-09-2015 19:40:02.481 +0000 INFO LMConfig - serverName=ip-172-31-35-141 guid=C0A9901E-8B38-4435-8677-2DA23C1595EA
01-09-2015 19:40:02.481 +0000 INFO LMConfig - connectiontimeout=30
01-09-2015 19:40:02.481 +0000 INFO LMConfig - send
timeout=30
01-09-2015 19:40:02.481 +0000 INFO LMConfig - receivetimeout=30
01-09-2015 19:40:02.481 +0000 INFO LMConfig - squash
threshold=2000
01-09-2015 19:40:02.481 +0000 INFO LMConfig - strictpoolquota=1
01-09-2015 19:40:02.481 +0000 INFO LMConfig - key=pool_suggestion not found in licenser stanza of server.conf, defaulting=''

0 Karma