About emcclure

emcclure · ‎02-25-2019

Ok I was able to build a new setup with Rancher and try and configure splunk again. However this time when I run the command I get this: {"text":"Data channel is missing","code":10} When I setup the HEC part in Rancher I used port 8088 for the endpoint and the token I got when I configured it. For the index I used the one I created in splunk and I left the Source field blank as I wasn't sure what needed to go there, and it saved anyway. What am I doing wrong? Is there something I'm missing still?

emcclure · ‎02-22-2019

Ok I'll do that, but I'll have to create a new setup of Rancher. Had to re-ip some hosts and it's caused nothing but issues for me. Rancher setup seems to have somehow 'reset' itself and wants me to basically start all over, as if I never did anything at all. So when I setup splunk in Rancher and I list the endpoint I should use port 8088? And then when I run the command as shown above just modify the hec.example.com to the splunk server and then the token I used to connect the splunk server to Rancher? If it's running http does that cause any issues? I'm guessing just changing the curl command to http is ok, or is more involved?

emcclure · ‎02-20-2019

Ok so if I understand this right in the command the https://hec.example.com is the name of the splunk server? And the port is what's used when logging in thru the browser, which I have at 8000, or should it be 8088? Then I need to specify the token name and token value as well where it says Splunk B5A79AAD-D822-46CC-80D1-819F80D7BFB0? Here's what I get when I try this: curl -k http://mysplunk.server:8000/services/collector/event -H "Authorization: Rancher " -d '{"event": "hello world"}' {"text": "Success", "code": 0} I get this: See Other The resource has moved temporarily here. curl: (3) [globbing] unmatched brace at pos 7 curl: (6) Could not resolve host: Success,; Unknown error curl: (6) Could not resolve host: code; Unknown error curl: (3) [globbing] unmatched close brace/bracket at pos 2 If I try it like this: curl -k http://mysplunk.server:8088/services/collector/event -H "Authorization: Rancher " -d '{"event": "hello world"}' {"text": "Success", "code": 0} I get this: curl: (52) Empty reply from server curl: (3) [globbing] unmatched brace at pos 7 curl: (6) Could not resolve host: Success,; Unknown error curl: (6) Could not resolve host: code; Unknown error curl: (3) [globbing] unmatched close brace/bracket at pos 2 If I make the command https and use this: curl -k https://mysplunk.server:8000/services/collector/event -H "Authorization: Rancher " -d '{"event": "hello world"}' {"text": "Success", "code": 0} I get this: curl: (35) SSL received a record that exceeded the maximum permissible length. curl: (3) [globbing] unmatched brace at pos 7 curl: (6) Could not resolve host: Success,; Unknown error curl: (6) Could not resolve host: code; Unknown error curl: (3) [globbing] unmatched close brace/bracket at pos 2 And finally if I use the same command but change the port to 8088 I get this: {"text":"Invalid authorization","code":3}curl: (3) [globbing] unmatched brace at pos 7 curl: (6) Could not resolve host: Success,; Unknown error curl: (6) Could not resolve host: code; Unknown error curl: (3) [globbing] unmatched close brace/bracket at pos 2 So what am I doing wrong here?

emcclure · ‎02-20-2019

Ok I will try that out. The port used in the web address is 8000. Is that an issue at all or do I need to specifiy port 8088 no matter what?

emcclure · ‎02-19-2019

Hello, I'm totally new to this and have been thrown into the fire to figure things out. I'm setting up Kubernetes with Rancher and want to integrate splunk monitoring with it. That's where I'm stuck. I've followed the steps here: https://rancher.com/docs/rancher/v2.x/en/tools/logging/splunk/ and also looked at these docs for creating the event collector tokens in splunk: https://docs.splunk.com/Documentation/Splunk/7.0.0/Data/UsetheHTTPEventCollector#Configure_HTTP_Event_Collector_on_Splunk_Enterprise, but I'm not seeing anything at all in splunk. I'm not sure if I've misconfigured something or what the deal is, but I've been looking around on the Rancher forums and on their slack channel and have received no help at all on it, so I'm hoping that someone can help me out here. I've never setup splunk before for Kubernetes and this is a totally new project, trial by fire you might say. Any help is appreciated. Please let me know if any other info is needed.

emcclure · ‎07-17-2018

So to update some findings here: If I install splunk universalforwarder 7.1.2 on a machine that's either splunktest or splunk-test and it's reporting to a splunk server that's 7.1.1 it will install, but the service will stop running. If I try to name the machine with an _ somewhere, say splunk_test Windows doesn't like it as it's a non-standard character. If I try a . it fails and tells me I can only use letters, numbers or hyphens, but hyphens seem to cause an issue. The machine that has VSE on it is still running strong, service hasn't been killed yet. So have I found a bug with the hyphen in the name? Is that fixed in 7.1.2, but I'll need the reporting server to be upgraded first?

emcclure · ‎07-17-2018

Well so what I've noticed so far is if the machine name has a - anywhere in it it fails, even though from what I can tell this shouldn't be the case. If I create a machine called splunktest then splunk universalforwarder installs just fine and the service continues to run w/o issue. If I create a machine called splunk-test the install completes, however the service will stop after a minute. If I then rename that machine to sptest then the service stays up. If I create a machine called -PC the install will not complete and give me this error: ERROR: serverName must start with a letter, number, or underscore. You have: -PC. The last one I find odd. It tells me what I can't have as the first character which is ok, but even if I have that - elsewhere I still have problems. I'll have to confirm if I still have this problem with 7.1.2. I think the server it reports to is 7.1.1, so I'm not sure if that would create an additional problem or not. Another issue I found is if you have the Windows firewall running with the default settings it will cause the splunk forwarder service to not work properly. I have also installed McAfee VSE 8.8 Patch 11 for this Windows 10 RS4 image. I found this link: https://docs.splunk.com/Documentation/Splunk/7.1.1/ReleaseNotes/RunningSplunkalongsideWindowsantivirusproducts and entered that info in as well to see if the service will stop or not. So far after the install and configuring VSE the service continues to run and is reporting to the splunk server. However I'm going to reboot and see if it still comes up. So far so good. I am going to see what happens for a bit and then update this post, I will also install 7.1.2 on a couple machines to see if I get the same results or not.

emcclure · ‎06-29-2018

Hello, I'm new to splunk so please bear with me. I have just installed the forwarder service on a Windows 10 RS4 x64 image. It appears after a couple minutes that the service just stops. I then get an error that you see here: answers.splunk.com/answers/609072/i-am-not-able-to-run-splunkkd-service-on-windows10.html and here answers.splunk.com/answers/301878/has-anyone-come-across-the-error-the-splunkforward.html. I have found the log and here are the errors I get: ERROR TcpOutputProc - LightWeightForwarder/UniversalForwarder not configured. Please configure outputs.conf. 06-29-2018 14:13:37.135 -0700 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-powershell.exe"" Method invocation failed because [System.Management.Automation.PowerShellAsyncResult] does not contain a method named 06-29-2018 14:13:37.135 -0700 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-powershell.exe"" 'Close'. 06-29-2018 14:13:37.135 -0700 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-powershell.exe"" At C:\Program Files\SplunkUniversalForwarder\bin\splunk-powershell.ps1:280 char:1 06-29-2018 14:13:37.135 -0700 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-powershell.exe"" + $psDisposer.runspace.Close() 06-29-2018 14:13:37.135 -0700 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-powershell.exe"" + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 06-29-2018 14:13:37.135 -0700 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-powershell.exe"" + CategoryInfo : InvalidOperation: (:) [], ParentContainsErrorRecordException 06-29-2018 14:13:37.135 -0700 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-powershell.exe"" + FullyQualifiedErrorId : MethodNotFound 06-29-2018 14:13:37.135 -0700 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-powershell.exe"" 06-29-2018 14:13:37.775 -0700 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\etc\apps\SA-ModularInput-PowerShell\bin\healthlog.bat"" ^C I didn't see a python folder as mentioned in the first link above and since it didn't say what was deleted I'm not going to attempt anything like that. I have tried to reinstall the service, but I get the same issue. Afaik we have it installed on other machines, I'm not sure about Win 10 RS4 but I'd like to think so. The plan is to have this installed on a template that would be cloned by users and then everything would report to the splunk server. However I don't know if that's possible, if I need to install it again as the splunk forwarder was installed on the template when I cloned it, so I don't know if that caused a problem or not. I'd also like to know if the splunk forwarder sends data to the splunk server even if nobody is logged onto the machine, or if someone was to create one from the template but not login to it for some time. Any and all help is appreciated. Thanks in advance.

Posts	8
Solutions	0
Karma Given	0
Karma Received	1
Member Since	‎06-29-2018

Online Status	Offline
Date Last Visited	‎06-05-2020 02:04 AM

Configuring Splunk and Rancher

splunkd service 7.1.1 on Windows 10 RS4 x64 keeps ...

Re: Configuring Splunk and Rancher

Re: Configuring Splunk and Rancher

Re: Configuring Splunk and Rancher

Re: Configuring Splunk and Rancher

Configuring Splunk and Rancher

Re: splunkd service 7.1.1 on Windows 10 RS4 x64 ke...

Re: splunkd service 7.1.1 on Windows 10 RS4 x64 ke...

splunkd service 7.1.1 on Windows 10 RS4 x64 keeps ...