Actually I just resolved this by merely clearing my browser cache in chrome!?!?!? What the heck, i spent weeks on this lol!! ... View more

This is what I am getting when I try: 12/15/15 9:15:59.810 AM 12-15-2015 14:15:59.810 +0000 ERROR UiAuth - Request from 172.16.2.11 to "/en-US/splunkd/__raw/servicesNS/admin/search/search/jobs" had multiple CSRF cookies with different values (first "4646275108905813148" then "12739196604488450756" Should I clear my browser? ... View more

Hey there... Just stubbed my toe on this post. I had to do the same thing and I used the splunk deployment server to do it. In my case I had to do it to windows servers and I wrote a powershell script that gets executed once (with interval=-1). If this is in fact windows here is a one liner to replace text in any file: PS C:\Users\597> cat .\test.txt Brent is not replaced PS C:\Users\597> $(gc .\test.txt).Replace("Brent","Alan") |sc .\test.txt PS C:\Users\597> cat .\test.txt Alan is not replaced PS C:\Users\597> So use this one-liner to replace text in a file right inline. Hope this trick helps. ... View more

It is important to note in this issue that this problem only exists when referring to the load balancer, if I go to each individual node it works just fine. It does give a warning because the SSL cert we are using is registered as the hostname of the loadbalancer and not the individual node. i dont think that splunk is "broken" per se, i think it is perhaps a configuration that needs to be done?!?! As always, thank you all for taking the time to help me out here. ... View more

I have done nothing out of the ordinary. Nothing is specifically configured, how can you tell that I am looking to port 8089. Any further help is MUCH appreciated! ... View more

Thank you VERY much for the quick response! What leads you to believe this? We just got the certs and they are set to expire in 3 years!?! Not saying that isnt the issue, just confusing to me. What are some further troubleshooting steps I can take? Again, thank you VERY much for your quick response! ... View more

I guess I needed to have python installed, which I did. Now I get this message when I run the sendemail.py script: [root@cdopeusvmesem01 ~]# python /opt/splunk/etc/apps/search/bin/sendemail.py Traceback (most recent call last): File "/opt/splunk/etc/apps/search/bin/sendemail.py", line 1, in import re, time, splunk.Intersplunk, splunk.mining.dcutils as dcu ImportError: No module named splunk.Intersplunk Is there a problem with the sendemail.py script? ... View more

This is still not working. I am able to send mail from the linux server itself. In spunk I am getting the following message still: 12/6/15 10:03:38.586 AM 2015-12-06 10:03:38,586 -0500 ERROR sendemail:355 - Connection unexpectedly closed while sending mail to: bver@rege.com host = cdopeusvmesem01 sourcetype = splunk_python 12/6/15 10:03:38.586 AM 2015-12-06 10:03:38,586 -0500 ERROR sendemail:113 - Sending email. subject="Splunk Alert: TestAlerts", results_link="https://cdopesem01:8000/app/search/@go?sid=scheduler__admin__search__TestAlerts_at_1449412200_76116", recipients="[u'amil@domain.com']", server="mail.pcloud.com" host = cdopeusvmesem01 sourcetype = splunk_python 12/6/15 10:00:09.945 AM 2015-12-06 10:00:09,945 -0500 ERROR sendemail:355 - Connection unexpectedly closed while sending mail to: sleppers@gmail.com host = cdopeuem01 sourcetype = splunk_python 12/6/15 10:00:09.945 AM 2015-12-06 10:00:09,945 -0500 ERROR sendemail:113 - Sending email. subject="Splunk Alert: Brents New Test", results_link="https://cdopesem01:8000/app/search/@go?sid=scheduler__admin__search__RMD5114f93919c0bc2ab_at_1449414000_76273", recipients="[u'bj@gmail.com']", server="mail.pcloud.com" The above is from the log file. What am I missing here?!?!?! It seems that I should not have to configure sendmail at all in linux?!? Any help is much appreciated as I have been working on this for a while. Please note that I edited the above to exclude my email and server info. ... View more

Ignore that I found it! ... View more

I am trying to send email from alerts and it is failied. Piping to sendemail works just fine. I dont get it! ... View more

Thank you SO MUCH! This got me what I needed to see. The following was in the log python log file: sendemail:355 - Connection unexpectedly closed while sending mail to: The following was in the scheduler log file: 12-05-2015 17:15:05.960 +0000 INFO SavedSplunker - savedsearch_id="212040597;search;Brents Test Alert", user="212040597", app="search", savedsearch_name="Brents Test Alert", status=success, digest_mode=1, scheduled_time=1449335700, dispatch_time=1449335703, run_time=2.146, result_count=20, alert_actions="email", sid="scheduler__212040597__search__RMD5aab130161880edb9_at_1449335700_69055", suppressed=0, thread_id="AlertNotifierWorker-0" I did not have sendmail installed on this server . Do I need to install and configure it? This is so confusing because when I run a search and pipe it thru sendemail to= I get the messages. Is there a how-to for setting up mail in splunk. What is the meaning of life? 🙂 Thanks everyone for your help. ... View more

So i just lost the battle to get a remote syslog server. My arch is as such: 1 Lic Server 1 Deployment Server 1 Master INdex Server 4 Index CLustered servers 1 isolated seach head with ES and barracuda installed on it 3 Node search head Where do I configure the syslog listener, port 5040 and port 5141? Is this done on the search head? Also given that we are a index cluster, will this index only be written on one of the index servers? Thanks for the help! ... View more

This issue has been solved by just using local sql account. Once I set this up all worked perfectly, so you guys are right, windows auth does not work in db conn v2. Thank you all for your help. ... View more

I am not even to the point where I can use inputs.conf. I cannot even get this setup to simply connect, so there is no inputs.conf. ... View more

Hey there sorry for the darkness... Yes it is something like that. It seems that there are hard coded paths in splunk URL's... We have 4 search heads. One 3 node sh cluster and one isolated sh for ES. We have this working for ES (indiv sh) but cannot get it to work with the three node cluster that is load balanced. ... View more

dwaddle - Thank you for the response. I love the term "racing tape", such a great visual! 🙂 All I did was disable the check functionally in Splunk and then reenabled it. ... View more

Rich - I never modified the script at all - just disabled it in Splunk apps?!?!? ... View more

The error I am getting is as follows: msg="A script exited abnormally" input="/opt/splunk/etc/apps/SA-Utils/bin/configuration_check.py" stanza="configuration_check://confcheck_es_system_requirements" status="exited with code 3" I am even more confused as I reenabled the check I had diabled in Alerts and reports. Iis called "Audit - ES System Requirements" . Any help is much appreciated! ... View more

I disabled this query/report and it gave me a wole new error. It looks like there is a script called /opt/splunk/etc/apps/SA-Utils/bin/configuration_check.py. can I just put an exit 0 at the begining? ... View more

Hey thanks for the response. My splunk cluster is not fully live yet and I want to reconfigure where all the index go because it is a mess right now... Is it possible to basically start over in term of how we lay out our indexes? I would need to do this for all indexes including the internal ones. ... View more