RE: Case #3270697 After upgrade to 9.1.01 not able to send emails eg. of critical alerts! [ ref:_00D409oyL._5005a2bGRKI:ref ]
After upgrade to v188.8.131.52 Splunk Enterprise, (single instance), last weekend (15 Juli 2023) + changing admin password as was suggested by Assist (which throws an error now !?)
1) Message when using sendemail:
Smpt setting: O365
Checked the login on O365, ofcourse
2) Assist stopped running???
4) new GUI / layout ?
5) Annoying and not working “Don’t show this again” message on every page. Just stepping to another dashboard on the same server/domain ??
6) endless waiting:
What is next?
Anyone else suffering from the same issues?
Last week v9.1.2 has been released. (6 nov 2023, I think it was)
After installing this version on my test instance (v9.1.1) everytyhing seems to work again including sendemail - no issues found. Great! 👍
After installing this version several days later on our production instance (184.108.40.206) also sendemail was working fine again. Great! 👍
NB. after that I was also able to fix all other issues on our production instance as mentioned before in this post, like: kvstore, secure gateway etc Great! 👍
Many thanks to support- and development team. I am now happy splunking again! 👍👍 😀
I hereby close this post.
You should replace mycert.pem with your own cert file. That name you could see by previous btool. Use also full path to this file. Sometimes it’s also needed to add “splunk cmd OpenSSL” instead of just “openssl”.
Our https certificate (web) is not expired. So what and where should I remove this old cert reference.
We only run Indexer, SH , licens Manager and Kvstore process....and nothing else
Please note: we run a small and single Splunk Enterprise instance on a Intel 16core , 64Gb mem, and 2Tb hardware under windows2019.
BTW How many certs do I need to run and manage in v220.127.116.11?
There are own certificates for web, splunkd and traffic between UFs and indexers. Mongod is using the same certificate as splunkd. You should check from server.conf where it is.
On my Test-server, I have copied a fresh server.conf from default directory and changed the license manager field. It is coming up and I can login but but still hangs on endless "loading..." when trying to change something (like restart)
Also Assist show an error:
Yesterday we had a remote session (Zoom) with Splunk Support. No quick fix found. It needs to be investigated further:
- After upgrading the Splunk version to 18.104.22.168, the sendemail command is not working it is giving some errors on the production instance and test instance.
command="sendemail", The email domains of the recipients are not among those on the allowed domain list. while sending mail to: a.pietersen@....
command="sendemail", (530, b'5.7.57 Client not authenticated to send mail. [AS4P195CA0039.EURP195.PROD.OUTLOOK.COM 2023-07-25T10:35:23.523Z 08DB887EFF8B2458]', 'pietersen@i....) while sending mail to: a.pietersen@....
- The top of the screen showed a loading icon as I was reviewing the Server settings for the Email settings. Nothing can be done on the Email settings page.
- Additionally, we attempted to restart the UI by accessing the server controls. The loading icon at the top of the page appeared on the server control page as well, indicating the same problem.
- Additionally, the Splunk assist page is not working and displays the error message "Error Loading Splunk Assist."
- Even though we attempted to restart the search head using the CLI, the errors and issues persisted after the restart.
Thanks, hope they can find a remedy soon, because I am out of my option for now.
Normally this message means, that your smtp relay don’t allow you to send email with your from-domain. Are you sure that your M365 exchange has configured to relay those emails?
Both account belongs to me but have different domains. (and no trust-relation in place) Manual sending is no problem and use this daily. It was working for years now based on O365, it sopped after the upgarde. Besides no messages were received form MS that O365 have changed their relay rules. So for now I suspect something not OK in the Splunk 22.214.171.124 environment.
My main problem was not able to send critical Alerts and other message by email any more after upgradding to v126.96.36.199.
Finally I was able to solve this by copying the latest 'sendemail.py" from my backup of v9.0.5. into directory: ....\Splunk\etc\apps\search\bin . After that I could send email as before.
Note: The "sendemail.py" that I copied from backup seems to be 2 lines longer than the new one installed by v188.8.131.52. Did not investigate it further, what the differences is. Busy to correct temp workarounds and made me and my users happy Splunking again. 🙂 Other issues are still open, but for now no showstoppers. Have send this info also to Splunk support but did not here back from them yet.
Last Friday Spiunk Support responded: problem with "sendemail" was identified as a bug and will be solved in next version.
Nb. Just noticed version v184.108.40.206 but there was no mention of/refering to this specific fix. I will wait for next version. Sendemail is still working after restoring the senemail.py form backup v9.0.5