Hello Everyone!
I installed Splunk and Alert Manager Enterprise in Virtualbox for learning purposes (4cpu /8gb ram).
I configured AME via the documentation.
Health Check is green. I can send test alerts, they appear in the ame_default index.
However the alerts don't appear in the Events. Hang up forever.
I have some broken pipe errors, but they also appear in an another working environment.
Thank you for your help.
A
Hi Andras
Are there any errors visible with
index=_internal source=*ame* ERROR | table _time host source _raw
Please open a support case if you cannot share this information publicly.
Regards,
Simon
Hi @seiimonn !
I noticed that, every time i start AME Events, i get the following error.
I appreciate your help.
A.
2/20/25 8:40:34.605 AM | 127.0.0.1 - splunk-system-user [20/Feb/2025:08:40:34.605 +0100] "GET /servicesNS/nobody/alert_manager_enterprise/messages/ame-index-resilience-default-error HTTP/1.0" 404 177 "-" "splunk-sdk-python/1.7.3" - - - 0ms |
Hi @seiimonn !
Debian GNU/Linux 12 (bookworm)
Splunk Enterprise 9.0.0
AME 3.0.8.
Sysinfo:
{"uuid":"95c6740c-9e0b-42b1-b2b9-b78067db6677","status":200,"messages":[],"payload":{"tenant_list":[{"tenant_uid":"default","role":"admin"}],"is_admin":true,"is_app_admin":true,"products":[],"necessary_tasks":[],"legacy_installed":false,"environment":"on_premises","timezone":"UTC"}}
There ara no errors now, if i run this script:
index=_internal source=*ame* ERROR | table _time host source _raw
But maybe these ara interesting in the splunkd.log:
19/02/2025 19:34:15.506 | 02-19-2025 19:34:15.506 +0100 WARN HttpListener [1069 HttpDedicatedIoThread-4] - Socket error from 127.0.0.1:37790 while accessing /servicesNS/nobody/alert_manager_enterprise/properties/server: Broken pipe
| |
19/02/2025 19:34:08.828 | 2025-02-19 19:34:08,828 INFO [assist::supervisor_modular_input.py] [context] [build_supervisor_secrets] [22691] Secret load failed, key=tenant_id, error=[HTTP 404] https://127.0.0.1:8089/servicesNS/nobody/splunk_assist/storage/passwords/tenant_id?output_mode=json
| |
19/02/2025 19:34:06.362 | 02-19-2025 19:34:06.362 +0100 WARN HttpListener [1068 HttpDedicatedIoThread-3] - Socket error from 127.0.0.1:52422 while accessing /servicesNS/nobody/alert_manager_enterprise/properties/server: Broken pipe
| |
19/02/2025 19:33:56.500 | 2025-02-19 19:33:56.500 +0100 Trace-Id= type=METER, name=ch.qos.logback.core.Appender.error, count=3, m1_rate=3.527460396057507E-12, m5_rate=9.325633072421824E-5, m15_rate=7.016228689718483E-4, mean_rate=0.0019981503731937404, rate_unit=events/second
| |
19/02/2025 19:33:54.415 | 2025-02-19 19:33:54,415 INFO [assist::supervisor_modular_input.py] [context] [build_supervisor_secrets] [22467] Secret load failed, key=tenant_id, error=[HTTP 404] https://127.0.0.1:8089/servicesNS/nobody/splunk_assist/storage/passwords/tenant_id?output_mode=json
|
I use this script, to create test alerts:
| makeresults | eval user="World", src="192.168.0.1", action="create test event" | sendalert create_alert param.title="Hello $result.user$" param.template=default param.tenant_uid=default
I think there is nothing interesting on the browsers developer console. What do you think about that?
Thanks for your helping.
Hi @Andras ,
you can see in Alert Manager App only alerts share at Global level, so you have to change the permissions in your alerts from App level to Global level.
Ciao.
Giuseppe
Hi @gcusello !
Also interesting that the alerts in the index seems good:
But the loading of the events in the Events dashboard never ending.
Hi @Andras ,
if you haven't the issue about the Global Sharing of the Alerts, check the macros used in the dashboards, probably you have to specify the index where Notables are located.
You can do it opening the dashboard.
Ciao.
Giuseppe
Hi Giuseppe!
Thank you for your answer.
I double checked, but my alerts are alredy global.
I think there is another problem.
Thanks,
A