- Splunk Answers
- :
- Using Splunk
- :
- Splunk Search
- :
- Re: Scheduled Searches delayed by one hour
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Scheduled Searches delayed by one hour
Hi,
I have lately seen an issue that some scheduled alerts that contain attachments seem to get emailed to me one hour later than scheduled. I assume this has to do with volume of the alerts and searches scheduled but I haven't been to able to nail it down. When I look in the metadata for information about the delayed scheduled search it does show that it ran at the scheduled time, however the email alert often arrives exactly one hour later in my inbox.
Has anyone experience with this kind of issue?
Oliver
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If Splunk says it sent the alert at 6AM (utc-5h) and it was recieved at 7AM(utc-5) then the delay is external to splunk.
Do you have mailserver logs or a mail provider you can investigate with?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I don't think it's the TZ since they are delayed only some of the time.
So the alert fired at 6:00 AM, when it is scheduled to run:
2020-02-07 06:00:27,173 -0500 INFO sendemail:134 - Sending email. subject="Splunk Alert: 24 Hour Remote Country Login with MFA Check", results_link="https://XXXX.splunkcloud.com/app/XX_custom_app/@go?sid=scheduler_c2VjX2das9saXZlci5tdWVsXXXbGVyQG9tZy5jcdasdmhvbWcuY29t_T01HX2N1c3RvbV9hcHA__RMD5a1b9cc0db37475e5_at_1581073200_64487", recipients="[u'XXX@XXX.com']", server="XX.XX.XX.1:25"
However the email arrived in my inbox at 7 AM. Yesterday it came at 6 AM as scheduled, so it's not very consistent. But when it is delayed it is delayed but 1 hour.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you, guys. The mail idea brought me on the right track.
The email apparently sometimes gets quarantined because of its attachment and then released automatically an hour later.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Could well be, reports are scheduled based on the TZ settings of the user who scheduled them.
If the user has the wrong TZ set, they will defer according to that offset.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
^ Exactly this, thanks for expanding
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The exactly one hour part makes me think timezone related for some reason, mainly as we've had to deal with GMT\BST data issues here, possibility at all?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Can you see from the _internal logs when the email was sent. Was it at the anticipated time, or shortly before recieved?
Routing logs with Splunk OTel Collector for Kubernetes
Welcome to the Splunk Community!
Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM
