Alerting

Why is my curl command unable to schedule an alert to a future time?

kamal_jagga
Contributor

Hi,

Ideally, below mentioned command should work, but Curl command is unable to schedule alert to future time

curl -k -u username:password http://.../services/saved/searches/${ALERT_NAME}/reschedule -d schedule_time=2016-08-15T14:11:01Z

It starts the alert/search at the time of executing the command, instead of required future time.

Kindly advise.

behlkush
Path Finder

After trying everything with reschedule i had to let go of it and as suggested by you Kamal_jagga i used what you have written as final solution to use cron_schedule instead of reschedule

0 Karma

frobinson_splun
Splunk Employee
Splunk Employee

Hello @jkat54 and @kamal_jagga,
I'm a tech writer here at Splunk and I am looking into this issue with our engineering team. There may be a problem with the time format for the parameter. Could you try passing in the schedule_time parameter in the following format?

sss[.fff] where sss is the number of seconds since Epoch (Jan 1, 1970) and, optionally, fff is the fractional seconds.

If you need to convert a timestamp to the number of seconds since epoch, there are a few tools available. For example:
http://www.epochconverter.com/

We are still looking into this, so this is just a suggestion to try out. Could you let me know if it works, either way?

Thank you.

jkat54
SplunkTrust
SplunkTrust

This epoch time is some day in 2022:

alt text

I get a good response, but the next_scheduled_date doesnt update:
alt text

next_scheduled_time 2016-01-23 01:49:19 Eastern Standard Time

0 Karma

jkat54
SplunkTrust
SplunkTrust

I tried with %3N (ms) too

0 Karma

frobinson_splun
Splunk Employee
Splunk Employee

Here are some additional tips from our engineering team.

-cron_schedule must be a five-field value. Can you try passing it in as one?
-Because of the way curl works, it might be best to not include % escaping inside the curl request. Pass no escaped characters.
-It's probably also better to do multiple --data options rather than manually using a & character since curl MAY escape that as well

I hope these help!

kamal_jagga
Contributor

@frobinson:

My Issue : Sometimes data files come later than their scheduled time. So my alerts don't produce any results, so i have to re-run them again later. In such cases I am trying to automatically reschedule the alert 2 or 3 hrs later than the original time.

0 Karma

frobinson_splun
Splunk Employee
Splunk Employee

@kamal_jagga,
Thank you for the details of your use case. I am sorry for the confusion about using this endpoint. I will look into whether there is a bug with its behavior.

For now, I would recommend considering some changes to your scheduling for the alert and adjusting its scheduling using the Splunk Web alert configuration UI. Could you adjust the alert timing or frequency in order to capture the results you are looking for? I'm not sure if you have looked into the search time range and the alert cron schedule, additionally. Sometimes when these are out of sync, results can be missed. In our docs:
"To avoid overlaps or gaps, the execution schedule should match the search time range. For example, to run a search every 20 minutes the search's time range should also be 20 minutes (-20m). "

http://docs.splunk.com/Documentation/Splunk/6.3.1511/Alert/Definescheduledalerts

Another option, to get more specific help with your use case and data, would be to contact support:
http://www.splunk.com/en_us/about-us/contact.html#customer-support

Thanks!

0 Karma

jkat54
SplunkTrust
SplunkTrust

You're using GET not POST. GET is default for curl and therefore your data in -d is being discarded by the API and it's falling back to another snipet of code and running the job immediately instead. The examples dont show using -XGET -XPOST -XPUT -XDELETE -XUPDATE etc. switches in curl but I"m almost certain this is your issue.

Try adding -XPOST to your curl command.

http://docs.splunk.com/Documentation/Splunk/6.2.5/RESTREF/RESTlist has this:
alt text

0 Karma

kamal_jagga
Contributor

I am still unable to figure out the exact format of using the curl command to reschedule the alert to a future time. Also, I am unable to find a way to log only the required stuff.

0 Karma

kamal_jagga
Contributor

Also, would you be able to suggest anything to get only required logs. When i try to enable logs, it puts all the XML response.

0 Karma

kamal_jagga
Contributor

The following options are also not giving the expected output.

  1. This is creating a new cloned alert and scheduling the search at the mentioned cron time.

    cron_schedule=0 21 * * * to schedule it at 9pm everyday(formatting issue in the below command)

    curl -k -u username:password -XPOST https://localhost:8089/servicesNS/nobody/search/saved/searches/Errors%20in%20the%20last%2024%20hours... -d 'output_mode=json&cron_schedule=0 21 * * * &is_scheduled=true'

    1. This is triggering at the alert immediately.

    curl -k -u username:password -XPOST https://localhost:9086/services/saved/searches/testkamal2/reschedule schedule_time=1455842777
    Not sure, if I am missing out anything. Kindly advise.

0 Karma

jkat54
SplunkTrust
SplunkTrust

Your curl command isnt correct in the example you gave. Now it could have been the forum auto-changing your formatting when you posted your comment. So I changed it so that everything is in code blocks (indented 5 spaces on every line, "101010" button).

I suggest you try the below format. -d is data you send... -XPOST or -XGET or -XDELETE or -XPUT dont go "inside" of the -d switch. -d must be in xml or json format according to rest documentation.

curl -k -u username:password -XPOST http://.../services/saved/searches/${ALERT_NAME}/reschedule  -d {"schedule_time":"2015-01-20T14:38:01Z"}

alt text

0 Karma

kamal_jagga
Contributor

Hey Michael,

This is also giving error 😞
curl -k -u username:password -XPOST http://.../services/saved/searches/${ALERT_NAME}/reschedule -d {schedule_time=2016-01-29T14:38:01Z}
curl -k -u username:password -XPOST http://.../services/saved/searches/${ALERT_NAME}/reschedule -d {"schedule_time":"2016-01-29T14:38:01Z"}

<msg type="ERROR">

In handler 'savedsearch': Argument "{schedule_time:2016-01-29T14:38:01Z}" is not supported by this handler.

Also, would you be able to suggest anything to get only required logs. When i try to enable logs, it puts all the XML response.

0 Karma

jkat54
SplunkTrust
SplunkTrust

I figured it out:

curl -k -u username:password -XPOST https://localhost:8089/servicesNS/nobody/search/saved/searches/Errors%20in%20the%20last%2024%20hours... -d 'output_mode=json&cron_schedule=5 * * * *&is_scheduled=true'
0 Karma

jkat54
SplunkTrust
SplunkTrust

Well... it works with cron_schedule but not next_scheduled_time. I found this link which says next_scheduled_time is read only (http://dev.splunk.com/view/javascript-sdk/SP-CAAAEKZ)

0 Karma

jkat54
SplunkTrust
SplunkTrust

It also says next_scheduled_time is read only in the earlier link i posted. http://docs.splunk.com/Documentation/Splunk/6.2.5/RESTREF/RESTsearch#saved.2Fsearches

I got this post to work but it doesnt update the search:

 curl -k -XPOST -u username:password https://localhost:8089/servicesNS/nobody/search/saved/searches/Errors%20in%20the%20last%20hour/reschedule -d 'output_mode=json&schedule_time=2017-01-22T21:55:00-05:00'
0 Karma

jkat54
SplunkTrust
SplunkTrust

If anyone is curious as to how to figure this out... i was able to use fiddler2 to see the post my browser was making. So fired up fiddler, went to saved searches, made an edit to the schedule, read the raw packets, decoded the post. And i dont know where i got output_mode=json from but i dont believe it is required.

0 Karma

jkat54
SplunkTrust
SplunkTrust

Your curl command isnt correct in the example you gave. Now it could have been the forum auto-changing your formatting when you posted your comment. So I changed it so that everything is in code blocks (indented 5 spaces on every line, "101010" button).

So here's what I suggest you try the below format. -d is data you send... -XPOST or -XGET or -XDELETE or -XPUT dont go "inside" of the -d switch.

curl -k -u username:password -XPOST http://.../services/saved/searches/${ALERT_NAME}/reschedule  -d {schedule_time=2015-01-20T14:38:01Z}
0 Karma

kamal_jagga
Contributor

Hey Thanks for replying and apologies for the late reply. But it didn't work

Actually i have tried all options :

curl -k -u username:password http://.../services/saved/searches/${ALERT_NAME}/reschedule -X POST schedule_time=2016-01-20T14:25:01Z
==> Triggers the search immediately.

curl -k -u username:password http://.../services/saved/searches/${ALERT_NAME}/reschedule -X POST
==> Triggers the search immediately.

curl -k -u username:password http://.../services/saved/searches/${ALERT_NAME}/reschedule  -d -X POST schedule_time=2015-01-20T14:38:01Z
==>
 In handler 'savedsearch': Argument "-X" is not supported by this handler.

Still I am unable to use the curl command to reschedule saved searches to future time.

Kindly advise.

0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...