https://community.splunk.com/t5/Getting-Data-In/How-to-assure-data-arrived-from-script-web-server/m-p/27315#M4575 <P>Hi,<BR /> I have a scheduled data input script in Splunk which runs every hour, the script is a basic wget from a web server, sometimes that web server is not responding well and causing us to lose data of an hour.<BR /> Is there any way to check if data has arrived and if not re-run the script? or any other more efficent way to retreive data from the web server?<BR /> the reason why we have this on an hourly cron is that the data only changes once an hour, and I have noticed that when we run it multiple times we get duplicate data in our searches(maybe there is a way to overcome this?)</P> <P>Thanks in advance</P> Mon, 11 Feb 2013 13:42:53 GMT nirt 2013-02-11T13:42:53Z https://community.splunk.com/t5/Getting-Data-In/How-to-assure-data-arrived-from-script-web-server/m-p/27315#M4575 <P>Hi,<BR /> I have a scheduled data input script in Splunk which runs every hour, the script is a basic wget from a web server, sometimes that web server is not responding well and causing us to lose data of an hour.<BR /> Is there any way to check if data has arrived and if not re-run the script? or any other more efficent way to retreive data from the web server?<BR /> the reason why we have this on an hourly cron is that the data only changes once an hour, and I have noticed that when we run it multiple times we get duplicate data in our searches(maybe there is a way to overcome this?)</P> <P>Thanks in advance</P> Mon, 11 Feb 2013 13:42:53 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-assure-data-arrived-from-script-web-server/m-p/27315#M4575 nirt 2013-02-11T13:42:53Z https://community.splunk.com/t5/Getting-Data-In/How-to-assure-data-arrived-from-script-web-server/m-p/27316#M4576 <P>You could run a scheduled search every hour to alert you whenever there is no/not enough data in the previous hour.</P> Mon, 11 Feb 2013 14:28:03 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-assure-data-arrived-from-script-web-server/m-p/27316#M4576 martin_mueller 2013-02-11T14:28:03Z https://community.splunk.com/t5/Getting-Data-In/How-to-assure-data-arrived-from-script-web-server/m-p/27317#M4577 <P>I thought about that but I wanted some more generic/automatic option.. the alert can trigger the script and then it will populate the data - that should work right?<BR /> Is there any more basic way to do it? it seems like some workaround</P> Mon, 11 Feb 2013 14:33:23 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-assure-data-arrived-from-script-web-server/m-p/27317#M4577 nirt 2013-02-11T14:33:23Z https://community.splunk.com/t5/Getting-Data-In/How-to-assure-data-arrived-from-script-web-server/m-p/27318#M4578 <P>The best option in my opinion is to build this logic into the script instead.</P> Mon, 11 Feb 2013 14:45:58 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-assure-data-arrived-from-script-web-server/m-p/27318#M4578 Ayn 2013-02-11T14:45:58Z