Your script must keep track of what has been read, and output only new items. If it is helpful and you are on version 5.0, you can use modular inputs which provides a checkpointing function that will make this tracking easier. http://docs.splunk.com/Documentation/Splunk/5.0/AdvancedDev/ModInputsCheckpoint ... View more

Assuming you are using Unix-like system here. Write a Bash script that will collect information from that URL using curl . Test it prints the output to STDOUT when you run the script. Once testing of script is complete, set-up a scripted input in Splunk... http://docs.splunk.com/Documentation/Splunk/5.0/AdvancedDev/ScriptSetup ... View more