I want to develop an application where users can select the index, timerange and sourcetype and all the logs found from that search will be collected and sent to a third-party system via their API for processing by their systems
The way I was planning to do it was as follows:
1- Use the Splunk SDK to create a custom search command that takes three parameters index, timerange and sourcetype.
2- The custom search command using Splunk SDK, runs a export search in the background to find all the logs and then dumps them in a file.
3- It then uploads the file to the third-party by using their upload API. All done at this point.
Please note that log file size can range from 5-10 GB so is the above a good way to achieve what I want? Kindly share your suggestions if there is a better way. I would ideally like to provide the user with a seamless experience.