One of our Web/mobile team is considering an innovation project involving Splunk integration. Basically, better tracking/early notification of errors so that we can prevent them. We’re attempting to work with the Splunk API.
What is the efficient way to access application logs? And moreover, where can we get sftp-ing log files and how to parse them manually?
What application are you referring to, Splunk or something else?
The only other thing I’m thinking of is sftp-ing log files off of the server and then parsing them manually, which doesn’t sound nearly as usable as the Splunk API.
Don't sftp if you can avoid it. It is FAR better to put a Universal Forwarder on the server where the logs are being written. If you have to use FTP, setup a Heavy Forwarder and build your own cron to pull the files in but do it atomically (copy them in with a temporary name/extension and once the files are down, rename them to what Splunk is expecting).