One of our Web/mobile team is considering an innovation project involving Splunk integration. Basically, better tracking/early notification of errors so that we can prevent them. We’re attempting to work with the Splunk API.
What is the efficient way to access application logs? And moreover, where can we get sftp-ing log files and how to parse them manually?
Don't sftp if you can avoid it. It is FAR better to put a Universal Forwarder on the server where the logs are being written. If you have to use FTP, setup a Heavy Forwarder and build your own cron to pull the files in but do it atomically (copy them in with a temporary name/extension and once the files are down, rename them to what Splunk is expecting).