Splunk HF exports logs using a CLI command as a Linux crontab?

[sarvananth]

I'm new to Splunk Enterprise, and my task is to forward logs from Splunk HF (AWS EC2 instance) to an AWS Cloud Watch log group.

I tried to export the logs using CLI commands and stored them on the Splunk HF server locally. Then, I used the Cloud Watch agent to send the logs to the Cloud Watch log group.

please refer the below Splunk cli command for export the logs

#./splunk search "index::***** sourcetype::linux_audit" -output rawdata -maxout 0 -max_time 5 -auth splunk:***** >> /opt/linux-Test01.log

The challenge I'm facing is that when I run the CLI command using a Linux crontab, it does not export the logs.

Are there any other solutions or guidance available to resolve this issue?

[isoutamo · Accepted solution]

Hi

when you are running commands from cron, you must remember that there is no sourced environments. Usually you should use some wrapped script to:

1. source your user's shell environment including env vars + etc.
2. run your commands / scripts with full paths!

In your case this means e.g.

#!/bin/bash
source ~<your user>/.bash_login
source ~<your user>/.bashrc
cd <where ever you thing you should be>
/opt/splunk/bin/splunk .....

Of course you should add needed error checks after commands etc.

r. Ismo 

[sarvananth]

Hi R.Ismo,

yes, it is working fine, and thank you very much for your help.
Is it possible to export the logs last 5 minutes using a CLI command?

[isoutamo]

Basically just …/splunk search ….just check correct syntax from docs. Thera are also defined output format etc. you should remember that some events could come later, so if you just export all events from last 5min you probably miss some….