topic Re: Splunk HF exports logs using a CLI command as a Linux crontab in Splunk Search

Splunk HF exports logs using a CLI command as a Linux crontab?

sarvananth — Fri, 21 Jul 2023 17:53:34 GMT

I'm new to Splunk Enterprise, and my task is to forward logs from Splunk HF (AWS EC2 instance) to an AWS Cloud Watch log group.

I tried to export the logs using CLI commands and stored them on the Splunk HF server locally. Then, I used the Cloud Watch agent to send the logs to the Cloud Watch log group.

please refer the below Splunk cli command for export the logs

#./splunk search "index::***** sourcetype::linux_audit" -output rawdata -maxout 0 -max_time 5 -auth splunk:***** >> /opt/linux-Test01.log

The challenge I'm facing is that when I run the CLI command using a Linux crontab, it does not export the logs.

Are there any other solutions or guidance available to resolve this issue?

Re: Splunk HF exports logs using a CLI command as a Linux crontab

isoutamo — Fri, 21 Jul 2023 10:39:58 GMT

when you are running commands from cron, you must remember that there is no sourced environments. Usually you should use some wrapped script to:

source your user's shell environment including env vars + etc.
run your commands / scripts with full paths!

In your case this means e.g.

#!/bin/bash source ~<your user>/.bash_login source ~<your user>/.bashrc cd <where ever you thing you should be> /opt/splunk/bin/splunk .....

Of course you should add needed error checks after commands etc.

r. Ismo

Re: Splunk HF exports logs using a CLI command as a Linux crontab

sarvananth — Tue, 25 Jul 2023 09:50:37 GMT

Hi R.Ismo,

yes, it is working fine, and thank you very much for your help.
Is it possible to export the logs last 5 minutes using a CLI command?

Re: Splunk HF exports logs using a CLI command as a Linux crontab

isoutamo — Tue, 25 Jul 2023 10:35:59 GMT

Basically just …/splunk search ….just check correct syntax from docs. Thera are also defined output format etc. you should remember that some events could come later, so if you just export all events from last 5min you probably miss some….