Moving eStreamer logs to different location

[gaddams]

The current partition where the flow logs from Sourcefire are getting collected is full and I want to change the location. How can I change estreamer.log to different location on my splunk server?

Thanks
Swetha

[cgrady_sf]

You could create a log directory symlink to another mounted volume. That would be my best recommendation.

[omgwut56k]

It's been a while since this post, does the current remove logs? or do I need to find another solution to keep this from filling up our heavy forwarder?

[GlennHofmann]

Realizing this was posted many moons ago, here is the solution I found for telling eStreamer where to put it's logs. If the app ever gets upgraded, it will be overwritten, but I don't think that is going to happen anytime soon. In the eStreamer/bin directory you can edit client_check.py and change the log_file directive as shown below. Works like a charm. And add the find command to your cron.daily to point to the directory you have moved your logs to and you are good to go.

# Set the rest of the paths relative to the splunk_path
app_path     = os.path.join(splunk_path, 'etc', 'apps', 'eStreamer')
app_bin_path = os.path.join(app_path, 'bin')
config_file  = os.path.join(app_path, 'local', 'estreamer.conf')
log_file     = ('/var/log/syslog-ng/estreamer/estreamer.log')
pid_file     = os.path.join(app_bin_path, 'estreamer_client.pid')
script_file  = os.path.join(app_bin_path, 'estreamer_client.pl')

[cgrady_sf]

At the moment, they do not get deleted by the app -- a current shortcoming. You can setup a cron job on the Splunk server to remove files older than, for example, 5 days with the following command:

find /path/to/files/* -mtime +5 -exec rm {} \;

Note, you will need to change the path, and I would recommend testing the command prior to placing it into a cron job entry.

[ryanoconnor]

This method worked for me as well. I used the following which searched for files older than an hour:

find /opt/splunk/etc/apps/eStreamer/log -mmin +59 -type f -exec rm “{}” \;

Like @cgrady_sf stated, you may not want to start out with executing the rm command. You could do something to simply move the files at first. The following will create a directory called "old" and move the files in there.

mkdir /opt/splunk/etc/apps/eStreamer/log/old

find /opt/splunk/etc/apps/eStreamer/log -mmin +59 -type f -exec mv “{}” /opt/splunk/etc/apps/eStreamer/log/old/ \;

[gaddams]

Thanks.

What is the retention period of these logs given that they are indexed by splunk indexer and then I don't think we need these log files?