All Apps and Add-ons

Moving eStreamer logs to different location

gaddams
Explorer

The current partition where the flow logs from Sourcefire are getting collected is full and I want to change the location. How can I change estreamer.log to different location on my splunk server?

Thanks
Swetha

0 Karma

cgrady_sf
Path Finder

You could create a log directory symlink to another mounted volume. That would be my best recommendation.

omgwut56k
Path Finder

It's been a while since this post, does the current remove logs? or do I need to find another solution to keep this from filling up our heavy forwarder?

0 Karma

GlennHofmann
Engager

Realizing this was posted many moons ago, here is the solution I found for telling eStreamer where to put it's logs. If the app ever gets upgraded, it will be overwritten, but I don't think that is going to happen anytime soon. In the eStreamer/bin directory you can edit client_check.py and change the log_file directive as shown below. Works like a charm. And add the find command to your cron.daily to point to the directory you have moved your logs to and you are good to go.

# Set the rest of the paths relative to the splunk_path
app_path     = os.path.join(splunk_path, 'etc', 'apps', 'eStreamer')
app_bin_path = os.path.join(app_path, 'bin')
config_file  = os.path.join(app_path, 'local', 'estreamer.conf')
log_file     = ('/var/log/syslog-ng/estreamer/estreamer.log')
pid_file     = os.path.join(app_bin_path, 'estreamer_client.pid')
script_file  = os.path.join(app_bin_path, 'estreamer_client.pl')
0 Karma

cgrady_sf
Path Finder

At the moment, they do not get deleted by the app -- a current shortcoming. You can setup a cron job on the Splunk server to remove files older than, for example, 5 days with the following command:

find /path/to/files/* -mtime +5 -exec rm {} \;

Note, you will need to change the path, and I would recommend testing the command prior to placing it into a cron job entry.

ryanoconnor
Builder

This method worked for me as well. I used the following which searched for files older than an hour:

find /opt/splunk/etc/apps/eStreamer/log -mmin +59 -type f -exec rm “{}” \;

Like @cgrady_sf stated, you may not want to start out with executing the rm command. You could do something to simply move the files at first. The following will create a directory called "old" and move the files in there.

mkdir /opt/splunk/etc/apps/eStreamer/log/old

find /opt/splunk/etc/apps/eStreamer/log -mmin +59 -type f -exec mv “{}” /opt/splunk/etc/apps/eStreamer/log/old/ \;

0 Karma

gaddams
Explorer

Thanks.

What is the retention period of these logs given that they are indexed by splunk indexer and then I don't think we need these log files?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...