All Apps and Add-ons

Cisco eStreamer for Splunk: How to configure eStream flows retention?

DomenicoFumarol
Explorer

Hi everybody,
we have the Cisco eStreamer for Splunk in our environment with the flows enabled that unfortunately are filling the disk space available very quickly.
Can we remove, via job script, older (more than a week for instance) estream flows or do we need to store them?

We can't find any field on the eStreamer App setting page related to flow retention.

Thank you.

0 Karma

douglashurd
Builder

What disk space is being filled?

The Flow data is an all or nothing proposition unless you use the Correlation engine to send you flow data when the flow meets some sort of criteria.

0 Karma

douglashurd
Builder

According to one of the experts here the best and easiest way to fix would be to create a symlink from one directory to the other :

ln -s /opt/splunk/splunk/etc/logs /splunkfs/estreamer/logs

What the command above is doing is symbolically linking the first directory to the second so the system will think its writing to the former but really writing to the desired latter.

0 Karma

DomenicoFumarol
Explorer

Thank you for your reply, but I think the symlink will only store the log files in another location that will have a maximum capacity too.
My question was instead around the need of having these estream flows or perhaps we can remove them, without losing useful data, after a period of time (i.e. a month).

If we setup a cronjob that will remove these estream flows older than 2 weeks, are we losing the capability of searching events older than 2 weeks or perhaps those are saved somewhere else?

0 Karma

afamuyiwa
Engager

I'm running into the same issue as well. I have estreamer setup on a heavy forwarder with limited storage space. Does anyone have recommendation how much disc storage is required for the estreamer.log for at least 7 days of retention to start off with?

0 Karma

DomenicoFumarol
Explorer

The path $SPLUNK_HOME/etc/apps/eStreamer/log on the indexer, configured as client eStreamer in FireSIGHT, containing logs in the form estreamer.log.1481198246.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...