What is average Internal Events count per day on average servers??

[hectorvp]

Hi Splunkers,

I've  been working over capacity planning where for estimating indexer requirement.

I'm stuck while calculating disk space.

Our indexers are supposed to only store internal logs. And internal logs are only from UFs.

Since no Splunk setup is present  at prod servers currently, I'm not able to get how many internal logs are generated per day by UF.

What I did is I spun a VM ,installed a UF and found that 288k internal events per day i.e 200 events per minute are generated.

While calculating disk space I'm considering 300 events/min i.e 200 + adding 100 events/min as a buffer.

Can anyone help me to give a  that on average how many internal events are generated by 1 UF on prod server at every day or at every minute??? 

I understand there may be many factors like no.of addons, sources and all;

just need to confirm that are prod servers event count around 300 events/minute,is the estimation in range??

In simple terms I need answer of this "query" for  prod servers to get events per minute

"tstats count where index=_internal and host ="ANY one host randomly" groupby index, _time span=60s | stats avg(count)

PS: I dont have splunk set up to check this prior deciding disk space using search head.

[somesoni2]

Average number of events per min per UF will be well within the range you selected, so with that buffer of 50%, you should be good. 

[richgalloway]

The number you have is as good as any.  As you said, there are many unknown factors.

I have to ask why (and how) you have indexers that collect only internal logs.  And if UFs are sending only internal logs, why have them?  The point of a UF is to forward data, not its own logs.

[hectorvp]

Hi @richgalloway ,

There would be other indexers who will be receiving actual logs from UF, those indexers are not going to be managed by our team.

Our team has only responsibility to manage UFs and DS for UF.

Although there are unknown factors, I atleast need to know whether eventcounts that I've estimated is in the range of prod server since we do not have approval to install UFs yet on servers.

We are implementing this for the first time and hence have no clue of ideally how many internal events get generated when agents are on prod.

Or shall I consider 500 events per minute??? To keep it at higher end or 300 events per minute are enough.

I'm calculating based events per minute bcoz metrics.log will fire logs at an interval of 30s so....