Splunk Enterprise Security

EPD compare to | tstats count

Communicator

Hi All,

I have encountered a miss match between the license EPD of the ES and the | tstats count command of the same index.

FYI the EPD is based on the _internal metrics.log data.

The number of events i can see with the tstats command is much lower than the number in the _internal metrics.log .

Can anyone please help me understand the reason for this ?

Thanks !

0 Karma

Motivator

Also consider that timestamps and index time can factor into this calculation as well.
If you add a new input on to a forwarder, you could potentially ingest data today that is more than a day old.
The license counts against data ingested in the current day regardless of whether the event timestamp is in the past or the future.
tstats will use what Splunk considers the event time (time) in count not the index time (indextime)

0 Karma

Communicator

Actually, i didn't think about this option, but the gap just seems to be too high.

Do you maybe know of a way to evaluate how much license will be saved by excluding specific windows Event ID ?

0 Karma

Motivator

If you're looking estimate license volume for specific events, you can use the length of an event in bytes and then convert to MB/GB.

index=your index here your search constraints here
| eval bytes=len(_raw) 
| timechart sum(bytes) as bytes 
| eval KB=bytes/1024
| eval MB=bytes/1024/1024
| eval GB=bytes/1024/1024/1024
0 Karma

Communicator

The problem is i want to check the size of a lot of Event IDs so it will be a bit of pain to do it this way.
And there is still the issue with the massive gap between the EPD and the tstats.

I am still going to mark this as the answer, because i don't think i will have any better answers.

0 Karma

Motivator

Try something like this for yesterday

index="yourindexhere" EventCode="*"
| eval bytes=len(_raw)
| bucket _time span=1d
| stats count sum(bytes) as bytes by _time sourcetype EventCode
| eval KB=round(bytes/1024,0)
| eval MB=round(bytes/1024/1024,0)
| eval GB=round(bytes/1024/1024/1024,2)
| table _time sourcetype EventCode count bytes KB MB GB
| sort -bytes

You can also filter for specific codes like this using the IN command with a list of codes or patterns

index="yourindexhere" EventCode="" EventCode IN(515,462*,40962)
| eval bytes=len(_raw)
| bucket _time span=1d
| stats count sum(bytes) as bytes by _time sourcetype EventCode
| eval KB=round(bytes/1024,0)
| eval MB=round(bytes/1024/1024,0)
| eval GB=round(bytes/1024/1024/1024,2)
| table _time sourcetype EventCode count bytes KB MB GB
| sort -bytes

I haven't tried getting an exact match, however you may have to search all time and use _indextime as constraint. _indextime being the time the indexer see the event. That's the time that should match with the ingest volume.

0 Karma

Communicator

So i ran this search:

index="my index" _indextime>=-25h
| stats count

and i could see that the count is like the count in the tstats, by the _time field, and not like what there is in the indexing audit of splunk audit

So i wonder what is the reason for the gap if it is not because of the _indextime.

So i will unaccept the answer for now in order not to misguide anyone.

0 Karma

Ultra Champion

HowSplunklicensingworks

the EPD is based on the _internal metrics.log data.

Indexing Audit
The Indexing Audit dashboard is designed to help administrators estimate the volume of event data being     indexed by Splunk Enterprise. The dashboard displays use EPD (Events Per Day) as a metric to track the event     volume per index, and the rate of change in the total event counts per index over time. The EPD applies only     to event counts, and is unrelated to the Volume Per Day metric used for licensing.

c.f https://docs.splunk.com/Documentation/ES/6.1.0/User/Audit

0 Karma

Communicator

Hi,
Thanks for the reply !

This still doesn't explain why the actual count of events in specific day is different than the metrics.

as fotr the indexing audit, i am not sure i understand why did you paste the explanation of the indexing audit.

0 Karma