Hi All,
I have encountered a miss match between the license EPD of the ES and the | tstats count command of the same index.
FYI the EPD is based on the _internal metrics.log data.
The number of events i can see with the tstats command is much lower than the number in the _internal metrics.log .
Can anyone please help me understand the reason for this ?
Thanks !
Also consider that timestamps and index time can factor into this calculation as well.
If you add a new input on to a forwarder, you could potentially ingest data today that is more than a day old.
The license counts against data ingested in the current day regardless of whether the event timestamp is in the past or the future.
tstats will use what Splunk considers the event time (_time) in count not the index time (_indextime)
Actually, i didn't think about this option, but the gap just seems to be too high.
Do you maybe know of a way to evaluate how much license will be saved by excluding specific windows Event ID ?
If you're looking estimate license volume for specific events, you can use the length of an event in bytes and then convert to MB/GB.
index=your index here your search constraints here
| eval bytes=len(_raw)
| timechart sum(bytes) as bytes
| eval KB=bytes/1024
| eval MB=bytes/1024/1024
| eval GB=bytes/1024/1024/1024
The problem is i want to check the size of a lot of Event IDs so it will be a bit of pain to do it this way.
And there is still the issue with the massive gap between the EPD and the tstats.
I am still going to mark this as the answer, because i don't think i will have any better answers.
Try something like this for yesterday
index="your_index_here" EventCode="*"
| eval bytes=len(_raw)
| bucket _time span=1d
| stats count sum(bytes) as bytes by _time sourcetype EventCode
| eval KB=round(bytes/1024,0)
| eval MB=round(bytes/1024/1024,0)
| eval GB=round(bytes/1024/1024/1024,2)
| table _time sourcetype EventCode count bytes KB MB GB
| sort -bytes
You can also filter for specific codes like this using the IN command with a list of codes or patterns
index="your_index_here" EventCode="" EventCode IN(515,462*,40962)
| eval bytes=len(_raw)
| bucket _time span=1d
| stats count sum(bytes) as bytes by _time sourcetype EventCode
| eval KB=round(bytes/1024,0)
| eval MB=round(bytes/1024/1024,0)
| eval GB=round(bytes/1024/1024/1024,2)
| table _time sourcetype EventCode count bytes KB MB GB
| sort -bytes
I haven't tried getting an exact match, however you may have to search all time and use _indextime as constraint. _indextime being the time the indexer see the event. That's the time that should match with the ingest volume.
So i ran this search:
index="my index" _indextime>=-25h
| stats count
and i could see that the count is like the count in the tstats, by the _time field, and not like what there is in the indexing audit of splunk audit
So i wonder what is the reason for the gap if it is not because of the _indextime.
So i will unaccept the answer for now in order not to misguide anyone.
the EPD is based on the _internal metrics.log data.
Indexing Audit
The Indexing Audit dashboard is designed to help administrators estimate the volume of event data being indexed by Splunk Enterprise. The dashboard displays use EPD (Events Per Day) as a metric to track the event volume per index, and the rate of change in the total event counts per index over time. The EPD applies only to event counts, and is unrelated to the Volume Per Day metric used for licensing.
c.f https://docs.splunk.com/Documentation/ES/6.1.0/User/Audit
Hi,
Thanks for the reply !
This still doesn't explain why the actual count of events in specific day is different than the metrics.
as fotr the indexing audit, i am not sure i understand why did you paste the explanation of the indexing audit.