Installation

How to display license consumed by an index over 24 hour period?

Mohsin123
Path Finder

Hi,

i am trying to display top 10 license consumed by an index over 24 hours split over 2 hours each . i have a doubt :

does sum(kb) split by series(index) in thruput_group of metrics.log same as sum(size_bytes) per index using eventcount command : |eventcount summarize=false report_size=true $tokApp$ $tokIndex$

I'm confused because my total license consumed per index over 24 hours is not matching according to 2 outputs.. below is my code:

index=_internal  source=*metrics.log group=per_index_thruput earliest=-24h@h  NOT (series=_* OR series=*summary)   | timechart span=2h sum(eval(kb/1024/1024)) as License_Used_GB by series limit=10 useother=f usenull=f|sort -License_Used_GB

2nd one is :

|eventcount summarize=false report_size=true
|stats sum(size_bytes) AS size_bytes  by  index
|eval size_bytes_GB=size_bytes/1024/1024/1024
|eval  size_bytes_GB= round(size_bytes_GB,3)
|rename size_bytes_GB as "Total License occupied by an appilcation by index (Gigs)"  

Kindly help . Total license occupied should match right ? or it can not match also because i am using different sources ? please note: metrics.log results less size than eventcount

Labels (2)
0 Karma
1 Solution

inventsekar
Super Champion

please note: metrics.log results less size than eventcount -
-- how different the metrics log results and event count result?

metrics.log is measuring the thruput of data being actually being indexed by Splunk, as a measure of how well your input and indexing pipelines are performing. The metrics.log file itself is indeed indexed to the _internal index because you can run a splunk search and have it show up.

However, this data and the other data indexed by Splunk about Splunk in _internal and _introspection and a few other indexes, does not actually count toward your license. Additionally data that is indexed by Splunk out of summarization queries run against other Splunk data and written into Summary Indexes is additionally not counted toward your license, however it is possible to configure your Splunk Server(s) to have inputs of their own and pick up data that isn't about Splunk itself, thus would actually count toward your license.

To figure out actual license impact (instead of performance metrics) you'll want to look on your license master, there should be a search called the "License Usage Data Cube" which helps build breakdowns and the License Usage Report View which will let you see the actual license impact against various indexes and hosts. (You should read the documentation page because there is squashing behavior that could take place in the data sent to the license master from each indexer.

PS ... If any post helped you in any way, pls give a hi-five to the author with an upvote. if your issue got resolved, please accept the reply as solution.. thanks.

View solution in original post

0 Karma

inventsekar
Super Champion

please note: metrics.log results less size than eventcount -
-- how different the metrics log results and event count result?

metrics.log is measuring the thruput of data being actually being indexed by Splunk, as a measure of how well your input and indexing pipelines are performing. The metrics.log file itself is indeed indexed to the _internal index because you can run a splunk search and have it show up.

However, this data and the other data indexed by Splunk about Splunk in _internal and _introspection and a few other indexes, does not actually count toward your license. Additionally data that is indexed by Splunk out of summarization queries run against other Splunk data and written into Summary Indexes is additionally not counted toward your license, however it is possible to configure your Splunk Server(s) to have inputs of their own and pick up data that isn't about Splunk itself, thus would actually count toward your license.

To figure out actual license impact (instead of performance metrics) you'll want to look on your license master, there should be a search called the "License Usage Data Cube" which helps build breakdowns and the License Usage Report View which will let you see the actual license impact against various indexes and hosts. (You should read the documentation page because there is squashing behavior that could take place in the data sent to the license master from each indexer.

PS ... If any post helped you in any way, pls give a hi-five to the author with an upvote. if your issue got resolved, please accept the reply as solution.. thanks.
0 Karma

Mohsin123
Path Finder

Thanks @inventsekar .
Could you please tell me one thing ..
in my license master, i can see the license quota and license used for an env , say prod env ..
does that license include only the apps or includes all summary, internal,introspection ?

0 Karma

inventsekar
Super Champion

summary, internal,introspection - all these three are not counted toward your license, it will not be reported on the license master.
you will see only all apps indexed data

PS ... If any post helped you in any way, pls give a hi-five to the author with an upvote. if your issue got resolved, please accept the reply as solution.. thanks.
0 Karma

Mohsin123
Path Finder

thanku so so so much ! this is what i wanted ....
well, in our license manster , for our dev pool / prod poo / qa pool the license used is like very less as compared to my query for fetching license usage using eventcount/metrics.log

say, LM shows dev pool consuming 100gb whereas my dashboard in the prod search head shows 400gb ! thankyou also for giving a clear understanding on metrics.log. Noted !

0 Karma
Get Updates on the Splunk Community!

Announcing General Availability of Splunk Incident Intelligence!

Digital transformation is real! Across industries, companies big and small are going through rapid digital ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...